To make it easier to add Broccoli support to your build environment, the distribution comes with a shell script that provides the necessary compiler and linker flags (similarly to many other packages): broccoli-config. Use the --cflags option to obtain the compiler options and --libs to obtain the linker options.
If you use the autoconf/automake tools, we recommend something along the following lines for your configure script:
dnl ################################################## dnl # Check for Broccoli dnl ################################################## AC_ARG_WITH(broccoli-config, AC_HELP_STRING([--with-broccoli-config=FILE], [Use given broccoli-config]), [ brocfg="$withval" ], [ AC_PATH_GENERIC(broccoli,, brocfg="broccoli-config", AC_MSG_ERROR(Cannot find Broccoli: Is broccoli-config in path? Use more fertilizer?)) ]) broccoli_libs=`$brocfg --libs` broccoli_cflags=`$brocfg --cflags` AC_SUBST(broccoli_libs) AC_SUBST(broccoli_cflags) |
Often you will want to make existing applications Bro-aware, that is, instrument them so that they can send and receive Bro events at appropriate moments in the execution flow. This will involve modifying an existing code tree, so care needs to be taken to avoid unwanted side effects. By protecting the instrumented code with #ifdef/#endif statements you can still build the original application, using the instrumented source tree. The broccoli-config script helps you in doing so because it already adds -DBROCCOLI to the compiler flags reported when run with the --cflags option:
cpk25@localhost:/home/cpk25 > broccoli-config --cflags -I/usr/local/include -I/usr/local/include -DBROCCOLI |
So simply surround all inserted code with a preprocessor check for BROCCOLI and you will be able to build the original application as soon as BROCCOLI is not defined.
Time for some code. In the code snippets below we will introduce variables whenever context requires them and not necessarily when C requires them. The library does not require calling a global initialization function. In order to make include broccoli.h:
#ifdef BROCCOLI #include <broccoli.h> #endif |
![]() | A note on Broccoli's memory management philosophy: Broccoli generally does not release objects you allocate. The approach taken is "you clean up what you allocate." |
You need to obtain a connection handle before you can receive or send events. Connection handles are pointers to BroConn structures, and kept opaque. Use bro_connect() or bro_connect_str() to obtain such a handle, depending on what parameters are more convenient for you: the former accepts the IP address and port number as separate numerical arguments, the latter uses a single string to encode both, in "hostname:port" format. bro_disconnect() terminates a connection and releases all resources associated with it. You can create as many connections as you like, to one or more peers. You can obtain the file descriptor of a connection using bro_conn_get_fd().
char host_str = "bro.yourcompany.com"; int port = 1234; struct hostent *host; BroConn *bc; if (! (host = gethostbyname(host_str)) || ! (host->h_addr_list[0])) { /* Error handling -- could not resolve host */ } /* Connect to Bro */ if (! (bc = bro_connect((struct in_addr*) host->h_addr_list[0], htons(port)))) { /* Error handling - could not connect to Bro */ } /* Send and receive events ... */ /* Disconnect from Bro */ bro_disconnect(bc); |
Or simply use the string-based version:
char host_str = "bro.yourcompany.com:1234"; BroConn *bc; /* Connect to Bro */ if (! (bc = bro_connect_str(host_str))) { /* Error handling - could not connect to Bro */ } |
In order to send an event to the remote Bro agent, you first create an empty event structure with the name of the event, then add parameters to pass to the event handler at the remote agent, and then send off the event. You need to make sure the remote Bro is interested in receiving that type of event. We'll talk more about this in a moment.
Let's assume we want to request a report of all connections a remote Bro currently keeps state for that match a given destination port and host name and that have amassed more than a certain number of bytes. The idea is to send an event to the remote Bro that contains the query, identifyable through a request ID, and have the remote Bro answer us with remote_conn events containing the information we asked for. The definition of our requesting event could look as follows in the Bro policy:
event report_conns(req_id: int, dest_host: string, dest_port: port, min_size: count); |
First, create a new event:
BroEvent *ev; if (! (ev = bro_event_new("report_conns"))) { /* Error handling - could not allocate new event. */ } |
Now we need to add parameters to the event. The sequence and types must match the event handler declaration — check the Bro policy to make sure they match. The function to use for adding parameter values is bro_event_add_val() All values are passed as a pointer arguments and are copied internally, so the object you're pointing to stays unmodified at all times. You clean up what you allocate. In order to indicate the type of the value passed into the function, you need to pass a numerical type identifier along as well. Table 1 lists the value types that Broccoli supports along with the type identifier and data structures to point to.
Table 1. Types, type tags, and data structures for event parameters in Broccoli
Type | Type Tag | Data structure |
---|---|---|
Boolean | BRO_TYPE_BOOL | int |
Integer value | BRO_TYPE_INT | int |
Counter (nonnegative integers) | BRO_TYPE_COUNT | uint32 |
Floating-point number | BRO_TYPE_DOUBLE | double |
Timestamp | BRO_TYPE_TIME | double (see also bro_util_timeval_to_double() and bro_util_current_time()) |
Time interval | BRO_TYPE_INTERVAL | double |
Strings (text and binary) | BRO_TYPE_STRING | BroString (see also the family of bro_string_xxx() functions) |
Network ports | BRO_TYPE_PORT | BroPort, with the port number in host byte order |
IPv4 address | BRO_TYPE_IPADDR | uint32, in network byte order |
IPv4 network | BRO_TYPE_NET | uint32, in network byte order |
IPv4 subnet | BRO_TYPE_SUBNET | BroSubnet, with the sn_net member in network byte order |
Record | BRO_TYPE_RECORD | BroRecord (see also the family of bro_record_xxx() functions and the explanation below) |
Knowing these, we can now compose a request_connections event:
BroString dest_host; BroPort dest_port; uint32 min_size; int req_id = 0; bro_event_add_val(ev, BRO_TYPE_INT, &req_id); req_id++; bro_string_set(&dest_host, "desthost.destdomain.com"); bro_event_add_val(ev, BRO_TYPE_STRING, &dest_host); bro_string_cleanup(&dest_host); dest_port.dst_port = 80; dest_port.dst_proto = IPPROTO_TCP; bro_event_add_val(ev, BRO_TYPE_PORT, &dest_port); min_size = 1000; bro_event_add_val(ev, BRO_TYPE_COUNT, &min_size); |
All that's left to do now is to send off the event. For this, use bro_event_send() and pass it the connection handle and the event. The function returns TRUE when the event could be sent right away, and FALSE when this was not possible. This does not mean that there was an error — likely the connection was just not ready to send the event at this point. In the latter case, the event is queued internally and sent at a later point. Whenever you call bro_event_send(), Broccoli attempts to send as much of an existing event queue as possible. Again, the event is copied internally to make it easier for you to send the same event repeatedly. You clean up what you allocate.
bro_event_send(bc, ev); bro_event_free(ev); |
Two other functions may be useful to you: bro_event_queue_empty() tells you whether there currently are queued events, and bro_event_queue_flush() attempts to flush the current event queue and returns the number of events that do remain in the queue after the flush. Note: you do not normally need to call this function, queue flushing is attempted every time you send an event.
Receiving events is a little more work because you need to
let the remote Bro agent know that you would like to receive those events,
tell Broccoli what to do with the various arriving events,
and when instrumenting an existing application, find a spot in the code path suitable for extracting and processing arriving events.
When Broccoli receives an event, it tries to dispatch the event through a callback registry. Any callbacks that are registered for the arrived event's name are invoked with the parameters shipped with the event. In order to register a callback, use bro_event_registry_add() and pass it the connection handle, the name of the event for which you register the callback, and the callback itself that matches the signature of the BroEventFunc type. The callback's type is defined rather generically as follows:
typedef void (*BroEventFunc) (BroConn *bc, ...); |
As you can see, all it requires is a connection handle as its first argument. Broccoli will pass the connection handle of the connection on which the event arrived through to the callback. BroEventFuncs are variadic, because each callback you provide is directly invoked with pointers to the parameters of the event, in a useful format. All you need to know is what type to point to in order to receive the parameters in the right layout. Refer to Table 1 again for a summary of those types. Continuing our example, we will want to process the connection reports that contain the responses to our report_conns event. Let's assume those look as follows:
event remote_conn(req_id: int, conn: connection); |
The reply events contain the request ID so we can associate requests with replys, and a connection record (defined in bro.init in Bro. (It'd be nicer to report all replies in a single event but we'll ignore that for now.) For this event, our callback would look like this:
void remote_conn_cb(BroConn *bc, int *req_id, BroRecord *conn); |
Now we register the callback using bro_event_registry_add():
bro_event_registry_add(bc, "remote_conn", remote_conn_cb); |
If you have multiple events you are interested in, register each one in the same way. At this point, Broccoli knows how to handle these events when they arrive. What's left to do is to let the remote Bro agent know that you would like to receive the events for which you registered. To do so, call bro_event_registry_request():
bro_event_registry_request(bc); |
Broccoli also supports unrequesting events, however support for dynamic event requesting/unrequesting is not implemented in Bro yet.
You will often find that you would like to connect data with a BroConn. Broccoli provides an API that lets you associate data items with a connection handle through a string-based key–value registry. The functions of interest are bro_conn_data_set(), bro_conn_data_get(), and bro_conn_data_del(). You need to provide a string identifier for a data item and can then use that string to register, look up, and remove the associated data item. Note that there is currently no mechanism to trigger a destructor function for registered data items when the Bro connection is terminated. You therefore need to make sure that all data items that you do not have pointers to via some other means are properly released before calling bro_disconnect().
Imagine you have instrumented the mother of all server applications. Building it takes forever, and every now and then you need to change some of the parameters that your Broccoli code uses, such as the host names of the Bro agents to talk to. To allow you to do this quickly, Broccoli comes with support for configuration files. All you need to do is change the settings in the file and restart the application (we're considering adding support for volatile configuration items that are read from the file every time they are requested).
Configuration is done in single configuration file per installation. You can obtain the location of this config file by running broccoli-config --config. In the configuration file, a "#" anywhere starts a comment that runs to the end of the line. Configuration items are specified as key-value pairs, as can be seen in this default file:
# This is the Broccoli system-wide configuration file. # # Entries are of the form <identifier> <value>, where the identifier # is a sequence of letters, and value can be a string (including # whitespace), and floating point or integer numbers. Comments start # with a "#" and go to the end of the line. For boolean values, you # may also use "yes", "on", "true", "no", "off", or "false". # Strings may contain whitespace, but need to be surrounded by # double quotes '"'. # # Examples: # Foo/PeerName mybro.securesite.com Foo/PortNum 123 Bar/SomeFloat 1.23443543 Bar/SomeLongStr "Hello World" |
You can name identifiers any way you like, but to keep things organized it is recommended to keep a namespace hierarchy similar to the file system. In the code, you can query configuration items using bro_conf_get_str(), bro_conf_get_int(), and bro_conf_get_dbl().
Broccoli provides an API for dynamically allocatable, growable, shrinkable, and consumable buffers with BroBufs. You may or may not find this useful — Broccoli mainly provides this feature in broccoli.h because these buffers are used internally anyway and because they are typical case of something that people implement themselves over and over again, for example to collect a set of data before sending it through a file descriptor, etc.
The buffers work as follows. The structure implementing a buffer is called BroBuf. BroBufs are initialized to a default size when created via bro_buf_new(), and released using bro_buf_free(). Each BroBuf has a content pointer that points to an arbitrary location between the start of the buffer and the first byte after the last byte currently used in the buffer (see buf_off in the illustration below). The content pointer can seek to arbitrary locations, and data can be copied from and into the buffer, adjusting the content pointer accordingly. You can repeatedly append data to end of the buffer's used contents using bro_buf_append().
<---------------- allocated buffer space ------------> <======== used buffer space ========> ^ ^ ^ ^ | | | | `buf `buf_ptr `buf_off `buf_len |
Have a look at the following functions for the details: bro_buf_new(), bro_buf_free(), bro_buf_append(), bro_buf_get(), bro_buf_get_end(), bro_buf_get_size(), bro_buf_get_used_size(), bro_buf_ptr_get(), bro_buf_ptr_tell(), bro_buf_ptr_seek(), bro_buf_ptr_check(), and bro_buf_ptr_read().
The Broccoli distribution comes with a few small test programs, located in the test/ directory of the tree. The most notable one is broping [1], a mini-version of ping. It sends "ping" events to a remote Bro agent, expecting "pong" events in return. The remote Bro agent needs to run the following policy for this to work (included in the distribution):
@load listen-clear global ping_log: file &redef; redef ping_log = open_log_file("ping"); event remote_connection_established(ip: addr, p: port) { request_remote_events(ip, p, /ping/); } event ping(src_time: time) { event pong(src_time, current_time()); } event pong(src_time: time, dst_time: time) { print ping_log, fmt("ping received, %f at src, %f at dest, one-way: %f", src_time, dst_time, dst_time-src_time); } |
The @load statement is necessary to load the listen-clear policy, which causes the remote agent to listen and respond to clear-text connection attempts. Have a look at the policy — you can configure what IP addresses and port number the agent will be listening on. In the next two lines, a log file is created to which a line will be written whenever a pong event is triggered in response to a received ping event. Next is a remote_connection_established event handler. This handler gets invoked whenever a Bro agent (broping in this case) connects to the agent. Here, we request delivery of all events whose names match the /ping/ regular expression, thus just the ping event. Next is the definition of the ping event handler. You can see that ping events contain a timestamp (the time at which broping sent out the event), and that it triggers a pong event with the same timestamp and the current time. What to do when said pong event is triggered comes next: a line is dumped to the log file.
So how do the pong events ever end up at the broping client? broping simply asks for delivery of all pong events from the remote agent, using the same mechanism as shown above, but hard-coded in C. After that, every triggered pong event is sent off to broping, along with the timestamp of the creation of the ping event and the timestamp of the creation of the pong event.
When running broping, you'll see something like this:
cpk25@localhost:/home/cpk25/devel/broccoli > ./test/broping pong event from 127.0.0.1: seq=1, time=0.004700/1.010303 s pong event from 127.0.0.1: seq=2, time=0.053777/1.010266 s pong event from 127.0.0.1: seq=3, time=0.006435/1.010284 s pong event from 127.0.0.1: seq=4, time=0.020278/1.010319 s pong event from 127.0.0.1: seq=5, time=0.004563/1.010187 s pong event from 127.0.0.1: seq=6, time=0.005685/1.010393 s |
[1] | Pronunciation is said to be somewhere on the continuum between "brooping" and "burping". |