Fibratus 0.6.0

I’m happy to announce the release of Fibratus 0.6.0. Let’s see what’s new in this version.

What’s new

High performance kernel event stream collector

The kernel event stream collector has been rewritten from ground up. It now operates without GIL (Global Interpreter Lock) limitations which makes the Python interpreter capable of executing the byte code in parallel. The result is an ultra fast kernel event processing component! The tricky part about releasing the GIL is that you can’t use any Python constructs before acquiring the GIL again. Such a simple operation as comparing strings has to be done by calling the function from the standard C library. Standard Python’s containers like list or dict had been replaced with analogous C++ vector and unordered_map containers. If you look at the source tree, you should notice the kstream directory has changed dramatically. This refactoring will make it easier adding new features to the kernel event stream collector.

Image meta information

Fibratus now supports querying the executable binary files for the PE (Portable Executable) headers. The CreateProcess kernel events can be enriched with a plethora of interesting info like binary sections, the imported symbols, file information, linkage time stamp, etc. Here is the JSON representation of the image meta information for the cmd.exe binary:

"image_meta":{  
         "internal_name":"cmd",
         "timestamp":"0x57899A99 [Sat Jul 16 02:23:21 2016 UTC]",
         "imports":{  
            "api-ms-win-core-libraryloader-l1-2-0.dll":[  
               "GetModuleFileNameW",
               "GetModuleHandleW",
               "LoadLibraryExW",
               "GetProcAddress"
            ],
            "api-ms-win-core-timezone-l1-1-0.dll":[  
               "SystemTimeToFileTime",
               "FileTimeToSystemTime"
            ],
            ...
         },
         "description":"Windows Command Processor",
         "copyright":"\\xa9 Microsoft Corporation. All rights reserved.",
         "version":"10.0.14393.0 (rs1_release.160715-1616)",
         "org":"Microsoft Corporation",
         "arch":"x86-64",
         "sections":[  
            {  
               "md5":"64a67cc3d9162fc0e8f3e342498b5521",
               "sha512":"a0f211244eb057bd57100a2bba4010a71b675f47571198004c66597f2d833086bf3aa1c3c4d97635185880f1ef42b45ce2d351f0d3f2bbe24a5b9e9841f38422",
               "sha256":"90bdb2677504f970636f313b32963a7fb6e4a95fefa19a2fae97a783ee3bd24d",
               "name":".text",
               "sha1":"59bef951b3e13ddf53778ad0eae69b32c65cebbf",
               "entropy":6.414245618720535
            },
            {  
               "md5":"1e99e85eb32e35b18a9741e2b83f75fd",
               "sha512":"ceedd390297dbcea20dd9065567fcf7138cd421007f7f57793e4f2b043c9137720d55ad7fee8ca5a687bb308759df48e001b2258578e69494a0661ee189a504c",
               "sha256":"0a17d6c0cdc1a52ea16364787979c1fc4145e82e56ae59620fe7fdc04cc7eec7",
               "name":".rdata",
            ...
         ],
         "num_sections":7
      }

Multiple output sinks

The kernel events can now be streamed to multiple output sinks like message brokers, Elasticsearch instances and of course the standard output. In the previous version of Fibratus you had to rely on filaments to transport the event payloads to remote endpoints.

Beside this relevant features, there been done a ton of code refactoring and other improvements. For example, the bootstrap stage logs a lot of useful information, like the skips being applied, the outputs and its configuration, total boot time, etc.

[2017-01-22 20:11:29.423254] INFO: Fibratus: Starting Fibratus...
[2017-01-22 20:11:29.423254] INFO: Fibratus: Enumerating system handles...
[2017-01-22 20:11:31.628521] INFO: Fibratus: 6352 handles found
[2017-01-22 20:11:31.644142] INFO: Fibratus: Adding skips for images ['svchost.exe', 'smss.exe', 'services.exe', 'taskmgr.exe', 'dwm.exe', 'vprot.exe', 'lsass.exe', 'sihost.exe', 'system']
[2017-01-22 20:11:31.659769] INFO: Fibratus: Deploying [console] output - [{'format': 'json'}]
[2017-01-22 20:11:31.659769] INFO: Fibratus: Deploying [elasticsearch] output - [{'bulk': False, 'hosts': ['localhost:9200'], 'ssl': False, 'document': 'threads', 'username': None, 'index': 'fibratus'}]
[2017-01-22 20:11:31.728789] INFO: Fibratus: Started in 0m:02s.2321

If you have any feedback or would like to contribute you can reach out via Github or drop me an email.