
Brim: Open-source desktop app to analyze large pcaps through the lens of Zeek - siskojr
https://github.com/brimsec/brim
======
tannerbrockwell
When you get to the level of analyzing packet data, the first obstacle is
actually capturing the conversations that you may find relevant. While a
capture filter can help on the acquisition, it is possible that an error in
the filter could exclude relevant hosts, or ports, or protocols. In general
wide open captures will then lead to very large capture files.

Wireshark performs poorly with opening such a large trace. Since Engineer's
are costing well north of $150 per hour, having them twiddle their thumbs
waiting is not economical or efficient.

Enabling a survey and filtering of the data as soon as possible from the pcap
file will allow interactive examination. With Brim that identified data can be
exported as a filtered, and thus much smaller file. This is similar to what
tools such as Riverbed's Packet Analyzer do. PA is a .net front end to tshark
type filtering and reporting. It ships with a number of useful filters to
survey for common problems.[1]

Brim and Zeek appear to fill the need between Packet Analyzer on one hand and
a regular Wireshark workflow on the other.

[1]:
[https://www.riverbed.com/products/steelcentral/steelcentral-...](https://www.riverbed.com/products/steelcentral/steelcentral-
packet-analyzer.html)

------
ghostpepper
Can anyone explain how this is better than analyzing wireshark captures within
wireshark?

What is zeek and can I use it to analyze wireshark captures?

~~~
tptacek
Wireshark dissects protocols and gives debugging traces. Zeek (FKA "Bro") is
an intrusion detection system that analyzes debugging traces (which it
generates itself, not through Wireshark dissectors) to generate usefully
semantic data, ranging from "which hosts are fetching this URL" to "where are
there signs of this vulnerability being exploited".

Both Wireshark and Zeek use libpcap, which is the de facto standard for raw
packet captures, so yes, the same data you'd be feeding to Wireshark to
browse, you can also feed to Zeek to analyze.

~~~
e12e
Hadn't heard of zeek before - actually find this submarine article useful - it
hints better at where and how, than the official zeek documentation. I still
don't know what storage space I should budget for a gbps-month of zeek
monitoring logs, though (I understand they're typically filtered... But also
text?).

[https://www.csoonline.com/article/3313050/zeek-a-free-
powerf...](https://www.csoonline.com/article/3313050/zeek-a-free-powerful-way-
to-monitor-networks-detect-threats.html)

As to the where - I guess I need to find some port that can give me a mirror
of traffic.... But not that easy with heterogeneous lan+cloud+SaaS setup to
figure out where/how to source "all" network traffic :/

------
WrtCdEvrydy
This looks pretty interesting, I wonder how it would do with the CTF dump from
Defcon 19 and 20.

------
rixed
Since everyone who are into pcap analyses are looking, I cannot help but drop
references to two other open source DPI products that are worth considering
for some use cases.

The first one I authored myself many years ago and has been dormant for a long
time although I'd certainly enjoy working on it again if an opportunity
arises: [https://github.com/rixed/junkie](https://github.com/rixed/junkie)

The other one seems to stand in the same niche of highly specific, optimised
programmable DPI, but I'm obviously much less familiar with it:
[https://github.com/DanieleDeSensi/peafowl](https://github.com/DanieleDeSensi/peafowl)

------
arminiusreturns
I've been meaning to dig into bpf as a packet tracing/sniffing tool for the
same reasons... pcap files can get enormous quickly enough to make space and
parsing a pita. Cool work.

~~~
tannerbrockwell
Agreed. using bpf can eliminate some of the requirements for packet captures.
Computing the metrics on the host, also eliminates the network traffic of
exporting logs or pcaps to for instance splunk. [1]

Brendan Gregg wrote a book that digs really deep into the bpf system. [2] I
recommend you check it out if you need to do real work fast.

[1]:
[https://docs.splunk.com/Documentation/StreamApp/7.2.0/Deploy...](https://docs.splunk.com/Documentation/StreamApp/7.2.0/DeployStreamApp/UseStreamtoparsePCAPfiles)
[2]: [http://www.brendangregg.com/bpf-performance-tools-
book.html](http://www.brendangregg.com/bpf-performance-tools-book.html)

~~~
tptacek
BPF for packet capture predates BPF for system instrumentation, and libpcap
itself is under the hood an elaborate, optimizing BPF compiler.

~~~
tannerbrockwell
Current convention is to refer to eBPF as BPF, and where referring to the
Original BPF as cBPF. This is why Brendan's book is "BPF Performance Tools"

"Then in 2013, Alexei Starovoitov completely reshaped it, started to add new
functionalities and to improve the performances of BPF. This new version is
designated as eBPF (for “extended BPF”), while the former becomes cBPF
(“classic” BPF). New features such as maps and tail calls appeared. The JIT
machines were rewritten. The new language is even closer to native machine
language than cBPF was. And also, new attach points in the kernel have been
created." [1]

[1]: [https://qmonnet.github.io/whirl-offload/2016/09/01/dive-
into...](https://qmonnet.github.io/whirl-offload/2016/09/01/dive-into-bpf/)

------
GekkePrutser
Nice, I like this!!

It should be standard in Kali :)

------
dang
We changed the URL from
[https://github.com/brimsec/brim](https://github.com/brimsec/brim) to the home
page it points to.

Edit: then we changed it back because the Github page seems to have a shorter
time-to-understanding. Preferences?

~~~
Etheryte
Definitely prefer the Github link, the short intro video gives you a very
quick understanding of a good part of the value proposition, what this tool
does in general, etc.

