Tuesday, July 14, 2015

Extracting Traffic from Rolling Capture Files

Every so often I need to extract a subset of traffic from a set of rolling timestamped pcap files. One common place I do this is with Security Onion; one of the great features of SO is its full-packet-capture feature: you can easily pivot from Snort, Suricata, or Bro logs to a full packet capture view, or download the associated pcap file.

But what if you don't have an associated alert or Bro log entry? Or if you're doing pcap on some system that's not as user-friendly as Security Onion, but nonetheless supports rolling captures?

The way I usually do this is with find and xargs. Here's an example of my most common workflow, using timestamps as the filtering criteria for find:

> find . -newerct "16:07" ! -newerct "16:10" | xargs -I {} tcpdump -r {} -w /tmp/{} host 8.8.8.8
> cd /tmp
> mergecap -w merged.pcap *.pcap

Translated:
  1. Find all files in the current directory created after 16:07 but not created after 16:10. This requires GNU find 4.3.3 or later. It supports many different time and date formats.
  2. Using xargs, filter each file with the "host 8.8.8.8" BPF expression and write it to /tmp with the same filename.
  3. Merge all the .pcap files in /tmp into merged.pcap.
You can easily modify this workflow to fit other use cases.

13 comments:

theFlakes said...

Caught this on the SO mailing list. I had a shell script to automate some of this but the pulling of the information from the multiple snort files wasn't as nice as yours. thanks

https://github.com/theflakes/security-onion-misc-scripts/blob/master/getPcap.sh

Greg said...

Thanks for sharing! I've been trying to figure out how to do this!

why college should not be free said...

I don't know how to extract traffic from rolling captures. I have read many blogs about this but I didn't understood well. After reading this post, I easily understood it very well.

Unknown said...

very nice interview questions
vlsi interview questions
extjs interview questions
laravel interview questions
sap bi/bw interview questions
pcb interview questions
unix shell scripting interview questions

Unknown said...

really awesome blog
hr interview questions
hibernate interview questions
selenium interview questions
c interview questions
c++ interview questions
linux interview questions

Unknown said...

nice blog
android training in bangalore
ios training in bangalore

Unknown said...

Hi There,


Great post. Well though out. This piece reminds me when I was starting out Loopback Mountain Key after graduating from college.

I am working LDPC encoding and decoding for mini project work. With my effort I have completed LDPC encoding but struggling with LDPC decoding. I need to complete this project complete this project within short period so please help me.

I don`t know how to start decoding algorithm in C language but i do have algorithm. So, can anyone help me in writing C language for decoding procedure.

Thank you very much and will look for more postings from you.


Thank you,
John

viswanath said...

AWS Training in Bangalore - Live Online & Classroom
myTectra Amazon Web Services (AWS) certification training helps you to gain real time hands on experience on AWS. myTectra offers AWS training in Bangalore using classroom and AWS Online Training globally. AWS Training at myTectra delivered by the experienced professional who has atleast 4 years of relavent AWS experince and overall 8-15 years of IT experience. myTectra Offers AWS Training since 2013 and retained the positions of Top AWS Training Company in Bangalore and India.

IOT Training in Bangalore - Live Online & Classroom
IOT Training course observes iot as the platform for networking of different devices on the internet and their inter related communication. Reading data through the sensors and processing it with applications sitting in the cloud and thereafter passing the processed data to generate different kind of output is the motive of the complete curricula. Students are made to understand the type of input devices and communications among the devices in a wireless media.

asha said...

Nice Post
best training institute for hadoop in Bangalore

best big data hadoop training in Bangalroe

hadoop training in bangalore

hadoop training institutes in bangalore

hadoop course in bangalore

Dharani M said...

Good information
best android training center in Marathahalli

best android development institute in Marathahalli

android training institutes in Marathahalli

ios training in Marathahalli

android training in Marathahalli

mobile app development training in Marathahalli

https://www.londongirls.co.uk/euston-escorts-area.html said...

I am really glad I’ve found this information.A good web site with interesting contentYour blog have nice informationI am very grateful for sharing such information.

Anonymous said...

More interesting info you can 
view here



Repute Digital Business Agency said...

Thanks for this blog. Keep posting more such blogs!

Top 10 Content Marketing Companies in Chennai-2023 Review
Top 10 PPC Companies in Coimbatore – 2023 Review
Top 10 Content Marketing Companies in Coimbatore – 2023 Review
Top 10 Search Engine Marketing Companies in Chennai- 2023 Review