Log TCP Trace

This log has been collected using the Tstat probe (running Tstat-3.0) installed at the egress link of our campus network during November 2015. The trace contains all TCP flows observed considering a one-hour long time interval. The trace have been collected and made shareable with the support of the mPlane project. In order to respect privacy, IP addresses have been anonymized, and any sensitive information has been removed.

The file has been compressed using xz, which reduced its size to 339MB from the original 2.1GB. The file can be easily accessed by using xzcat and using a pipe to input the uncompressed data to the post-processing script/command (e.g., "xzcat log_tcp_complete.xz | head -n 10").

Examples of post-process commands:

1) Count the number of flows whose FQDN field contains "facebook":

  xzcat log_tcp_complete.xz | awk '$127 ~ "facebook" {cnt++} END {print cnt}'

2) Count the number of flows carrying HTTP or HTTPS traffic, headed to "facebook" and generated by internal clients:

  xzcat log_tcp_complete.xz | awk '$38 == 1 && $127 ~ "facebook" && ($42 == 1 || $42 == 8192) {cnt++} END {print cnt}'

3) Count the number of server IP addresses associated to "facebook" flows:

  xzcat log_tcp_complete.xz | awk '$127 ~ "facebook" {sips[$15] = 1} END {print length(sips)}'

    or

  xzcat log_tcp_complete.xz | awk '$127 ~ "facebook" {print $15}' | sort | uniq | wc -l

Detailed information about the meaning assigned to each column can be found at this link. Examples of scripts to post-process this kind of traces can be found at this link