4 min read

Last week, a new traffic analysis framework, Netcap (NETwork CAPture) was released. It converts a stream of network packets into accessible type-safe structured data for representing specific protocols or custom abstractions.

This project was implemented in Go programming language that provides a garbage collected memory safe runtime as parsing of untrusted input could be dangerous. It was developed for a series of experiments like filtering, dataset labeling, encoding, error logging, etc in the thesis: Implementation and evaluation of secure and scalable anomaly-based network intrusion detection.

The Netcap project won the second place at Kaspersky Labs SecurIT Cup 2018 in Budapest.

Why was Netcap introduced?

Corporate communication networks are attacked frequently with previously unseen malware or insider threats, which makes defense mechanisms such as anomaly-based intrusion detection systems necessary for detecting security incidents. The signature-based and anomaly detection strategies rely on features extracted from the network traffic that requires secure and extensible collection strategies.

The solutions that are available are written in low-level system programming languages that require manual memory management and suffer from vulnerabilities that allow a remote attacker to disable the network monitor. Others lack in terms of flexibility and data availability. To tackle these problems and ease future experiments with anomaly-based detection techniques, Netcap was released.

Netcap uses Google’s protocol buffers for encoding its output which helps in accessing it across a wide range of programming languages. The output can also be emitted as comma separated values, which is a common input format for data analysis tools and systems. Netcap is extensible and it provides multiple ways of adding support for new protocols and also implements the parsing logic in a memory safe way.

It provides high dimensional data of observed traffic and allows the researcher to focus on new approaches for detecting malicious behavior in network environments, instead of opting data collection mechanisms and post-processing steps. It features a concurrent design that makes use of multi-core architectures. This command-line tool focuses on usability and readability and displays progress when processing packets.

Why Go?

Go, commonly referred to as Golang, is a statically typed programming language which was released by Google in 2009. Netcap opted Go as its syntax is similar to the C programming language and also has a lot of adopted ideas from other languages, such as Python and Erlang. It is commonly used for network programming and backend implementation. With Go Netcap can compile faster and generate statically linked binaries, easily.

Goroutine, an asynchronous process is multiplexed onto threads of the OS as required. In case a goroutine blocks, the corresponding OS thread blocks as well, but the other goroutines aren’t affected. So, this proves to be helpful in Netcap as it doesn’t disturb the functioning. Also, Goroutines are less expensive as compared to a thread and allocate resources dynamically as needed. Since, Go offers channels as a lightweight way to communicate between goroutines, the synchronization and messaging process gets easier in Netcap.

Design Goals of Netcap

  • Netcap provides memory safety when parsing untrusted input.
  • It features ease of extension.
  • The output format is interoperable with many different programming languages.
  • It features concurrent design.
  • It comes with output with small storage footprint on disk.
  • It provides with maximum data availability.
  • It allows implementation of custom abstractions
  • It comes with a rich platform and architecture support

Future Scope

Future development on Netcap will focus on increasing the unit test coverage and performance critical operations. The output of Netcap will be compared to other tools, to ensure no data is missed or misinterpreted. Netcap will be extended in future with functionalities like support for extracted features.

This framework might be used for experiments on datasets for accurate predictions on network data. Encoding feature vectors could also be implemented as part of the Netcap framework. An interface for adding additional application layer encoders can be added in future. Netcap will be evaluated for monitoring industrial control systems communication. The recently open sourced fingerprinting strategy for SSH handshakes (HASSH) by salesforce could prove beneficial in future.

Check the slides of this project from the presentation by Philipp Mieden (the creator of Netcap) at the Leibniz Supercomputing Centre of the Bavarian Academy of Sciences and Humanities on Researchgate.

Many users are appreciating the efforts taken for this project and eagerly awaiting for the features that might be released in the future.

But a few Hacker News users think that the functionality provided by this application is still unclear. The thesis misses a lot of points with the major one being as to how this tool is actually warranted as a whole. The question is as to how will the anomalies of this project get detected? A lot of questions are still unanswered but it would be interesting to see what Philipp comes up with next.

Read Next

Stanford researchers introduce DeepSolar, a deep learning framework that mapped every solar panel in the US

Netflix adopts Spring Boot as its core Java framework

Facebook open-sources PyText, a PyTorch based NLP modeling framework

Amrata Joshi