Indexing Network Traffic with Moloch and Elastic

Published in

The Startup

7 min readFeb 4, 2020

Data is Everything

Whether hunting down hackers in your network, optimizing traffic flow, or just trying to build a network baseline, you’ll learn that data is everything. Data is our gateway to gaining deeper insights into the patterns or behaviors of our users and devices. The data we are hoping to gain insights into is network traffic.

Following the Data, Information, Knowledge, Wisdom (DIWK) model, we’re going to walk through building a pipeline for turning network traffic data into network traffic knowledge. Out two tools of choice will be Moloch and Elastic.

What is Moloch?

In short terms, Moloch is an open-source, scalable packet capture and indexing solution. Initially developed by AOL for handling their large volumes of traffic, the kind folks there released it under the Apache 2.0 license for use by the community. What makes Moloch a fantastic tool for Security Operations is that one of its core components is built on Elasticsearch already, allowing you to leverage your existing Elastic Stack if you choose to.

The main components of Moloch are as follows:

MolochCapture: PCAP capture and analysis
MolochViewer: Web-based GUI viewing data and transferring PCAPs
Elasticsearch: Search the database

Design Considerations

Ideally, Moloch would be deployed much like an IDS sensor. If your network is small, centrally located, or cloud-based, it may be possible to implement a single instance and capture all your organization’s traffic. However, if you have a network the sprawls out physically, such as a university campus, then it may make sense to deploy multiple “sensors” to capture your core routing and switching backbone. In the same vein, it would be up to you to decide if you want to host your Elastic Stack on the same machine as Moloch or elsewhere This flexibility is part of what makes Moloch such a unique tool for gaining insight into your network traffic.

Deploying Elastic Stack

Here we’ll be using an Elastic stack container in Photon OS. Photon is a lightweight Operating System built by VMware and designed to run in virtual environments. I’m going to assume you’ve already installed Photon OS on your hypervisor of choice and have installed all updates. If not, there are plenty of tutorials on building and deploying hypervisors, with just as many debates on which vendor or technology is better.

Note: This guide should work regardless of what OS you decide to use. The main requirement is Docker, and you aren’t constrained to running it in a VM. If you deploy Docker on a bare-metal OS, everything should still work correctly.

Before pulling down our Elastic Stack image, we have to verify the virtual memory mapping (mmap). The Elastic Stack requires a mmap count equal to or more than 262,144.

$ sysctl vm.max_map_count
vm.max_map_count = 262144

If you need to change the vm.max_map_count of your machine, enter the following:

sysctl -w vm.max_map_count=262144

Since we have Docker, already installed, and our mmap set correctly, let us go ahead and pull the sebp/elk image for our Elastic Stack and run it.

$ docker pull sebp/elk
$ docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -it \
--name elk sebp/elk

Let’s verify our Elastic Stack is listening.

$ netstat -plnt

We can see our container is up and listening now. The sebp image uses the following port assignments:

5601 (Kibana web interface)
9200 (Elasticsearch JSON interface)
5044 (Logstash Beats interface)

Now that our Elastic Stack is up and running, it’s time to deploy Moloch.

Also, if you want to collect Windows logs into your elastic stack, checking out my article on Winlogbeats!

Collecting Windows Logs with Elastic’s Winlogbeats

Begin Threat Hunting with Elastic

kyletopasna.medium.com

Deploying Moloch

For this guide, we’ll only be using one Moloch instance, running on a Centos 7 machine. Like with the Docker section, I’ll assume you’ve installed Centos 7 on your own and upgraded all it’s core packages.

First, we’ll pull down the rpm and install it.

$ wget https://files.molo.ch/builds/centos-7/moloch-2.2.1-1.x86_64.rpm
$ sudo yum install moloch-2.2.1-1.x86_64.rpm

Once installed, we’ll go ahead and configure Moloch. Be sure to have the IP address for your Photon VM. In this case, our Photon VM is 192.168.1.67.

$ cd /data/moloch/bin/
$ ls -l
total 6636
-rwxr-xr-x. 1 root root    9851 Jan 21 05:19 Configure
-rwxr-xr-x. 1 root root     100 Jan 21 05:19 moloch_add_user.sh
-rwxr-xr-x. 1 root root 6767952 Jan 21 05:16 moloch-capture
-rwxr-xr-x. 1 root root     829 Jan 21 05:19 moloch_update_geo.sh
lrwxrwxrwx. 1 root root      35 Feb  3 17:57 node -> ../node-v10.18.1-linux-x64/bin/node
lrwxrwxrwx. 1 root root      34 Feb  3 17:57 npm -> ../node-v10.18.1-linux-x64/bin/npm
lrwxrwxrwx. 1 root root      34 Feb  3 17:57 npx -> ../node-v10.18.1-linux-x64/bin/npx
-rwxr-xr-x. 1 root root    2007 Jan 21 05:16 taggerUpload.pl
$ sudo ./Configure

Next, initialize the database.

$ sudo /data/moloch/db/db.pl http://ESHOST:9200 init

Let’s add a user account to be able to reach the web front end.

$ /data/moloch/bin/moloch_add_user.sh admin "Admin User" SECRETPASSWORD --admin

Now that Moloch is configured and initialized, we’ll go ahead and start the molochcapture and molochviewer services as well as set them to start after reboots.

$ sudo systemctl enable molochcapture.service
$ sudo systemctl enable molochviewer.service$ sudo systemctl start molocapture.service
$ sudo systemctl start molochviewer.service

View our services to ensure they started.

$ systemctl status molochcapture molochviewer

As we saw, our molochcapture failed to start. After viewing the logs at /data/moloch/logs/capture.log, we discovered that molochcapture was trying to capture on eth1, not eth0. We can fix this by opening /data/moloch/etc/config.ini.

$ sudo vim /data/moloch/etc/config.ini

Here at line 56, I changed the interface from eth1 to eth0, saved the config, and restarted my services. ALL GREEN! (I learned after I selected the wrong interface when I ran the ./Configure script.)

Time to add the finishing touches for Moloch. Depending on your setup, you may have to adjust your iptables or firewalld setting to be able to reach web GUI from another machine. In our case, firewalld is the Centos 7 firewall, so we’ll add entries to allow access.

$ sudo firewall-cmd --add-port=8005/tcp
success$ sudo firewall-cmd --runtime-to-permanent
success

Boom! Our Moloch sensor is deployed and configured.

Configuring Kibana

Molochviewer is excellent and everything, but we started this to set up an architecture that can handle multiple Moloch sensors, and we wanted a way to analyze our data in aggregate. To that end, we’ll put the finishing touches on our Elastic Stack to handle the newly indexed Moloch data.

We’ll connect to our Kibana application on port 5601. If this was a fresh install of Elastic Stack, select “Explore on my own.”

Click the “Discover” tab on the right. From there, we’ll enter “sessions*” as our index pattern and click next.

After we select “timestamp” as our Time Filter field name, then click the “Create Index Pattern.”

When we go back to the “Discover” tab, we’ll see that our packet capture information has now populated our Elastic Stack!

Conclusion

Here we went over standing up a Moloch sensor and sending the network traffic to an Elastic Stack. Both tools are potent in their own right. Still, when leveraged together, it will allow the security analyst or network engineer to gain far greater insights into the traffic traversing their network without the need for expensive licenses or proprietary software solutions.

The next step would be setting up visualizations and dashboards in Kibana, but the content for those would depend entirely on your organization’s needs.

Before You Go…

Check out my latest article regarding how Adversaries can exploit BioCyber Security:
https://kyletopasna.medium.com/the-dawn-of-biocyber-warfare-a83c575d5e25