Moloch: Open Source Large Scale Full Packet Capturing Tool

Moloch is an open source, large scale, full packet capturing, indexing, and database system. Moloch augments your current security infrastructure to store and index network traffic in standard PCAP format, providing fast and indexed access. An intuitive and simple web interface is provided for PCAP browsing, searching, and exporting. Moloch exposes APIs which allow for PCAP data and JSON formatted session data to be downloaded and consumed directly. Moloch stores and exports all packets in standard PCAP format allow you to also use your favorite PCAP ingesting tools, such as wireshark, during your analysis workflow.

The Moloch system is comprised of 3 components:

  • Capture – A threaded C application that monitors network traffic, writes PCAP formatted files to disk, parses the captured packets and sends Meta data (SPI data) to elastic search.
  • Viewer – A node.js application that runs per capture machine and handles the web interface and transfer of PCAP files.
  • Elastic search – The search database technology powering Moloch.

Hardware Requirements

Moloch is built to run across many machines for large deployments. What follows are rough guidelines for folks capturing large amounts of data with high bit rates, obviously tailor for the situation. It is not recommended to run the capture and elasticsearch processes on the same machines for highly utilized GigE networks. For demo, small network, or home installations everything on a single machine is fine.

  1. Moloch capture/viewer systems
    • One dedicated management network interface and CPU for OS
    • For each network interface being monitored recommend ~10G of memory and another dedicated CPU
    • If running suricata or another IDS add an additional two (2) CPUs per interface, and an additional 5G memory (or more depending on IDS requirements)
    • Disk space to store the PCAP files: We recommend at least 10TB, xfs (with inode64 option set in fstab), RAID 5, at least 5 spindles)
    • Disable swap by removing it from fstab
    • If networks are highly utilized and running IDS then CPU affinity is required

     2. Moloch elastic search systems

    • 1/4 * Number_Highly_Utilized_Interfaces * Number_of_Days_of_History is a ROUGH guideline for number of elasticsearch instances (nodes) required. (Example: 1/4 * 8 interfaces * 7 days = 14 nodes)
    • Each elasticsearch node should have ~30G-40G memory (20G-30G [no more!] for the java process, at least 10G for the OS disk cache)
    • You can have multiple nodes per machine (Example 64G machine can have 2 ES nodes, 22G for the java process 10G saved for the disk cache)
    • Disable swap by removing it from fstab
    • Obviously the more nodes, the faster responses will be
    • You can always add more nodes, but it’s hard to remove nodes (more on this later)

Ehacking Staff
With more than 50 global partners, we are proud to count the world’s leading cybersecurity training provider. EH Academy is the brainchild of Ehacking, which has been involved in the field of training since the past Five years and continues to help in creating professional IT experts.

Most Popular

Blind SQL Injection Tutorial to Hack a Website

In the previous article, we have the basics of SQL Injection; what SQLi is and what are the types of SQL injection. And, In...

What is SQL Injection? Tutorial: Type and Example

What is SQL injection, and what are the types of SQL injection? These are the common questions, and we will seek the answer to...

Are Cisco 300-410 Exam and Its Related Certification Your Pathway to Career Success? Find Out about This

Introduction Career success can mean different things to different people. For some, it could mean having a prestigious title and for others, it could be...

How to Hack Windows 10 Password Using FakeLogonScreen in Kali Linux

This article demonstrates an in-depth guide on how to hack Windows 10 Passwords using FakeLogonScreen. Hacking Windows 10 password is an exciting topic and...

LOOKING FOR HACKING RECIPES FORM THE PRO?

Then sign up for FREE to the ehacking’s exclusive group. You will get the exclusive tips/tricks, tutorials, webinars & courses that I ONLY share with my fellow on this exclusive newsletter.