socfortress.medium.com Open in urlscan Pro
2606:4700:7::a29f:9904  Public Scan

Form analysis
0 forms found in the DOM

Text Content

Open in app

Sign up

Sign in

Write


Sign up

Sign in



Top highlight


BUILD YOUR OWN SIEM STACK WITH OPEN SOURCE TOOLS SERIES

SOCFortress

·

Follow

9 min read
·
Oct 1, 2022

252

5

Listen

Share

Intro to our Worlds Best SOC Built on Open Source Tools series.



PART ONE: Backend Storage

PART TWO: Log Ingestion

PART THREE: Log Analysis

PART FOUR: Wazuh Agent Install — Endpoint Monitoring

PART FIVE: Intelligent SIEM Logging

PART SIX: Best Open Source SIEM Dashboards

PART SEVEN: Firewall Log Collection Made Easy

PART EIGHT: Firewall Threat Intel With GreyNoise




INTRO

Hey Defenders, have you struggled with obtaining the proper insight into
collecting, analyzing, and storing your security events? While there are plenty
of commercial products on the market, these often come with high price tags.
These high price points make it difficult for organizations to keep their
environments secure while leaving room in the budget for R&D, marketing, and
profit generating expenditures.



SOCFortress believes network security is a right, not a privilege.

This blog series will cover how you can build your own Open Source SOC!




SOLUTION

What if we could combine the best of what Open-Source software has to offer to
create a secure, robust, intelligent, and proactive cyber defense solution? Well
that’s exactly what we decided to do. Why spend hundreds of thousand on security
software when there are free tools out there for use? Use that saved money to
increase R&D, hire better talent, or reach out to more potential customers. All
with the peace of mind of a secure and certified (ISO, GDPR, SOC, etc.)
environment.





WHAT KEY ELEMENTS MAKE UP A SIEM STACK?

We first need to understand what key pieces make up a SIEM stack and why.
Without the proper tools in place, it will be difficult for our security team to
detect, assess, triage, and respond to security events. This is especially true
as your networks grow and the amount of ingested logs increases.

The below are key elements, in my opinion, that must be built into any SIEM
stack.

 1. Log Ingestion
 2. Log Analysis
 3. Backend Storage
 4. Visualization
 5. Intelligence Enrichment
 6. Case Management
 7. Automation
 8. Investigation and Response
 9. Health Monitoring





LOG INGESTION

Before our analysts can view security logs, we first need to ingest them. It is
helpful to go through an exercise of determining what log sources you want to
ingest. Common logs include:

 * Endpoint Logs (Windows Events, Sysmon, Powershell logs, etc.)
 * Network Devices (Firewalls (IDS/IPS), Switches, Access Points)
 * Proxies (Apache, NGINX, etc.)
 * 3rd Parties (AWS Cloud Trail, O365, Tenable, etc.)

As we start to ingest logs from multiple sources, it is crucial for us to ensure
our logs are being normalized to common field names. For example, source_ip ,
source_ipv4_ip should both be rewritten to a src_ip field. This will save us
time and energy later as we start to develop dashboards and alerting strategies.



With log normalization we can now build a universal dashboard that displays all
network connections regardless of the source we collected the log from.




GRAYLOG

Graylog is our tool of choice when it comes to log ingestion. Graylog is
responsible for collecting logs from various log sources:

 * Wazuh Manager
 * Network Devices
 * Syslog Forwarders from 3rd parties (Cylance, Crowdstrike, etc.)
 * And a ton more!

Graylog also handles index management of indices that are stored within the
Wazuh-Indexer backend to fit the index lifecycle of choice.





LOG ANALYSIS

While collecting a wealth of logs is a good starting point, we need to be able
to analyze the meta details within our collected logs to accurately build
alerting and prioritize our security events. For example, maybe I want to detect
when powershell was invoked with the -ExecutionPolicy Bypass flag.

 * Analyze logs received from endpoints / services.
 * Determine the severity of the logs ingested via log analysis. Supported
   ability for custom rules.
 * Ability to discard noisy alerts to limit overflow of unnecessary data.

All networks are different and having the flexibility to create your own custom
rules is a must.

You can get started with our publicly provided Wazuh custom rules on your own:
FREE Advanced Wazuh Detection Rules.


WAZUH

Wazuh is a great tool that allows us to not only collect logs from our
endpoints, but comes built with baked in rules that analyze the contents of our
logs to detect attacks.



Wazuh also provides:

 * Configuration Assesssment
 * File Integrity Monitoring
 * Vulnerability Detection
 * And much more!




BACKEND STORAGE

Ingesting and analyzing logs is great, but where are those logs going to be
stored? It is imperative for us to provide a backend storage architecture that
allows for:

 * Storing, searching, and viewing of data (our collected security events)
 * High Availability
 * Robust performance
 * Ability to scale


WAZUH-INDEXER

The Wazuh-Indexer which is Wazuh’s forked version of OpenSearch allows us to do
just that. It’s feature rich API also allows us to plug other tools into the
Wazuh-Indexer stack such as Grafana, Elastalert, etc.




VISUALIZATION

Storing our logs is only one piece of the puzzle. Without a way for our SOC
analysts to easily view, pivot, triage, and search for security threats, or
stack is really no good. We would suffer a slow death of exhaustion trying to
shift through all these logs trying to find the needle in a haystack.

Our visualization tools must give us the ability to:

 * View logs via widgets / dashboards / etc.
 * Fast searching and viewing of data.
 * Support the ability to read from multiple log storages (Wazuh-Indexer, csv
   files, MySQL, etc.)




GRAFANA

Grafana is our recommended choice when it comes to our visualization tool.
Grafana is lightening quick (compared to Kibana), fully customizable, rich with
pre built widgets, backed by a strong community, and offers multi tenant
support!

Grafana allows us to build out a “single pane of glass” approach to view all of
our security events.




INTELLIGENCE ENRICHMENT

Apart from just analyzing our logs, we need a way to enrich our logs with
intelligence to help our analyst quickly spot potential malicious activity. For
example, is this IP address that I see interacting with my website malicious or
not? We need a solution that can:

 * Enrich received logs with threat intelligence gathered from various
   providers.
 * Parse and store selected responses so that only crucial data is stored.
 * Automated so that your SOC analysts are not having to manually attempt to
   enrich received logs.


OPENCTI

The first purpose of the OpenCTI platform is to provide a powerful knowledge
management database with an enforced schema especially tailored for cyber threat
intelligence and cyber operations.




MISP

MISP provides metadata tagging, feeds, visualization and even allows you to
integrate with other tools for further analysis thanks to its open protocols and
data formats.



Both tools offer a rich API that allows us to automate threat intel lookups on
the fly!




CASE MANAGEMENT

As the SOC team grows, we need to provide a platform that allows them to
collaborate, enrich, and respond to alerts. Providing playbooks, tasks, and
procedures for your SOC analysts will help guide them through alerts detected
and allow them to focus on CRITICAL alerts.

 * Platform to view and react to HIGH severity events.
 * Allow collaboration with multiple SOC analysts.
 * Allow responsive actions so that analysts can trigger events on their
   endpoints.





THEHIVE/CORTEX

TheHIVE gives us the ability to manage, organize, correlate your incidents and
automate your forensics analysis while leveraging a powerful collaborative
capability.

While Cortex provides investigation of observables (file hashes, IPs, domains,
etc.) from third party or legacy services and automated active response.




AUTOMATION

As our log collection increases, we need a tool that can automate many of our
tasks such as:

 * Case creation
 * Phishing analysis
 * Failed healthchecks
 * Report generation
 * Literally anything :)

All tools mentioned in this post give provide their own API which we can plug
our SOAR platform into to automate anything and everything!




SHUFFLE

Shuffle is an Open Source interpretation of SOAR. It aims to bring all the
capabilities necessary to transfer data throughout an enterprise with
plug-and-play Apps, making automation approachable for everyone. It should
remove the need for a coder on the team (I still suggest having at least one..),
empowering everyone by being able to deploy new, complicated (or simple)
workflows in minutes rather than hours or days.

Follow Frikky’s blog series for a deeper dive into Shuffle before we explore it
on our own :)




INVESTIGATION

Receiving alerts is only half the battle, we must provide our SOC analysts with
the ability to thoroughly investigate alerts by interacting with the monitored
endpoints in a manner that is scalable and fast. Some techniques include:

 * Listing running processes
 * Enumerating logged in users
 * Detect listening ports
 * View downloaded files
 * Quarantining a device
 * And much more!



Without this ability, it is difficult for our SOC analyst to quickly assess the
actual severity of an alert.


VELOCIRAPTOR

Velociraptor is an advanced digital forensic and incident response tool that
enhances your visibility into your endpoints. At the press of a (few) buttons,
we can perform targeted collection of digital forensic evidence simultaneously
across our endpoints, with speed and precision. Their solid API allows us to
automate and trigger evidence collection when we need it.


HEALTH MONITORING

With our stack now built it out, we need to monitor the health of our overall
SIEM stack to ensure smooth operations and minimize the risk of missing alerts.
I like to think of monitoring in two phases:

 * Endpoint resources (CPU, RAM, Disk, Processes, etc.)
 * WebUI uptime

For example, maybe the Grafana service is running just fine, but there was a
firewall change made that prohibits users from accessing the Grafana WebUI.
While Grafana itself is healthy, our analysts would be unable to view alerts
without access to the WebUI and ultimately rendering Grafana useless until the
issue is resolved.




INFLUXDB/TELEGRAF

InfluxDB combined with the Telegraf agent allows us to collect all endpoint
metrics and offers built in alerting for when thresholds are met or critical
processes (such as the wazuh-indexer) are not running. This allows our
engineering team to proactively respond to potential issues before they escalate
to a serious impact.


UPTIME KUMA

Uptime Kuma is a monitoring tool that you can use to monitor websites and
applications in real time. Features include:

 * Monitors uptime for HTTP(s) websites, TCP Ports, and Docker containers and
   retrieves information such as DNS records.
 * Sends notifications via Email (SMTP), Telegram, Discord, Microsoft Teams,
   Slack, Promo SMS, Gotify, and 90+ notification services.
 * Supports multiple languages.
 * Provides multiple Status Pages.
 * Provides Proxy Support.
 * Displays SSL certificate information.
 * Maps Status Page to Domain.




CONCLUSION

Security is no easy task. Nor does it have to break the bank. There are a wealth
of Open Source tools available for us to build our own SIEM stack with minimal
cost. The flexibility of Open Source also allows us to customize anything and
everything that pertains to your unique networks and needs. Throughout this
series we will be deep diving into each of these tools to help serve as a guide
as you start to build our own in house SOC that is up to par with commercial
tools currently on the market.

Our deployment strategy (coming in future posts) will be as followed:

 1.  Wazuh Indexer (OpenSearch)
 2.  Wazuh Dashboards (OpenSearch Dashboards)
 3.  Graylog
 4.  Wazuh Manager / Agents
 5.  Grafana
 6.  MISP
 7.  OpenCTI
 8.  TheHIVE / Cortex
 9.  Velociraptor / Agents
 10. Shuffle
 11. InfluxDB / Telegraf
 12. Uptime Kuma

We cannot wait to see you guys there…happy defending :)




NEED HELP?

The functionality discussed in this post, and so much more, are available via
the SOCFortress platform. Let SOCFortress help you and your team keep your
infrastructure secure.

Website: https://www.socfortress.co/

Platform Demo: https://www.socfortress.co/demo_access.html



Be part of a better internet.
Get 20% off membership for a limited time.


FREE



Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.


Sign up for free


MEMBERSHIP

Get 20% off


Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app


Try for $5 $4/month
Siem
Soc
Open Source
Information Security
Cybersecurity


252

252

5


Follow




WRITTEN BY SOCFORTRESS

3.2K Followers

SOCFortress is a SaaS company that unifies Observability, Security Monitoring,
Threat Intelligence and Security Orchestration, Automation, and Response (SOAR).

Follow





MORE FROM SOCFORTRESS

SOCFortress


PART 1. WAZUH INDEXER — SIEM BACKEND


LET’S INSTALL THE BACKEND SERVICE THAT WILL STORE OUR COLLECTED SECURITY LOGS,
WAZUH-INDEXER.

Oct 3, 2022
108
2



SOCFortress


PART 2. GRAYLOG INSTALL — LOG INGESTION


NORMALIZE AND ENRICH SECURITY LOGS WITH GRAYLOG!

Oct 9, 2022
39
9



SOCFortress


UPGRADING WAZUH TO VERSION 4.8.0: A STEP-BY-STEP GUIDE


IN THIS TUTORIAL, WE’LL WALK YOU THROUGH UPGRADING YOUR WAZUH SETUP TO THE
LATEST 4.8.0 RELEASE. THIS UPGRADE INCLUDES SIGNIFICANT CHANGES…

Jun 22
5



SOCFortress


PART 10. MISP THREAT INTEL


DEPLOY YOUR OWN THREAT INTEL IN UNDER 10MINUTES!

Dec 11, 2022
72
1


See all from SOCFortress



RECOMMENDED FROM MEDIUM

Stefan Bargan

in

CyberScribers


ESSENTIAL TOOLS FOR SOC ANALYSTS


AS A SECURITY OPERATIONS CENTRE (SOC) ANALYST, HAVING THE RIGHT TOOLS AT YOUR
DISPOSAL IS CRUCIAL FOR EFFECTIVE INVESTIGATION, REPUTATION…


Jun 9
28
3



Jonathan Mondaut


HOW CHATGPT TURNED ME INTO A HACKER


DISCOVER HOW CHATGPT HELPED ME BECOME A HACKER, FROM GATHERING RESOURCES TO
TACKLING CTF CHALLENGES, ALL WITH THE POWER OF AI.


Jun 18
489
14




LISTS


TECH & TOOLS

17 stories·277 saves


DATA SCIENCE AND AI

40 stories·206 saves


ICON DESIGN

36 stories·372 saves


NATURAL LANGUAGE PROCESSING

1616 stories·1182 saves


N1neKitsune


[MONITORING] INSTALLATION AND CONFIGURATION OF AURORA AGENT: THE FREE AND
POWERFUL EDR


TODAY, WE’RE DIVING INTO THE USE OF AURORA AGENT, AN ESSENTIAL TOOL FOR
SAFEGUARDING YOUR COMPUTER SYSTEMS…


May 24
20



Yogasatriautama


SOC: INSTALL OPENCTI


OPENCTI (OPEN CYBER THREAT INTELLIGENCE) IS AN OPEN-SOURCE PLATFORM DESIGNED TO
COLLECT, STORE, AND UTILIZE CYBER THREAT DATA. IT HELPS…


Jul 21
30



Kristjan Hiis


INFLUXDB FOR PROXMOX UP AND RUNNING


WHEN I STARTED TWEAKING AROUND MY HOME LAB TODAY, I WANTED TO GET MY HOME LAB
MONITORING INTO GRAFANA. AT FIRST, IT WAS A VERY…


Jul 7
7



Corey Jones

in

T3CH


SETTING UP SNORT TO MONITOR YOUR SOC LAB


May 12
115


See more recommendations

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams