trycatchdebug.net Open in urlscan Pro
104.21.112.1  Public Scan

Form analysis
0 forms found in the DOM

Text Content

Home 🔥 Popular
🌙


SETTING UP ELK STACK WITH DOCKER AND MULTIPLE SERVERS FOR GLOBAL LOGGING

Abstract: Learn how to set up ELK Stack with Docker and multiple servers for
efficient global logging.

2024-12-03 by Try Catch Debug


In this article, we will discuss how to set up the ELK Stack (Elasticsearch,
Logstash, and Kibana) using Docker, with a focus on configuring global logging
from multiple servers. The ELK Stack is a popular open-source solution for log
collection, processing, and analysis.


PREREQUISITES

Before we begin, ensure that you have the following prerequisites in place:

 * Docker installed on all servers
 * Access to all servers for configuration


SETTING UP ELASTICSEARCH

Elasticsearch is the backbone of the ELK Stack, responsible for indexing and
searching the logs. To set up Elasticsearch using Docker:

 1. Create a new directory for Elasticsearch:

    mkdir elk_stack cd elk_stack

 2. Create a new file named docker-compose.yml in the elk_stack directory:

    touch docker-compose.yml

 3. Edit the docker-compose.yml file to configure Elasticsearch:

     version: '3'
     services:
     elasticsearch:
     image: elasticsearch:latest
     container_name: elasticsearch
     ports:
     - "9200:9200"
     - "9300:9300"
     environment:
     - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
     
     

 4. Start Elasticsearch:

    docker-compose up -d elasticsearch


SETTING UP LOGSTASH

Logstash is responsible for collecting and processing logs. To set up Logstash
using Docker:

 5. Create a new file named logstash.conf in the elk_stack directory:

     input {
     beats {
     port => 5044
     }
    }
    
    
    filter {
    
    
    
    Add your filters here
    
    
    }
    
    
    output {
    elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    }
    }
    
    

    

 6. Edit the docker-compose.yml file to configure Logstash:

     services:
     elasticsearch:
     # ...
     logstash:
     image: docker.elastic.co/logstash/logstash:latest
     container_name: logstash
     ports:
     - "5044:5044"
     volumes:
     - "./logstash.conf:/usr/share/logstash/pipeline.conf"
     depends_on:
     - elasticsearch
    
     

 7. Start Logstash:

    docker-compose up -d logstash


SETTING UP KIBANA

Kibana is the user interface for visualizing and analyzing logs. To set up
Kibana using Docker:

 9.  Edit the docker-compose.yml file to configure Kibana:

      services:
      elasticsearch:
      # ...
      logstash:
      # ...
      kibana:
      image: docker.elastic.co/kibana/kibana:latest
      container_name: kibana
      ports:
      - "5601:5601"
      depends_on:
      - elasticsearch
      environment:
      - "KIBANA_HOST=elasticsearch"
     
      

 10. Start Kibana:

     docker-compose up -d kibana


CONFIGURING GLOBAL LOGGING

To configure global logging from multiple servers:

 13. Install the Filebeat agent on each server:

# For Debian-based systems apt-get install filebeat

 14. Edit the filebeat.yml file on each server:

 output.logstash:
 hosts: ["logstash:5044"]

 

 15. Restart Filebeat:

systemctl restart filebeat

In this article, we discussed how to set up the ELK Stack using Docker and
configure global logging from multiple servers. We covered the setup of
Elasticsearch, Logstash, and Kibana, as well as configuring Filebeat to send
logs to Logstash. For more information:

 * ELK Stack Docker
 * Logstash User Guide
 * Kibana User Guide

Tags::  ELKStack Docker Logging Kibana Logstash Elasticsearch




LATEST NEWS

 * Cloud Run Service Not Receiving SIGTERM Shutdown: A Guide for Express-Based
   Node.js Web Services
 * SSL Certificate Issue with Python 3.12.0: Installing an Alternative Version
 * Making Ionic textarea fill parent container
 * Adding Custom Fonts to Quill Text Editor in Next.js: A Solution to Pasting
   Issues
 * Optimizing Loops with g++-O3 Flag: A Software Development Insight
 * Setting Multiple Elements in Polars DataFrame Conditionally: A Polars
   Equivalent to Pandas
 * Fixing Flutter APK Build after Failed Upgrade to 3.7.21
 * Zero Downtime Data Migration from Index A to Index B in Elasticsearch 7.10.0
 * std::bind Virtual Base Class Member Function Failure: Uninitialized Warning
   with AArch32 and GNU 14.2
 * Laravel Passport Token Authentication: Resolving $tokenResult Null Issue
 * GHC Strict Field Packing using Unboxed Types Explicitly: A Brief Overview
 * Creating a Carousel Similar to SlideSwiper in Flutter
 * Add Shopify Button to Angular 16 Application
 * Fixing Text Overflow Issues in Table Displays: A CSS Solution
 * Optimizing Performance of Slow Spring Data JPA Query DSL FindAll Queries
 * Efficiently Reading MP4 Files in a Folder with Software Development
 * Filtering Actual Group Name in Bundle Status Bar Notifications using
   NotificationListenerService
 * Creating Overlapping Panoramic Stream from Two Webcam Images - Not Working
 * Creating Dynamically Loadable Shared Archives in AIX using CMake
 * Fetching Specific Entities from Excel using C# and ASP.NET MVC
 * Querying Large Datasets with EF Core 8 Taking Longer Time in Software
   Development Site - Balzor WCF Application
 * Managing Drive Folder Sharing with Google Apps Script: Updating Folder
   Viewers Based on a Separate Service's Email List
 * Understanding Snyk Delta's Behavior with Azure DevOps
 * Newline Added: File Writing with String IO in Software Development
 * Could Convert MongoDB Aggregation Pipeline Updates to Spring Data MongoDB
   (Reactive) Typecheck?
 * Flutter App Development: Obsolete and Removed Future Release Warning
 * Transparent Non-Clickable Window with Clickable Button using JNA in Java
 * Unpacking New Generic Syntax in Python 3.13 Causes MyPy Error?
 * Creating a React-Based Board Game: Trigon - Triangle Block Puzzle
 * Pandas Don't Drop Level MultiIndex: Getting First Value
 * Exploring Alternative Methods: Haskell for Software Development
 * SvelteKit 5: Going back to URL doesn't update the page
 * Initializing Kivy App with 3rd Party .NET DLLs: Keeping the Main Thread
   Responsive
 * Implementing Role-based Authorization in ASP.NET Web API .NET 8: A Guide
 * Turn Function Parser in Haskell with Megaparsec



We try to keep you informed about the latest software development news and more.

TryCatchDebug.net - All rights reserved 2023  | About | Privacy | Contact |
Archive