trycatchdebug.net Open in urlscan Pro
2606:4700:3030::6815:6001  Public Scan

Form analysis
0 forms found in the DOM

Text Content

Home 🔥 Popular
🌙


SETTING UP ELK STACK WITH DOCKER AND MULTIPLE SERVERS FOR GLOBAL LOGGING

Abstract: Learn how to set up ELK Stack with Docker and multiple servers for
efficient global logging.

2024-12-03 by Try Catch Debug


In this article, we will discuss how to set up the ELK Stack (Elasticsearch,
Logstash, and Kibana) using Docker, with a focus on configuring global logging
from multiple servers. The ELK Stack is a popular open-source solution for log
collection, processing, and analysis.


PREREQUISITES

Before we begin, ensure that you have the following prerequisites in place:

 * Docker installed on all servers
 * Access to all servers for configuration


SETTING UP ELASTICSEARCH

Elasticsearch is the backbone of the ELK Stack, responsible for indexing and
searching the logs. To set up Elasticsearch using Docker:

 1. Create a new directory for Elasticsearch:

    mkdir elk_stack cd elk_stack

 2. Create a new file named docker-compose.yml in the elk_stack directory:

    touch docker-compose.yml

 3. Edit the docker-compose.yml file to configure Elasticsearch:

     version: '3'
     services:
     elasticsearch:
     image: elasticsearch:latest
     container_name: elasticsearch
     ports:
     - "9200:9200"
     - "9300:9300"
     environment:
     - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
     
     

 4. Start Elasticsearch:

    docker-compose up -d elasticsearch


SETTING UP LOGSTASH

Logstash is responsible for collecting and processing logs. To set up Logstash
using Docker:

 5. Create a new file named logstash.conf in the elk_stack directory:

     input {
     beats {
     port => 5044
     }
    }
    
    
    filter {
    
    
    
    Add your filters here
    
    
    }
    
    
    output {
    elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    }
    }
    
    

    

 6. Edit the docker-compose.yml file to configure Logstash:

     services:
     elasticsearch:
     # ...
     logstash:
     image: docker.elastic.co/logstash/logstash:latest
     container_name: logstash
     ports:
     - "5044:5044"
     volumes:
     - "./logstash.conf:/usr/share/logstash/pipeline.conf"
     depends_on:
     - elasticsearch
    
     

 7. Start Logstash:

    docker-compose up -d logstash


SETTING UP KIBANA

Kibana is the user interface for visualizing and analyzing logs. To set up
Kibana using Docker:

 9.  Edit the docker-compose.yml file to configure Kibana:

      services:
      elasticsearch:
      # ...
      logstash:
      # ...
      kibana:
      image: docker.elastic.co/kibana/kibana:latest
      container_name: kibana
      ports:
      - "5601:5601"
      depends_on:
      - elasticsearch
      environment:
      - "KIBANA_HOST=elasticsearch"
     
      

 10. Start Kibana:

     docker-compose up -d kibana


CONFIGURING GLOBAL LOGGING

To configure global logging from multiple servers:

 13. Install the Filebeat agent on each server:

# For Debian-based systems apt-get install filebeat

 14. Edit the filebeat.yml file on each server:

 output.logstash:
 hosts: ["logstash:5044"]

 

 15. Restart Filebeat:

systemctl restart filebeat

In this article, we discussed how to set up the ELK Stack using Docker and
configure global logging from multiple servers. We covered the setup of
Elasticsearch, Logstash, and Kibana, as well as configuring Filebeat to send
logs to Logstash. For more information:

 * ELK Stack Docker
 * Logstash User Guide
 * Kibana User Guide

Tags::  ELKStack Docker Logging Kibana Logstash Elasticsearch




LATEST NEWS

 * iPhone App Development on iOS 18: WKWebView's On-Screen Keyboard Issue in
   YoutubeViewController
 * Dart Code Suggestion Not Working in IntelliJ IDEA
 * Using Minimum Value with fill function in MATLAB for 7301x1 variable
 * Understanding CoroutineScope.launch Function in Kotlin
 * Error in Filtering Non-existent Levels: Tidyverse Factor Variable
 * Understanding __annotations__ and Instance Variables in Python 3.13: Test
   Class
 * Batch Inference with AWS Bedrock: Handling Large Data using JSON Format
 * RecordBatchReader Not Accessing Data in Parquet Files with Multiple Row
   Groups
 * Updating SearchParams without Causing Page Renders in React 18
 * Protecting CSRF Token for API Endpoint in Django DRF and Solidjs
 * Choosing XML Layout vs Jetpack Compose for Your Android Project
 * PyTorch DeiT Model Keeps Predicting One Class: A Fine-Tuning Dilemma
 * Specify Mock Get Null Call Else Block in Java: New Look Answered
 * Spring Boot 3.1.x to 3.3.x Upgrade: Failed Unit Tests and
   ClassNotFoundException
 * Loading .NET Dependencies instead of PowerShell Modules in the Current
   Working Directory
 * Clarifying the Meaning of LastModificationTime in CyberArk Account Objects
 * Efficiently Handling Time Waiting Response with Elastic Beanstalk and EKS
 * Cloud Run Service Not Receiving SIGTERM Shutdown: A Guide for Express-Based
   Node.js Web Services
 * SSL Certificate Issue with Python 3.12.0: Installing an Alternative Version
 * Making Ionic textarea fill parent container
 * Adding Custom Fonts to Quill Text Editor in Next.js: A Solution to Pasting
   Issues
 * Optimizing Loops with g++-O3 Flag: A Software Development Insight
 * Setting Multiple Elements in Polars DataFrame Conditionally: A Polars
   Equivalent to Pandas
 * Fixing Flutter APK Build after Failed Upgrade to 3.7.21
 * Zero Downtime Data Migration from Index A to Index B in Elasticsearch 7.10.0
 * std::bind Virtual Base Class Member Function Failure: Uninitialized Warning
   with AArch32 and GNU 14.2
 * Laravel Passport Token Authentication: Resolving $tokenResult Null Issue
 * GHC Strict Field Packing using Unboxed Types Explicitly: A Brief Overview
 * Creating a Carousel Similar to SlideSwiper in Flutter
 * Add Shopify Button to Angular 16 Application
 * Fixing Text Overflow Issues in Table Displays: A CSS Solution
 * Optimizing Performance of Slow Spring Data JPA Query DSL FindAll Queries
 * Efficiently Reading MP4 Files in a Folder with Software Development
 * Filtering Actual Group Name in Bundle Status Bar Notifications using
   NotificationListenerService
 * Creating Overlapping Panoramic Stream from Two Webcam Images - Not Working



We try to keep you informed about the latest software development news and more.

TryCatchDebug.net - All rights reserved 2023  | About | Privacy | Contact |
Archive