Logstash Basics

Let's learn about Logstash

Reference: Logstash docsarrow-up-right

What is Logstash?

Logstash

  • An open source data collection engine with real-time pipeline capabilities

  • Can dynamically unify different data sources and normalize data to a specified destination

  • Can cleanse and democratize data

The Power of Logstash

1. Powerful Ingestion for Elasticsearch and More

A horizontally scalable data processing pipeline leveraging the synergy of Elasticsearch and Kibana

2. Pluggable pipeline architecture

Mix & match and orchestrate various inputs, filters, and outputs harmoniously in the pipeline

3. Community-extensible and developer-friendly plugin ecosystem

Over 200 plugins available & flexible enough to create and provide your own plugins

Logstash Loves Data

  • The more data you collect, the more you can learn

  • Logstash can handle data of all shapes and sizes

Logs and Metrics

Where everything begins

  • Can handle all types of logging data

    • Collects various web logs (ex. Apache) and application logs (ex. log4j for Java)

    • Collects syslog, networking and firewall logs in various formats

  • Can securely forward logs in conjunction with Filebeatarrow-up-right

The Web

Realizing the true utility of the World Wide Web

  • Convert HTTP requestsarrow-up-right to events

    • Use Twitter and other firehoses for social sentiment analysis

    • Webhook support for Github, Jira, and numerous other applications

    • Support for various Watcherarrow-up-right alerting use cases

  • Generate events by polling HTTP endpointsarrow-up-right as needed

    • Collect status, performance, metrics, and other data types from web application interfaces

Data Stores and Streams

Discover greater value from data you already have

Easily Enrich Everything

  • Get better knowledge from better data

    • Clean and transform data during ingestion so you can gain insights in near real-time at the time of indexing or output

    • Logstash provides various aggregation and mutation capabilities along with pattern matching, geo mapping, dynamic lookup features, and more

Last updated