# Logstash Basics

> Let's learn about Logstash
>
> Reference: [Logstash docs](https://www.elastic.co/guide/en/logstash/current/introduction.html)

\ <br>

## What is Logstash?

<br>

### Logstash

* An open source data collection engine with **real-time pipeline** capabilities
* Can dynamically **unify** different data sources and **normalize** data to a specified destination
* Can **cleanse** and **democratize** data

\ <br>

## The Power of Logstash

<br>

### 1. Powerful Ingestion for Elasticsearch and More

A horizontally scalable data processing pipeline leveraging the synergy of Elasticsearch and Kibana

<br>

### 2. Pluggable pipeline architecture

Mix & match and orchestrate various inputs, filters, and outputs harmoniously in the pipeline

<br>

### 3. Community-extensible and developer-friendly plugin ecosystem

Over 200 plugins available & flexible enough to create and provide your own plugins

<br>

\ <br>

## Logstash Loves Data

* The more data you collect, the more you can learn
* Logstash can handle data of all shapes and sizes

<br>

### Logs and Metrics

Where everything begins

* Can handle all types of **logging data**
  * Collects various **web logs** (ex. Apache) and **application logs** (ex. log4j for Java)
  * Collects syslog, networking and firewall logs in various formats
* Can securely forward logs in conjunction with [Filebeat](https://www.elastic.co/products/beats/filebeat)

<br>

### The Web

Realizing the true utility of the World Wide Web

* Convert [HTTP requests](https://www.elastic.co/guide/en/logstash/7.10/plugins-inputs-http.html) to events
  * Use Twitter and other **firehoses** for social sentiment analysis
  * **Webhook** support for Github, Jira, and numerous other applications
  * Support for various [Watcher](https://www.elastic.co/products/x-pack/alerting) alerting use cases
* Generate events by polling [HTTP endpoints](https://www.elastic.co/guide/en/logstash/5.4/plugins-inputs-http_poller.html) as needed
  * Collect **status**, **performance**, **metrics**, and other data types from web application interfaces

<br>

### Data Stores and Streams

Discover greater value from data you already have

* Better understand data from related databases or NoSQL stores through the [JDBC](https://www.elastic.co/guide/en/logstash/5.4/plugins-inputs-jdbc.html) interface
* Consolidate various data streams provided by messaging queues such as Apache [Kafka](https://www.elastic.co/guide/en/logstash/5.4/plugins-outputs-kafka.html), [RabbitMQ](https://www.elastic.co/guide/en/logstash/5.4/plugins-outputs-rabbitmq.html), [Amazon SQS](https://www.elastic.co/guide/en/logstash/5.4/plugins-outputs-sqs.html), and [ZeroMQ](https://www.elastic.co/guide/en/logstash/5.4/plugins-outputs-zeromq.html)

\ <br>

## Easily Enrich Everything

* Get better knowledge from better data
  * **Clean** and **transform** data during ingestion so you can gain **insights** in near **real-time** at the time of indexing or output
  * Logstash provides various aggregation and mutation capabilities along with **pattern matching**, **geo mapping**, **dynamic lookup** features, and more


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://chloe-codes1.gitbook.io/til/infra/elk/03_logstash_basics.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.