On teh way

In the last blog, we have introduced some concepts in Logstash: the log data flow from input to filter to output, the buffer & batch etc. In the this blog, we focus on how to setup Logstash. Settings Files After installing Logstash, we can find its settings files under /etc/logstash (in linux): logstash.yml: Logstash parameter config file log4j2.properties: Logstash logging config jvm.options: Logstash JVM config startup.options: It is used by system-install script in /usr/share/logstash/bin to build the startup script. It will setup options like user, group, service name, and service decription. logstash.yml Except the normal form of yml config file functionality, the logstash.yml file also supports bash-style interpolation of environment variables in setting values. pipeline: batch: size: ${BATCH_SIZE} delay: ${BATCH_DELAY:5} node: name: "node_ ${LS_NODE_NAME} " path: queue: "/tmp/ ${QUEUE_DIR:queue} " We can al...

Today, we will learn some basic concepts of Logstash, which is a important component in ELK and is responsible for log aggregation. This serial will include following topics: Introduction: this post Configuration Application Concepts Logstash has three main conponents: input, filter and output. It is designed to follow the principle of loose coupling between components. So, it adopt the Pipe and Filter design patterns, making the plugins of Logstash very easy to be added or removed in execution pipeline of log. Pipeline Configuration When using Logstash to handle logs, we use pipeline to define the flow of logs. A simple pipeline configuration file can be looked like following: input { stdin {} } output { stdout {} } This config only defines input and output components, because filter is a optional component. There exists many different plugins can be used in different components and we will introduce them in next blog . Batch & Buffer In o...

Introduction SLF4J is a very commonly used log library in many Java application. It is favored because it allows the end-user to plug in the desired logging framework at deployment time. How is this feature implemented? How to use it? Let’s continue. Binding API & Implementation SLF4J library has two main parts: api and implementations. Its implementations are referred as bindings. In order to use SLF4J we have to both the api jar and bindings. We can replace slf4j bindings on our class path to switch logging frameworks. If we start a project with only slf4j-api-x.jar dependency, we will see the following warning: SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder" . SLF4J: Defaulting to no-operation ( NOP ) logger implementation SLF4J: See http://www .slf 4j .org /codes .html #StaticLoggerBinder for further details. This is because if no binding is found on the class path, SLF4J will default to a no-operation implementation. C...