Introducing "FluentBit - A multi-platform Log Processor and Forwarder"

October 1, 2018

Outline of Service

fluentbit-image

FluentBit is (like colectd, StatsD, Telegraf, snap-telemetry) a Log Forwarder and Log Processor that helps you collect data from different sources, merge them and then send them to different destinations.
 FluentBit is written in C programming language and it supports around 30 extensions.
 It is regarded as an OSS (Open Source Software) which is a computer software that allows every user to modify the program to its needs. It’s a form of open collaboration. 
 An OSS program that’s changed in that way is then free to be redistributable to anyone by the client that changed it.

https://github.com/fluent/fluent-bit/

FluentBit is mostly used by Linux users and it comes with a native metrics system that allows you to collect CPU data and the amount of memory usage by your Linux OS.
FluentBit uses a configuration file that allows the user to specify how it will behave and how routing is performed.
When FluentBit is delivering data to different destinations, output plugins inherit full TLS capabilities.

FluentBit is part of the Fluentd project ecosystem. It is widely used to solve logging problems in Cloud Native environments where Docker and Kubernetes are key components.
FluentBit can successfully deal with different sources of information, different data formats and multiple destinations all at once.
 It is created and sponsored by Treasure Data. Fluentd is complemented by FluentBit, since the first one has stronger aggregation features while the other has stronger Log Forwarding features.

||Fluentd|FluentBit| |:---|:---|:--| |Scope|Containers/Servers|Containers/Servers| |Language|C & Ruby|C| |Memory|~40MB|~450KB| |Performance|High|High| |Dependencies|Some Dependencies|Zero Dependencies| |Plugins|650+|Around 35| |License|Apache License 2.0|Apache License 2.0|

Useful Features

Event Driven:


Fluent Bit as a service is fully event-driven, it only use asynchronous operations to collect and deliver data.

Routing:


The data that comes through an Input plugin, is tagged. The Routing mechanism allows to specify where these records should be delivered. It also allows to deliver one record to multiple destinations.

Configuration:


The service use a simple and human-readable configuration file, it allow to specify how it will behave, which features to enable and how Routing is performed. Optionally Fluent Bit can run from the command line without it.

I/O Handler:


The Input/Output layer provides an abstraction that allow to perform read/write operations in an asynchronous way.

Upstream Manager:


When dealing with upstream backends like remote network services, handling TCP connections can be challenging. The Upstream manager simplify the connection process and take care of timeout/network exceptions and Keepalive states.

TLSv1.2 / Security:


When delivering data to destinations, output plugins inherit full TLS capabilities in an abstracted way. Add your certificates as required.

Extensible:


All input and output features are implemented through the plugins interface. At build enable just include what you really need. Create your own plugins easily.

Lightweight:


It have been designed to be lightweight and provide high performance. All I/O operations are made in asynchronous mode (event-driven), all data internally is serialized in binary mode.

Use Cases

Featured funded issues

Why don’t we make fund and support FluentBit developer community? 🌈