This repository contains all neccessary bits to run the M3 Fluent Bit workshop stack locally on either Mac or Windows. Before getting started with the workshop, make sure you have met the following prerequisites below.
Prerequisites
Download Docker Desktop for Mac or Windows. Docker Compose will be automatically installed. On Linux, make sure you have the latest version of Compose.
Adjust “Resources” in Docker to have at least 4 GB of memory.
Stack overview
The stack consists of the latest versions of these components:
Prometheus instance is configured to scrape itself and the Fluentbit agent’ output compatible w/ Prometheus exporter format. The M3 Coordinator instance takes all read and write requests, and then distributes them to the cluster of M3DB nodes.
The M3 Query instance is used to query all data from the M3DB node(s) via Grafana.
At the start, Grafana will be configured with 2 different data sources - Prometheus and M3Query:
Once you see the following output (with code 0 at the end), the stack is configured and ready to be used:
provisioner_1 | Waiting until shards are marked as available
provisioner_1 | Provisioning is done.
provisioner_1 | Prometheus available at http://localhost:9090
provisioner_1 | Prometheus replica is available at http://localhost:9091
provisioner_1 | Grafana available at http://localhost:3000
m3-workshop_provisioner_1 exited with code 0
Logs of the provisioner process can be seen either by following the output of the docker-compose up or by running the following command: docker-compose logs provisioner
Step 3: Open up Grafana
Once the stack is up and running, login into the Grafana using admin:admin credentials and then head to the Explore tab.
Note: Press “skip screen” in Grafana when prompting you to set a password.
Step 4: Explore the Prometheus data sources in Grafana
In the Explore tab of Grafana, you will see three datasources - default, Prometheus, and M3Query.
Prometheus instance is scraping itself and the node_exporter metrics of fluentbit container being exported via one of the FluentBit outputs. FluentBit node_exporter metrics also end up
in the M3DB node via Prometheus remote write API. You can use up{} PromQL query in the Grafana to inspect which datasources return the results.
Step 5 - Spinning down the stack
Press Ctrl+C to interrupt the already running docker-compose up process, or run the following command:
$:~ docker-compose down
Recommended step: it leaves container disks intact. If you want to get rid of the data as well, run the following command:
FluentCon M3 + FluentBit Workshop
About
This repository contains all neccessary bits to run the M3 Fluent Bit workshop stack locally on either Mac or Windows. Before getting started with the workshop, make sure you have met the following prerequisites below.
Prerequisites
Download Docker Desktop for Mac or Windows. Docker Compose will be automatically installed. On Linux, make sure you have the latest version of Compose.
Adjust “Resources” in Docker to have at least 4 GB of memory.
Stack overview
The stack consists of the latest versions of these components:
Prometheus instance is configured to scrape itself and the Fluentbit agent’ output compatible w/ Prometheus exporter format.
The M3 Coordinator instance takes all read and write requests, and then distributes them to the cluster of M3DB nodes.
The M3 Query instance is used to query all data from the M3DB node(s) via Grafana.
At the start, Grafana will be configured with 2 different data sources - Prometheus and M3Query:
List of container instances
Instructions for the workshop
Step 1: Go to the M3 Workshop Fluentcon repo and clone the repo onto your local machine:
Link to repo: https://github.com/fluent/m3-workshop-fluentcon
Step 2: Start up the stack via Docker-Compose
Run the following command:
$:~ docker-compose upOnce you see the following output (with code 0 at the end), the stack is configured and ready to be used:
Logs of the
provisionerprocess can be seen either by following the output of thedocker-compose upor by running the following command:docker-compose logs provisionerStep 3: Open up Grafana
Once the stack is up and running, login into the Grafana using
admin:admincredentials and then head to the Explore tab.Grafana: http://localhost:3000
Note: Press “skip screen” in Grafana when prompting you to set a password.
Step 4: Explore the Prometheus data sources in Grafana
In the Explore tab of Grafana, you will see three datasources - default, Prometheus, and M3Query. Prometheus instance is scraping itself and the
node_exportermetrics offluentbitcontainer being exported via one of the FluentBit outputs. FluentBitnode_exportermetrics also end up in the M3DB node via Prometheus remote write API. You can useup{}PromQL query in the Grafana to inspect which datasources return the results.Step 5 - Spinning down the stack
Press
Ctrl+Cto interrupt the already runningdocker-compose upprocess, or run the following command:$:~ docker-compose downRecommended step: it leaves container disks intact. If you want to get rid of the data as well, run the following command:
$:~ docker-compose down -v