Filebeat

Overview

The role uses the official container provided and maintained by Elastic.

• Image source: https://www.docker.elastic.co/r/beats/filebeat

• Documentation: https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html

Container

If you want to use different version of container you can modify the following variable:

filebeat_docker_image: docker.elastic.co/beats/filebeat:8.7.0

User

The container runs as root. The reason for this is that it needs permission to read Docker logs and the socket file under the following paths:

/var/lib/docker/containers
/var/run/docker.sock

Note: we don't like running container as root and it should be avoided. This will be addressed in future releases.

Volumes

The following volumes are mapped to the container:

volumes:
   - {{ observability_root_path }}/filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
   - filebeat-data:/usr/share/filebeat/data:rw
   - /var/run/docker.sock:/var/run/docker.sock:ro
   - /var/lib/docker/containers:/var/lib/docker/containers:ro

From top to bottom they are:

1. Filebeat config file

2. Working dir and registry

3. Docker socket file required to enable Filebeat Autodiscovery

4. Docker logs location on the host

Data persistence

Filebeat container uses a named volume (refer to the section above) to persist the registry data. The registry is what Filebeat uses to know where it finished harvesting the log files. Without persistent registry Filebeat would forward all logs every time the container got recreated.

Deleting data

If you want to forward all logs present on the server (for example if you started from scratch with the ELK container) you will need to execute the following commands on the server:

docker kill filebeat 
docker container rm filebeat 
docker volume rm agents_filebeat-data

Configuration

By default the role configures Filebeat to only forward logs generated by the Ethereum client containers. The configuration utilized Filebeat Autodiscovery feature - official documentation https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html

You can can review the configuration here: https://github.com/SlingNode/slingnode-ansible-ethereum-observability/blob/master/templates/filebeat/filebeat.yml.j2

The configuration uses docker labels which are configured by the slingnode.ethereum role, refer to the documentation for implementation details: https://docs.slingnode.com/slingnode.ethereum/logging.

By default Filebeat will automatically discover logs generated by any of the supported clients. You can customize the configuration by editing the following variable's src key and providing your own config file:

# Filbeat configuration template
filebeat_config_template:
  - src: templates/filebeat/filebeat.yml.j2
    dest: "{{ observability_root_path }}/filebeat/filebeat.yml"

The configuration will work without any modification for single server and distributed deployments. In a single server deployment Filebeat will connect to Logstash over Docker network using ELK container name, in distributed deployments the tartget will be set to the IP address of the monitoring server.

output.logstash:
  hosts: ["{{ container_name_elk if single_server_deployment else monitoring_server_host }}:{{ logstash_port }}"]
  enabled: true
  timeout: 15