Testing
slingnode.ethereum roles ships with a comprehensive suite of infrastructure tests using Anisble Molecule project. Ansible Molecule is designed to aid in testing Ansible roles (it can be used to test full Playbooks too). You can easily use the included tests to see what the role does and how it behaves with different variables. You can also copy and adapt the tests to your own infrastructure to aid in your playbook development - whether or not using slingnode.ethereum role.
This page is meant to provide overview of the included test scenarios and how to get going with it. It is not meant as a Molecule's user guide. Please refer to the official docs if required.
Test summary
The table below outlines the summary of the test scenarios included with the role. The "purpose" column is only a high-level description, there are a lot of role variable combinations tested in each scenario.
| Scenario name | Driver/provider | Purpose |
|---|---|---|
default | Vagrant/Libvirt | Test defaults against all supported OSes |
default_single_instance | Vagrant/Libvirt | Test defaults against ad-hoc OS |
expose_ports | Vagrant/Libvirt | Test mapping ports to the host |
jwt | Vagrant/Libvirt | Test predefined JWT |
multi_tier | Vagrant/Libvirt | Test distributed deployment |
default_docker | Docker | Test defaults against d-hoc OS in Docker |
Molecule drivers
Molecule supports various "drivers" to create and manage instances during test lifecycle. The easiest one to use (and the default for Molecule) is Docker. The only requirement for using Docker driver is a running Docker daemon. Sadly, this does not work well for slingnode.ethereum role. The role uses Docker to manage client lifecycle which means that the tests have to run Docker containers in Docker containers (DIND). While this works, it requires specially prepared Docker images which are not readily available.
slingnode.ethereum test scenarios use Vagrant as the "Molecule driver" and Libvirt/KVM as the "Vagrant provider". This enables the tests to run in full unmodified VMs with widely available OS images for all supported platforms. The downside is that the Vagrant / Libvirt set up is more involved and not as readily available in CI systems as Docker. For this reason a single Docker based Molecule scenario is shipped with the role - "default_docker".
default_docker scenario
This scenario uses rockylinux8 image. Rocky8 is the only docker image we identified that was not causing issues.
Set up
This section outlines high-level steps to setup up and execute Molecule test suite:
For Docker
follow steps for your OS to install Docker engine
For Vagrant:
Install Libvirt
Install Vagrant
The installation and configuration of the per-requisites is outside of the scope of this documentation, there are many guides on the internet that explain how to set them up.
Functional testing
Molecule executes a set of user defined tests running "verify.yml" playbook during the "verify" stage. This playbook includes a set of Ansible tasks that inspect the state of the instances after the role has run (converge stage). This allows for ascertaining that the instances are in expected state (the role did what it was supposed to based on the variables). The tests included with the role range from simple checking if containers are running all the way to making Ethereum API calls and ensuring the clients are in working condition. Check "verify.yml" playbook in each scenario to see the test tasks.
Role dependencies
Molecule will install required dependencies during the "prepare" stage. Those are defined in the "prepare.yml" playbook.
Resources
In order to keep the tests DRY and ease the maintenance, various playbooks and test tasks are re-used where appropriate.
Running tests
Molecule configuration options support environment variable substitution. Any value defined in molecule.yml can be defined as env var. This makes the tests flexible and helps to keep them DRY (Don't Repeat Yourself). This enables us to define a single scenario and pass OS platform or Ethereum client names as variables to the test. slingnode.ethereum test suite heavily relies on this feature. The following environment variables can be used:
Test using Docker driver - default_docker
The clients to test can be specified using SLINGNODE_ variables. To test all clients (not all combinations though) use the following script:
Test all supported OSes in parallel - default
To test all clients (not all combinations though) against all supported OSes use the following script:
Test specific clients and specific OS
The OS to test can be specified using SLINGNODE_BOX variable. This can be any Vagrant box that supports Libvirt provider. For example:
Inspecting state of test instance during test lifecycle
Molecule enables us to execute each stage separately. This is very useful during development or anytime we want to inspect the state of an instance manually by logging in to the instance.
Last updated