Announcing v4 of the Sensu Puppet module

The following is a guest post from Trey Dockendorf of Tailored Automation. He is a maintainer and the top contributor to the Sensu Puppet module. Trey has been developing Puppet modules for 10 years and specializes in system administration for high-performance computing.

It’s time for the next major release of the Sensu Puppet module, which unlocks many new capabilities of Sensu Go. In order to take advantage of the new Sensu Go features, v4.0.0 will introduce some breaking changes, but they’re worth it! These changes will align the Puppet module parameters and the Sensu Go data types, making it much easier for everyone to operate Sensu Go. And, with the Sensu Puppet module reaching over three million downloads (not to mention the fact it’s the most popular partner module on Puppet Forge!), we figured we’d take this opportunity to share what’s new.

CLI Class

In order to be more flexible and allow the deployment of sensuctl on more than the backend nodes, the logic for installing and configuring sensuctl has been moved to a public class. This means the Sensu Go CLI can be managed on systems that are not Sensu Go backends. This removed a limitation of the Sensu Puppet module where it was only possible to define resources like sensu_check on the backend. It is now possible to include the sensu::cli class on agents and have those agents define their own resources, which allows for distributed configuration of Sensu Go resources (previously this was only possible with Exported Resources).

One benefit of defining resources like checks on the agents is it reduces the size of the catalog for the backend which can improve the performance of Puppet when dealing with large environments. This can be achieved by including the sensu::cli class on agents and defining resources like sensu_check on the agent that then get added to the backend.

One drawback to defining resources on agents is you can no longer utilize resource purging. The catalog on the backend must contain all resources if you wish to purge unmanaged resources. This can be achieved by defining resources all on the backend or using exported resources where the agent exports a resource like a sensu_check that are then collected on the backend.

The sensu::cli class is going to be required in order to use the new sensu Bolt inventory plugin, as any node executing the inventory query will need sensuctl.

Bolt inventory plugin

Beginning with Bolt inventory v2, there is now support for plugins to define Bolt targets for commands. The sensu-puppet module adds a sensu plugin that can populate the Bolt v2 inventory targets using data from Sensu Go! This plugin allows the execution of Bolt tasks on nodes where the groups in Bolt can be defined using existing data already stored in Sensu Go.

In order to use the Sensu inventory plugin, the host executing the bolt command must have the sensu::cli class.

The following shows an example of configuring the Bolt inventory with two groups. The linux group pulls Sensu Go entities in the default namespace with the linux subscription. The linux-qa group is the same as linux group but instead pulls entities from the qa namespace.

version: 2
groups:
  - name: linux
    targets:
      - _plugin: sensu
        namespace: default
        subscription: linux
  - name: linux-qa
    targets:
      - _plugin: sensu
        namespace: qa
        subscription: linux

If your entities have more than one network interface it may be necessary to specify the order of interfaces to search when looking for the IP address:

version: 2
groups:
  - name: linux
    targets:
      - _plugin: sensu
        namespace: default
        subscription: linux
        interface_list:
          - eth0
          - eth1

The following rules for interface matching determine the value used for the Bolt inventory URI.

1. If interface_list is defined then find first match
2. If interface_list is not defined and only one interface, use that as IP address
3. If interface_list is not defined and more than one interface, use name

API providers

The initial release of sensu-puppet v3 only supported adding resources like checks by executing sensuctl on the host running sensu-backend. All the core resources now have a provider that manages resources using the Sensu Go API. The new provider should be used if you need to define resources on a host other than the Sensu backend and do not want to install sensuctl on those hosts.

The new provider can be used by setting the provider parameter on a resource to sensu_api. The default provider is still sensuctl but it’s possible to change the provider when defining a resource. For example, the following will create a check that can be defined on an host that’s not the sensu-backend.

include ::sensu::api
sensu_check { "check-cpu-${facts['hostname']}":
  ensure        => 'present',
  command       => 'check-cpu.sh -w 75 -c 90',
  interval      => 60,
  subscriptions => ["entity:${facts['hostname']}"],
  provider      => 'sensu_api',
}

The sensu::api class is required in order to configure the credentials and URL used to communicate with the Sensu backend API.

The API URL, username, and password used for the API are set in the sensu class and can be set easily with Hiera:

sensu::api_host: sensu-backend.example.com
sensu::api_port: 8080
sensu::username: admin
sensu::password: supersecret
sensu::old_password: 'P@ssw0rd!'

Improvements to native Puppet types

Several Puppet types had their properties changed in order to match the Sensu Go specifications for the resources managed by the Puppet types. This should allow someone using the Puppet types to more easily understand what each property does based on the Sensu Go documentation as the Puppet types will map directly to a Sensu Go resource. This also simplified the Ruby code used to define the Puppet types.

Here are the details on the updates:

• Replace sensu_check proxy_requests* properties with proxy_requests Hash
• Replace sensu_entity deregistration_handler with deregistration Hash
• Replace sensu_handler socket_* properties with socket Hash
• Refactor sensu_ldap_auth and sensu_ad_auth on how properties are defined
• Move server_binding, server_group_search, and server_user_search into servers property

Replace ad-hoc types with Bolt tasks

The Sensu Go event and silenced resources were moved from Puppet types to Bolt tasks. These resources are more ad-hoc and do not necessarily define a system state which is better suited for Bolt.

These new Bolt tasks must be run on nodes (--nodes) that have sensuctl configured by the sensu::cli Puppet class.

Examples of new Bolt tasks:

Resolve an event

bolt task run sensu::event action=resolve entity=sensu_agent check=test --nodes sensu_backend

Delete an event

bolt task run sensu::event action=delete entity=sensu_agent check=test --nodes sensu_backend

Create a silenced resource

bolt task run sensu::silenced action=create subscription=entity:sensu_agent expire_on_resolve=true --nodes sensu_backend

Delete a silenced resource

bolt task run sensu::silenced action=delete subscription=entity:sensu_agent --nodes sensu_backend

PDK support

The sensu-puppet module now supports the Puppet Development Kit (PDK) to make it easier for module developers to run tests and validations without needing to install a full-blown Ruby environment. The following are some examples of testing the sensu-puppet module using the PDK.

Run module validation

pdk validate

Run module unit tests

pdk test unit

Run acceptance tests that require Docker

# to set up dependencies
pdk bundle install

# for any platform you want to test
BEAKER_set=centos-7 BEAKER_PUPPET_COLLECTION=puppet6 pdk bundle exec rake beaker

The use of the pdk bundle is currently experimental but does work to run acceptance tests.

The possible values for BEAKER_set and BEAKER_PUPPET_COLLECTION can be found in sensu-puppet .travis.yml

Parameter changes

Several changes were made to class parameters to support the new sensu::cli class. Moving and renaming parameters to the sensu class was done so that defining a parameter like sensu::api_host could be done centrally and used by both the sensu::cli and sensu::backend classes without having to define those values multiple times.

Changes to the location of the agent password streamlines changing the agent password so one parameter modifies both agent and backend resources.

Detailed parameter changes:

• Move sensu::backend::cli_package_name to sensu::cli::package_name
• Move sensu::backend::sensuctl_chunk_size to sensu::cli::sensuctl_chunk_size
• Move sensu::backend::url_host to sensu::api_host
• Move sensu::backend::url_port to sensu::api_port
• Move sensu::backend::password to sensu::password
• Move sensu::backend::old_password to sensu::old_password
• Move sensu::backend::agent_old_password to sensu::agent_old_password
• Move sensu::backend::sensuctl_chunk_size to sensu::cli::sensuctl_chunk_size

The following parameters were moved from sensu::backend class to sensu::resources class. Example: sensu::backend::checks becomes sensu::resources::checks. The parameters are the following:

• ad_auths
• assets
• bonsai_assets
• checks
• cluster_members
• cluster_role_bindings
• cluster_roles
• configs (removed)
• entities
• etcd_replicators
• filters
• handlers
• hooks
• ldap_auths
• mutators
• namespaces
• oidc_auths
• role_bindings
• roles
• users

Going forward

We hope this got you excited about the new and improved Sensu Puppet module, as well as gave you a sense of how to start automating your monitoring with Sensu Go and Puppet. For further reading, check out the Sensu Puppet repo — which includes example use cases with Slack, PagerDuty, Vault, and more, plus a super easy-to-use Vagrant setup — and head to Puppet Forge to try out the Sensu Puppet module for yourself!