Skip to main content
Version: 1.0.0

utstatusmirroring

The default interval for fetching the current state from Cribl Stream is set to 300s. Adapt inputs.conf if this interval needs to be adapted.

Sources & Destination

UTStream starts monitoring the health of sources and destinations as soon as a Cribl Stream instance is configured. Based on the state of the source or destination, a Bulletin Message is shown to users with applicable role.

If a source or destination is healed in Cribl Stream, the message gets removed. A message, that is removed by a user before the issue was fixed will be recreated.

Worker Nodes

The current Cribl Stream version (3.5) does not provide a method to track worker nodes assigned to a worker group in Cribl Stream. To still get a notification if a worker node is no longer active in a worker group, a best effort monitoring is implemented in UTStream.

The monitoring works as follows:

  • utstatusmirroring queries all worker nodes for the configured worker group
  • utstatusmirroring compares the list of retured workers with a list saved inside of Splunk
    • If a host fetched from Cribl Stream is not present in the list of Splunk add host to list
    • If a host in the list of Splunk was not fetched create a message in the Bulletin Board and remove the host from the list

This monitoring poses the following issues:

  • Worker Nodes that were never only while utstatusmirroring will not be notified
  • A message that was removed by a user will not be recreated if the host is still down
  • If a host is decommissioned, a message will be shown even if down is expected