Monitoring with Devo
Devo offers a generic collector monitoring activeboard.
To query SQS messages successfully processed by SQS collectors, use
from devo.collectors.out where toktains(msg,"Acked message receiptHandle"), toktains(collector_image,"aws_sqs_if") group every 5m by collector_name, job_id select count() as messages_successfully_processed
The number of SQS messages may be more or less than the number of events sent to Devo. An inactivity alert that keeps track of job_id will inform you if messages are not being processed.
Monitoring with AWS
CloudWatch allows you to create alarms for
Messages sent below threshold, which can occur if data is no longer entering the S3 bucket.
Number of messages visible is too high, which can occur if the SQS collector was turned off.