Vital - Notice history

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The changes we implemented to clear the bottleneck in message processing improved throughput, messages in the backlog were processed, and new messages should no longer be delayed.

An unexpectedly large amount of messages sent into our systems lead to a bottleneck in the message processing, causing delays to all our customers. We have implemented rules to help clear this bottleneck, and are monitoring the situation closely.

We've identified the cause of the issue, and have taken actions to restore live processing of messages. Message queues are gradually draining, but updates may still be delayed.

This incident has been resolved.

The system has recovered and messages are being sent again. We'll continue to monitor the situation before marking this incident as resolved.