The changes we implemented to clear the bottleneck in message processing improved throughput, messages in the backlog were processed, and new messages should no longer be delayed.
Posted Jan 18, 2024 - 14:12 EST
Monitoring
An unexpectedly large amount of messages sent into our systems lead to a bottleneck in the message processing, causing delays to all our customers. We have implemented rules to help clear this bottleneck, and are monitoring the situation closely.
Posted Jan 18, 2024 - 13:06 EST
Identified
We've identified the cause of the issue, and have taken actions to restore live processing of messages. Message queues are gradually draining, but updates may still be delayed.
Posted Jan 18, 2024 - 12:50 EST
Investigating
We are currently investigating an issue with a delay in processing information relating to visit updates.
Posted Jan 18, 2024 - 12:40 EST
This incident affected: SMS (SMS Notifications, Welcome & Invite SMS Messages, Customized Scheduled SMS Messages), ERAdvisor (Patient Service Requests, AI Wait Times, Labs & Imaging), and CareAdvisor (Patient Service Requests, Labs & Imaging, Recover).