Vital - Degraded trackboard and API performances – Incident details

All systems operational

Degraded trackboard and API performances

Resolved
Major outage
Started over 6 years agoLasted about 6 hours
Updates
  • Resolved
    UTC
    Resolved

    This issue has been resolved. Thank you for your patience.

  • Update
    UTC
    Update

    We are continuing to monitor for any further issues.

  • Update
    UTC
    Update

    The backlog of messages have finished processing, and both the trackboard and APIs have recovered in their performance. We will continue monitoring to ensure stability.

  • Update
    UTC
    Update

    We are continuing to review the accuracy of our trackboard. API performance has been restored, and are currently working on clearing out the backlog of messages.

  • Monitoring
    UTC
    Monitoring

    A fix has been deployed and we are monitoring the backlog of estimations to ensure it is cleared.

  • Identified
    UTC
    Identified

    The team has identified a potential cause for the degraded API services, and is currently working on developing a potential fix. We will continue to provide updates as often as possible, and provide additional details as they become available.

  • Update
    UTC
    Update

    We are continuing to investigate this issue.

  • Investigating
    UTC
    Investigating

    On 2019-12-04 at 10:00 AM EST, we noticed severe performance issues with our trackboard and the data being used to populate it. We are currently investigating this issue, and will provide updates as often as possible