Alertmedia outage history

We have identified an issue with one of our messaging deliver providers (Twilio) that is impacting the delivery of some SMS messages and Voice Delivery from the platform (notifications, threat warnings, incident response, travel risk warnings). Our Engineering team is investigating and reviewing impacts and potential mitigations. We are working with Twilio to monitor the issue. Twilio's status page: https://status.twilio.com/ Roughly 10% of SMS message are failing.

We continue to monitor the issue with Twilio. The issue of Voice deliverability has been identified and a fix is being deployed. The issue for SMS delays has been identified and engineers are working on a fix. Twilio's status page: https://status.twilio.com/

Fixes have been applied for SMS and Voice and we are starting to see a reduction in error rates. Functionality for Voice and SMS returning to normal. We will continue to monitor the issue. Twilio's status page: https://status.twilio.com/

This incident has been resolved.

We are aware of some users experiencing issues with login in via mobile for both Android and iOS systems. The web app is working as expected. The team is investigating

We are continuing to investigate the issue

The issue has been identified and we are working to resolve this issue for customers as soon as possible

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We have identified an issue that is causing some user imports to fail under specific conditions and are working to implement a fix.

We are currently deploying a fix for this issue.

This incident has been resolved.

We are currently monitoring the ongoing issues with Verizon's (US) mobile network. SMS and Voice messages to Verizon phone numbers may be delayed or fail to deliver while these issues persist.

The outage is resolved, according to Verizon.

We are currently investigating reports of intermittent latency in the app and on login

The issue has been identified and we are working on a fix.

A fix has been implemented and we are monitoring the results.

This incident has been resolved

We are investigating an incident that is preventing users from logging in.

Systems are back to normal, users should be able to login. We are monitoring the issue

This incident has been resolved.

We have identified an issue when attempting to load the Threat History and Safety maps in our dashboard and are working on a fix.

A fix has been implemented and we are monitoring the results. Both Threat History and Safety Maps are responsive.

We’re currently investigating reports of SFTP upload failures affecting some customers. Our engineering team is investigating the issue.

A fix has been applied and we are monitoring the environment.

This incident has been resolved.

AWS services are recovering and we are mitigating impact where possible. The service responsible for notification sends is degraded, delaying notifications, threat warnings, and incident response tasks.

Notification sends are processing. Recovery in progress.

Updated impacted components

Notification sends and threat warnings are operational. FTP imports are not currently processing.

FTP imports are not currently processing. Employee Safety Monitoring is down.

FTP imports are operational in the NA2 production environment. They are not processing in the NA1 production environment.

FTP Imports are not processing in either environment

We are experiencing delayed notifications due to downstream providers. FTP Import performance is still degraded for both environments

There remain issues related to SFTP (AWS Transfer Family Service) that are still causing import issues.

We are continuing to see delays in sending Threat Warnings

Threat Warnings are operational and we are continuing to monitor the situation for Imports

All systems are back to operational status. We will continue monitor the situation.

This incident has been resolved.

Escalation messages are unable to send for some customers. All other message types are able to send. Our team is investigating the issue.

The issue has been identified and a fix is being applied.

A fix has been implemented and we are monitoring the results.

The incident has been resolved. Escalations function as expected for all customers.

Threat warnings are degraded for some customers and we are working to restore full service. Some threat warnings were not delivered due to missing data during processing. We’ve identified the issue and are actively backfilling the data to ensure alerts are complete and accurate. Full restoration will take a few hours.

Threat warnings are degraded for some customers and we are working to restore full service. Some threat warnings were not delivered due to missing data during processing. We’ve identified the issue and are actively backfilling the data to ensure alerts are complete and accurate. Full restoration will take a few hours.

Threat warnings are degraded for some customers and we are working to restore full service. Some threat warnings were not delivered due to missing data during processing. We’ve identified the issue and are actively backfilling the data to ensure alerts are complete and accurate. Full restoration will take a few hours.

The backfill of the data is almost complete. Full threat warning coverage has been restored for the majority of customers.

The backfill process has completed and full threat warning coverage has been restored for all customers.

We have identified an issue that is preventing some users from being able to log in and are working on a fix.

This issue has been resolved and all users should be able to log in as normal.

We are currently investigating an issue that is causing some users to be unable to access the AlertMedia Dashboard.

We are monitoring the issue, services are restored.

This incident has been resolved.

We are currently investigating an issue that is causing user import via SFTP to be inaccessible for some customers.

We have implemented a fix for the SFTP issues we were experiencing and are monitoring the system as performance improves.

This incident has been resolved.

We are currently investigating an issue that is causing slowness in the AlertMedia systems

The system should be coming back to normal performance levels. We are continuing to investigate the issue

The incident has been resolved, all systems should be running as expected.

We are currently investigating an issue that is causing some voice notifications and incoming calls to Emergency Lines to fail.

We have identified the issue causing some voice notifications and incoming calls to Emergency Lines to fail and are working with our partners to resolve the issue.

This incident has been resolved. Voice notifications and incoming calls to Emergency lines have returned to normal.

Users are experiencing intermittent delays in loading map layers. The root cause is performance issues with the ArcGIS service (see https://status.arcgis.com/). We are monitoring the issue.

We are seeing recovery of our Visual Intelligence systems as ArcGIS reports the issue on their end has stabilized.

We are currently investigating an issue that is causing users to be unable to access the AlertMedia Dashboard.

The AlertMedia dashboard has returned to a healthy operational state. We will continue to monitor the fix in place to ensure continued performance.

This incident has been resolved.

We are investigating an issue with Imports and group updates.

The issue has been identified and we are working on a fix

We are continuing to work on implementing the fix for ongoing performance issues with imports.

The fix has been deployed to production, and we're seeing the system recover. We will continue to monitor and update this page once all systems are confirmed to be recovered and stable.

This incident has been resolved.