История сбоев Duo Security

1. 3 сентября 2026 г. в 22:04 UTCinvestigating

   Мы расследуем частичное отключение, влияющее на аутентификацию токенов Duo для клиентов в пострадавших развертываниях в США. Клиенты, использующие аппаратные токены, могут испытывать периодические сбои аутентификации. Основываясь на текущей информации, повторная попытка аутентификации, как ожидается, будет успешной. Ожидается, что другие методы аутентификации Duo не будут затронуты и могут использоваться в качестве альтернативных методов, если они настроены и доступны. Наша инженерная команда активно исследует и приступила к мерам по смягчению последствий. По мере поступления новой информации будут предоставляться обновления и дополнительные сведения. Мы ценим ваше терпение и понимание в это время.

2. 3 сентября 2026 г. в 22:30 UTCmonitoring

   Мы видели, что уровень отказов для аутентификации токенов Duo снизился на затронутых развертываниях США. Клиенты, использующие аппаратные токены, могут по-прежнему испытывать периодические сбои аутентификации, пока мы продолжаем отслеживать. Ожидается, что повторная попытка аутентификации будет успешной. Ожидается, что другие методы аутентификации Duo не будут затронуты и могут использоваться в качестве альтернативных методов, если они настроены и доступны. Мы будем продолжать следить за стабильностью обслуживания и предоставлять дополнительные обновления по мере необходимости.

3. 3 сентября 2026 г. в 23:24 UTCresolved

   Этот вопрос решен. Аутентификация токенов Duo теперь нормально работает на затронутых развертываниях США. Ожидается, что другие методы аутентификации Duo не будут затронуты. Спасибо за ваше терпение. Если вы продолжаете испытывать проблемы, пожалуйста, свяжитесь с Duo Support.

Автоматический перевод официального обновления инцидента.

1. 10 августа 2026 г. в 20:34 UTCinvestigating

   We are investigating an issue affecting the signup service. New account registrations are temporarily unavailable. Our team is actively investigating and working to restore normal service. We apologize for any inconvenience and appreciate your patience.

2. 10 августа 2026 г. в 21:04 UTCidentified

   We have identified the issue affecting new account registrations. Our team is working on a fix and expects to release it soon. We apologize for any inconvenience and appreciate your continued patience.

3. 10 августа 2026 г. в 22:46 UTCresolved

   This incident has been resolved. New Duo account registrations are now functioning normally. Thank you for your patience. If you continue to experience issues, please contact Duo Support.

1. 8 августа 2026 г. в 21:13 UTCidentified

   We have identified an issue causing phone call authentication failures for customers in the United States. This is related to a third-party carrier disruption. We are actively monitoring the carrier issue and working with our provider as needed. Please check back here or subscribe to updates for any changes.

2. 8 августа 2026 г. в 23:04 UTCmonitoring

   We are seeing recovery for the issue related to a third-party carrier disruption that caused phone call authentication failures for customers in the United States. We are actively monitoring phone call authentication delivery and will provide another update once the issue is fully resolved.

3. 9 августа 2026 г. в 00:07 UTCresolved

   Phone call authentication delivery for customers in the United States has returned to normal operation. If customers continue to experience issues, please contact Duo Support.

1. 23 июля 2026 г. в 18:02 UTCinvestigating

   Мы расследуем вопрос, касающийся доставки VoIP и SMS сообщений для пользователей в Израиле. Проблема связана с восходящим перевозчиком. Наша команда работает с нашими поставщиками, чтобы исследовать и восстановить нормальное обслуживание. Там, где это возможно, пользователи могут использовать альтернативные методы аутентификации, такие как Duo Push. Мы будем предоставлять обновления по мере поступления дополнительной информации.

2. 23 июля 2026 г. в 18:39 UTCmonitoring

   Мы внедрили изменения для решения проблемы доставки SMS и VoIP, затрагивающей пользователей в Израиле. Доставка SMS и VoIP восстановилась, и мы проводим мониторинг, чтобы подтвердить постоянную стабильность.

3. 24 июля 2026 г. в 15:53 UTCmonitoring

   Ставки ошибок при доставке SMS и VoIP в Израиле снизились после внесения изменений. Мы продолжаем следить за поведением сервиса и расследовать сообщения о периодических сбоях, затрагивающих некоторых клиентов. Мы предоставим еще одно обновление по мере поступления дополнительной информации.

4. 27 июля 2026 г. в 20:41 UTCresolved

   Проблема доставки SMS и VoIP, затрагивающая пользователей в Израиле, решена. Если клиенты продолжают испытывать проблемы, пожалуйста, свяжитесь с Duo Support.

Автоматический перевод официального обновления инцидента.

1. 7 июля 2026 г. в 23:15 UTCinvestigating

   We are investigating a third-party carrier issue affecting VoIP and SMS services in the Australia region. Customers in Australia may experience delays or failures when using phone call or SMS authentication. Other Duo authentication methods, such as Push Delivery, are not expected to be impacted and can be used as an alternate method of authentication if available. Our team is monitoring the carrier issue and working with the provider as needed. Updates and additional details will be provided as new information becomes available. We appreciate your patience and understanding during this time.

2. 8 июля 2026 г. в 01:22 UTCmonitoring

   The third-party carrier has indicated that the issue affecting VoIP and SMS services in the Australia region has been resolved. Customers in Australia should see improved delivery for phone call and SMS authentication. Other Duo authentication methods, such as Push Delivery, remain available as alternate authentication methods if configured. We are continuing to monitor service stability and will provide additional updates when the incident is fully resolved.

3. 8 июля 2026 г. в 15:01 UTCresolved

   The third-party carrier issue affecting VoIP and SMS services in the Australia region has been resolved. Phone call and SMS authentication services are operating normally. We appreciate your patience and understanding while we addressed this issue.

1. 23 июня 2026 г. в 03:58 UTCidentified

   We have identified the issue related to SMS for UAE region and are working to resolve it. Users have the option to use alternative auth methods, like Duo Push or VoIP. Please check back here or subscribe to updates for any changes.

2. 23 июня 2026 г. в 08:29 UTCmonitoring

   We have mitigated the SMS delivery issue affecting the UAE region. We are actively monitoring the situation and will provide a final update once the issue is fully resolved.

3. 23 июня 2026 г. в 14:21 UTCresolved

   The SMS delivery issue affecting the UAE region has been resolved.   Our team has verified that SMS services are operating normally and full functionality has been restored. Thank you for your patience while we resolved this issue

1. 10 июня 2026 г. в 14:17 UTCinvestigating

   We are currently investigating an issue affecting AI Assistant functionality in the US-W1 (US West-1) region. Users may experience errors or be unable to access AI Assistant features. Our engineering teams are actively investigating the issue and working to restore normal service as quickly as possible. Please check back here or subscribe to updates for the latest information.

2. 10 июня 2026 г. в 15:35 UTCresolved

   The issue affecting AI Assistant functionality in the US-W1 region has been resolved. Our team has verified that services are operating normally, and monitoring has confirmed system stability. Thank you for your patience while we resolved this issue.

1. 18 мая 2026 г. в 18:37 UTCinvestigating

   We are currently investigating an issue with AI Assistant not functioning on all US deployments and are working to correct the issue as soon as possible. Please check back here or subscribe to updates for any changes.

2. 18 мая 2026 г. в 19:27 UTCmonitoring

   We have implemented mitigation measures and are observing improved AI Assistant performance. We are continuing to monitor success rates and overall service stability.

3. 18 мая 2026 г. в 20:30 UTCresolved

   The issue affecting AI Assistant functionality across all US deployments has been resolved, and full functionality has been restored. We will continue to monitor service performance to ensure stability.

1. 13 мая 2026 г. в 13:22 UTCinvestigating

   We are currently investigating an issue causing limited authentication failures on DUO57 and are working to correct the issue as soon as possible.

2. 13 мая 2026 г. в 14:52 UTCidentified

   We have identified the cause of the issue causing authentication failures on DUO57 with the Duo Core Authentication Service and are actively working to restore service.

3. 13 мая 2026 г. в 15:37 UTCmonitoring

   We have implemented a fix for the issue causing authentication failures on DUO57 with the Duo Core Authentication Service. We will continue to monitor system performance and authentication success rates and will provide further updates once the incident is fully resolved.

4. 13 мая 2026 г. в 16:03 UTCresolved

   The issue causing authentication failures on DUO57 with the Duo Core Authentication Service is now resolved and full functionality has been restored.

1. 7 мая 2026 г. в 14:17 UTCinvestigating

   We are investigating an issue causing failures when administrators located in the EU attempt to access the AI Assistant for the first time. Any administrators who have used the Assistant before should still be able to at this time

2. 7 мая 2026 г. в 16:30 UTCidentified

   We’ve identified a problem impacting first-time access to the AI Assistant for administrators in the EU and are actively working to restore normal access. We’ll continue to share updates as progress is made.

3. 7 мая 2026 г. в 22:40 UTCresolved

   The issue affecting first-time access to the AI Assistant for administrators in the EU has been fully resolved, and normal functionality has been restored.

1. 30 апреля 2026 г. в 17:54 UTCidentified

   We are currently addressing an issue impacting desktop authenticator enrollments .A subset of customers may experience intermittent delays or failures during desktop authenticator enrollment. Mitigation actions are in progress and are helping to reduce impact. Core authentication, SSO, and delivery services  remain fully operational.

2. 30 апреля 2026 г. в 17:57 UTCmonitoring

   Mitigation actions have been implemented. The service has stabilized and recovery is underway. We will continue to monitor to ensure full resolution.

3. 30 апреля 2026 г. в 18:07 UTCresolved

   The issue has been resolved. Services have returned to normal operation, and customers should no longer experience delays or failures. We apologize for any inconvenience caused and thank you for your patience.

1. 28 апреля 2026 г. в 14:22 UTCinvestigating

   We are investigating elevated authentication latency affecting customers . Authentication requests are continuing to succeed at this time, but customers may experience slower-than-normal response times and degraded performance during login attempts. Our team is actively working to identify the cause and restore normal performance as quickly as possible.

2. 28 апреля 2026 г. в 14:56 UTCmonitoring

   We have applied mitigation steps and are seeing improved authentication performance for customers in the deployment. Authentication latency has decreased, and the service is recovering. Our team is continuing to monitor the environment closely to ensure stability while we investigate the underlying cause.

3. 28 апреля 2026 г. в 15:26 UTCresolved

   The issue causing elevated authentication latency has been fully resolved. Authentication performance has returned to normal levels, and customers should no longer experience delays during login attempts.

1. 18 апреля 2026 г. в 04:30 UTCinvestigating

   We are currently detecting an issue with carriers in the US region for SMS delivery. We are currently working with providers to restore the service. In the meantime, please use other means of authentication such as Duo Push.

2. 18 апреля 2026 г. в 05:39 UTCinvestigating

   We are continuing to investigate the SMS delivery issue impacting all customers in the US region. Push notifications will continue to work as expected. Further updates will be provided as more information becomes available.

3. 18 апреля 2026 г. в 07:09 UTCresolved

   The SMS delivery issue impacting customers in the US region has been resolved. SMS passcode authentication is now functioning normally. We apologize for any inconvenience caused and thank you for your patience.

1. 22 марта 2026 г. в 22:13 UTCinvestigating

   We are currently observing a recurrence of the issue causing VOIP authentication failures in all deployments in the US region. We are working to correct the issue as soon as possible. While we work to resolve this, please use alternative authentication methods such as Duo Mobile Push, SMS passcodes etc. A full list of available authentication methods can be found here: https://duo.com/docs/policy#authentication-methods

2. 23 марта 2026 г. в 00:01 UTCmonitoring

   We are now observing high rates of success of VoIP authentication. The issue appears to be resolved but we will continue to monitor the situation.

3. 23 марта 2026 г. в 02:23 UTCresolved

   VoIP authentication has returned to normal operation with sustained success rates observed.

1. 22 марта 2026 г. в 17:18 UTCinvestigating

   We are currently investigating an issue causing VOIP authentication failures in all the deployments in the US region and are working to correct the issue as soon as possible. At this time please use alternative authentication methods such as Duo Mobile Push, or SMS passcodes. A list of all other alternative authentication methods can be found here: https://duo.com/docs/policy#authentication-methods Please check back here or subscribe to updates for any changes.

2. 22 марта 2026 г. в 19:13 UTCinvestigating

   We are observing a recovery in voice call processing across the U.S. and are continuing to investigate the root cause of this impact.

3. 22 марта 2026 г. в 20:00 UTCmonitoring

   We are now observing high rates of success of VoIP authentication. We will continue to monitor the situation.

4. 22 марта 2026 г. в 20:33 UTCresolved

   VoIP authentication has returned to normal operation with sustained success rates observed.

1. 2 марта 2026 г. в 08:30 UTCinvestigating

   We are currently investigating a service degradation on our DUO81 deployment and are working to restore services as soon as possible.

2. 2 марта 2026 г. в 08:31 UTCinvestigating

   We are continuing to investigate this issue.

3. 2 марта 2026 г. в 09:01 UTCidentified

   We have isolated the issue to a regional outage and are working with our partners to alleviate the degraded performance.

4. 2 марта 2026 г. в 14:00 UTCidentified

   We are continuing to work on a fix for this issue.

5. 2 марта 2026 г. в 14:44 UTCidentified

   Our cloud provider is responding to a power outage at a primary data center that continues to cause elevated error rates and connectivity issues across the region. While early signs of recovery were observed, the provider has since confirmed the impact is ongoing. Full restoration and environment stability are expected to take several hours. We will provide our next update within 6 hours.

6. 2 марта 2026 г. в 20:27 UTCidentified

   We remain in active coordination with our cloud service provider as they work to resolve a power outage affecting a primary data center. Connectivity issues and elevated error rates across the region will persist until power has been fully restored. The provider has indicated that full restoration and environment stabilization may take a day or more.

7. 3 марта 2026 г. в 19:18 UTCidentified

   As previously communicated, a major power and connectivity outage at a third-party data center in the UAE operated by a third-party vendor has impacted Cisco Duo functionality in the region. Our engineering teams have worked continuously to implement local mitigations, but the persistent instability of the regional infrastructure has required more comprehensive measures to ensure business continuity. To fully resolve these issues and restore service stability, Cisco Duo has officially initiated its Disaster Recovery (DR) process. This involves transitioning critical service dependencies and user data away from the affected UAE region to secure, high-availability data centers.

8. 3 марта 2026 г. в 23:46 UTCresolved

   The service outage affecting DUO81 has been resolved. All impacted customers have been successfully migrated to an alternative Duo deployment. This migration has restored full service functionality for all affected users.

1. 1 марта 2026 г. в 13:29 UTCinvestigating

   We are currently investigating a service degradation on our DUO81 deployment and are working to restore services as soon as possible.

2. 1 марта 2026 г. в 14:13 UTCidentified

   We have isolated the issue to a regional outage and are rerouting our traffic to alleviate the degraded performance.

3. 1 марта 2026 г. в 15:33 UTCidentified

   We are continuing to work to reroute traffic in order to alleviate the degraded performance.

4. 1 марта 2026 г. в 16:09 UTCmonitoring

   We have rerouted traffic and are monitoring the behavior for stable, positive results.

5. 1 марта 2026 г. в 16:39 UTCresolved

   The issue was resolved by rerouting traffic.

1. 23 февраля 2026 г. в 15:04 UTCinvestigating

   We are currently investigating an issue where 504 Gateway timeouts are present when accessing Entra when Duo is called for MFA. We are working to resolve as quickly as possible.

2. 23 февраля 2026 г. в 16:20 UTCinvestigating

   We are continuing to investigate an issue where 504 Gateway timeouts are present when accessing Entra when Duo is called for MFA. We are working with our respective partners to resolve as quickly as possible.

3. 23 февраля 2026 г. в 16:38 UTCmonitoring

   We have deployed a fix for the gateway timeouts and are now seeing successful authentications. We are currently monitoring for any further issues.

4. 23 февраля 2026 г. в 17:52 UTCresolved

   The issue causing 504 gateway timeouts during Microsoft Entra logins when Duo MFA was called has been fully resolved. Authentication services are operating normally at this time. We will share a root cause analysis once it becomes available.

5. 25 февраля 2026 г. в 15:01 UTCpostmortem

   # Summary On February 23, 2026, US-based users utilizing the Duo integration with Azure Identity management experienced increased authentication failures and timeouts. These errors were occurring specifically when accessing applications using Microsoft Entra and Duo as the MFA provider. Duo engineering was alerted to the issues via monitoring at 9:15 AM and promptly began investigating various factors that could have caused this error to occur, including checking the health of the different third-party services involved. Finally, the root cause was identified to be increased CPU usage resulting from a code change that added additional validation for security operations. The incident was remediated by scaling the backend services that support this integration. During the incident, updates were posted to the status page as new information became available. At the beginning of the incident, the impact of the issue was underestimated, and thus notifications were not set to go out along with these status updates. When the impact assessment changed, notifications should have been set to go out, but this was missed. Processes have been improved to help prevent this from happening in the future. # Timeline of Events | **Time \(in EST\) 2/23/2026** | **Event** | | --- | --- | | 9:15 am | Engineering begins investigating AzureAuth errors | | 9:27 am  | Duo begins receiving reports of 504 gateway timeout errors for customers accessing applications using Microsoft Entra and Duo.   | | 10:02 am   | First status page from Duo is put up regarding investigating the root cause. This status page is posted without notifications due to assumed scale of incident.   | | 11:10 am  | Duo scales resources for the Azureauth US deployment.  | | 11:19 am   | Errors begin to subside.  | | 11:24 am | The incident is moved to monitoring.  | | 11:30 am  | The incident is resolved.  | # Root Cause\(s\) The Duo team follows a regular and disciplined release process across all services. Changes are routinely deployed to improve functionality, security posture, and overall reliability. All releases undergo comprehensive validation, including: * Unit testing * Integration testing * Frontend testing * Performance testing This layered testing strategy is designed to identify functional defects, regressions, and performance impacts prior to production deployment. In the days leading up to the incident, a software update was deployed to the backend service that supports the Duo integration with Azure Identity management. This update was part of a broader security improvement effort. The change successfully passed all standard validation processes and was not expected to introduce downtime or performance degradation. During the Monday morning traffic peak, this backend service experienced elevated CPU utilization, resulting in service degradation. The following points all contributed to the service interruption: * **Increased CPU Utilization Per Request** * The deployed change approximately doubled CPU consumption per request within this service. * This materially increased overall compute demand during peak traffic. * **Performance Test Coverage Gaps** * Existing performance tests did not identify the increased CPU usage introduced by the change. * The specific workload characteristics that amplified CPU consumption under peak conditions were not fully represented in pre-production testing. * **Monitoring Gaps** * A recent migration of our monitoring services missed CPU monitors for this service, leading to an alerting gap when CPU consumption started trending upward. * Alerts were triggered only after peak load conditions were reached. # Remediation and Future Work To address all the causes for the incident, Duo is committed to doing the following: * **Addressing increased CPU Utilization per Request** * **This backend service has been scaled up \(Completed\)** * Additional resources have been added to the infrastructure powering this backend service to provide additional compute headroom. * This ensures sufficient capacity to absorb higher per-request of CPU costs. * **Connection Efficiency Improvements \(In Progress\)** * Investigate using persistent connection pooling, and/or caching mechanism to reduce connection overhead and lower per-request for CPU consumption. * **Graceful Degradation Enhancements \(In Progress\)** * Improvements being explored to ensure systems degrade gracefully under high CPU load, with the goal of protecting critical authentication flows and reducing user impact during resource saturation events. * **Addressing Performance Testing Coverage Gaps** * **Expanded Performance Test Coverage \(In Progress\)** * Performance testing will be updated to explicitly model this failure scenario. * Peak-load and stress conditions will be more accurately simulated. * **Performance-Based Release Gates \(In Progress\)** * Performance tests will become release-gating criteria. * Changes that materially impact resource consumption will be identified before production deployment. * **Addressing Monitoring Gaps** * **Capacity Monitoring Enhancement \(Completed\)** * Missing capacity monitors \(inadvertently omitted during a recent migration\) have been restored. * Monitoring coverage for this service has been expanded. * **Comprehensive Monitoring Audit \(Completed\)** * Full audit of all capacity monitors underway to ensure complete monitoring coverage and eliminate similar gaps across services. * **Customer Communication Improvements** * **Improved Status Page Notifications** * Status page processes have been updated to ensure faster and clearer communication. * Commitment to timely, transparent, and regular updates to all customers during future incidents, with notifications.

1. 23 февраля 2026 г. в 14:02 UTCinvestigating

   We are currently investigating an issue causing an abnormal phone call authentication success rate for the US region. Please check back here or subscribe to updates for any changes.

2. 23 февраля 2026 г. в 17:26 UTCmonitoring

   The issue causing an abnormal phone call authentication success rate is tied to a third-party carrier. We are monitoring the third-party's communications at this time and can confirm that Duo telephony services are functioning as expected at this time.

3. 23 февраля 2026 г. в 18:20 UTCresolved

   The issue causing an abnormal phone call authentication success rate has been resolved by the third-party carrier.

1. 19 февраля 2026 г. в 14:46 UTCinvestigating

   We are investigating an issue causing the AI Assistant to be unavailable for administrators in India. Please check back here or subscribe to updates for any changes.

2. 19 февраля 2026 г. в 17:47 UTCmonitoring

   We have deployed a fix for the issue causing the AI Assistant to be unavailable for administrators in India and are monitoring the results of the fix at this time.

3. 19 февраля 2026 г. в 18:01 UTCresolved

   The issue causing the AI Assistant to be unavailable for administrators in India has been resolved.