Duo Security 장애 기록

1. 2026년 9월 3일 PM 10:04 UTCinvestigating

   우리는 영향을받는 미국 배포에 고객을 위한 듀오 하드웨어 토큰 인증에 영향을 미치는 부분 부족을 조사하고 있습니다. 하드웨어 토큰을 사용하는 고객은 intermittent 인증 실패를 경험할 수 있습니다. 현재 정보에 기반하여 인증 시도가 성공할 것으로 예상됩니다. 다른 듀오 인증 방법은 영향을받지 않고 구성 및 사용 가능한 경우 대체 방법으로 사용할 수 있습니다. 우리의 엔지니어링 팀은 적극적으로 투자하고 소송 단계를 시작했습니다. 업데이트 및 추가 정보는 새로운 정보로 제공됩니다. 우리는 이 시간에 당신의 인내 및 이해를 평가합니다.

2. 2026년 9월 3일 PM 10:30 UTCmonitoring

   우리는 영향을받는 미국 배포에 듀오 하드웨어 토큰 인증에 대한 실패율을 보았습니다. 하드웨어 토큰을 사용하는 고객은 여전히 간헐적인 인증 실패를 경험할 수 있습니다. 인증 시도가 성공할 것으로 예상됩니다. 다른 듀오 인증 방법은 영향을받지 않고 구성 및 사용 가능한 경우 대체 방법으로 사용할 수 있습니다. 우리는 서비스 안정성을 계속 모니터링하고 필요에 따라 추가 업데이트를 제공합니다.

3. 2026년 9월 3일 PM 11:24 UTCresolved

   이 문제는 해결되었습니다. Duo 하드웨어 토큰 인증은 미국 배포에 일반적으로 작용하고 있습니다. 다른 듀오 인증 방법은 영향을받지 못했습니다. 감사합니다. 문제가 계속되면 Duo Support에 문의하십시오.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 8월 10일 PM 8:34 UTCinvestigating

   We are investigating an issue affecting the signup service. New account registrations are temporarily unavailable. Our team is actively investigating and working to restore normal service. We apologize for any inconvenience and appreciate your patience.

2. 2026년 8월 10일 PM 9:04 UTCidentified

   We have identified the issue affecting new account registrations. Our team is working on a fix and expects to release it soon. We apologize for any inconvenience and appreciate your continued patience.

3. 2026년 8월 10일 PM 10:46 UTCresolved

   This incident has been resolved. New Duo account registrations are now functioning normally. Thank you for your patience. If you continue to experience issues, please contact Duo Support.

1. 2026년 8월 8일 PM 9:13 UTCidentified

   We have identified an issue causing phone call authentication failures for customers in the United States. This is related to a third-party carrier disruption. We are actively monitoring the carrier issue and working with our provider as needed. Please check back here or subscribe to updates for any changes.

2. 2026년 8월 8일 PM 11:04 UTCmonitoring

   We are seeing recovery for the issue related to a third-party carrier disruption that caused phone call authentication failures for customers in the United States. We are actively monitoring phone call authentication delivery and will provide another update once the issue is fully resolved.

3. 2026년 8월 9일 AM 12:07 UTCresolved

   Phone call authentication delivery for customers in the United States has returned to normal operation. If customers continue to experience issues, please contact Duo Support.

1. 2026년 7월 23일 PM 6:02 UTCinvestigating

   우리는 이스라엘 사용자를위한 VoIP 및 SMS 메시지 배달에 영향을 미치는 문제를 조사하고 있습니다. 문제는 업스트림 캐리어와 관련이 있습니다. 우리의 팀은 우리의 공급자와 협력하여 정상적인 서비스를 조사하고 복원합니다. 어디서든, 사용자는 Duo Push와 같은 대안 인증 방법을 사용할 수 있습니다. 우리는 더 많은 정보로 업데이트를 제공 할 것입니다.

2. 2026년 7월 23일 PM 6:39 UTCmonitoring

   우리는 이스라엘 사용자에 영향을 미치는 SMS 및 VoIP 배송 문제를 해결하기 위해 변화를 구현했습니다. SMS와 VoIP 납품은 회복되고, 우리는 계속 안정성을 확인하기 위하여 감시하고 있습니다.

3. 2026년 7월 24일 PM 3:53 UTCmonitoring

   이스라엘의 SMS 및 VoIP 배송에 대한 오류율은 구현 된 변경을 통해 감소했습니다. 우리는 서비스 행동을 감시하고 몇몇 고객에 영향을 미치는 간헐적인 실패의 보고를 조사합니다. 우리는 더 많은 정보로 또 다른 업데이트를 제공 할 것입니다.

4. 2026년 7월 27일 PM 8:41 UTCresolved

   사용자에 영향을 미치는 SMS 및 VoIP 배송 문제는 해결되었습니다. 고객이 문제가 계속되면 Duo Support에 문의하십시오.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 7월 7일 PM 11:15 UTCinvestigating

   We are investigating a third-party carrier issue affecting VoIP and SMS services in the Australia region. Customers in Australia may experience delays or failures when using phone call or SMS authentication. Other Duo authentication methods, such as Push Delivery, are not expected to be impacted and can be used as an alternate method of authentication if available. Our team is monitoring the carrier issue and working with the provider as needed. Updates and additional details will be provided as new information becomes available. We appreciate your patience and understanding during this time.

2. 2026년 7월 8일 AM 1:22 UTCmonitoring

   The third-party carrier has indicated that the issue affecting VoIP and SMS services in the Australia region has been resolved. Customers in Australia should see improved delivery for phone call and SMS authentication. Other Duo authentication methods, such as Push Delivery, remain available as alternate authentication methods if configured. We are continuing to monitor service stability and will provide additional updates when the incident is fully resolved.

3. 2026년 7월 8일 PM 3:01 UTCresolved

   The third-party carrier issue affecting VoIP and SMS services in the Australia region has been resolved. Phone call and SMS authentication services are operating normally. We appreciate your patience and understanding while we addressed this issue.

1. 2026년 6월 23일 AM 3:58 UTCidentified

   We have identified the issue related to SMS for UAE region and are working to resolve it. Users have the option to use alternative auth methods, like Duo Push or VoIP. Please check back here or subscribe to updates for any changes.

2. 2026년 6월 23일 AM 8:29 UTCmonitoring

   We have mitigated the SMS delivery issue affecting the UAE region. We are actively monitoring the situation and will provide a final update once the issue is fully resolved.

3. 2026년 6월 23일 PM 2:21 UTCresolved

   The SMS delivery issue affecting the UAE region has been resolved.   Our team has verified that SMS services are operating normally and full functionality has been restored. Thank you for your patience while we resolved this issue

1. 2026년 6월 10일 PM 2:17 UTCinvestigating

   We are currently investigating an issue affecting AI Assistant functionality in the US-W1 (US West-1) region. Users may experience errors or be unable to access AI Assistant features. Our engineering teams are actively investigating the issue and working to restore normal service as quickly as possible. Please check back here or subscribe to updates for the latest information.

2. 2026년 6월 10일 PM 3:35 UTCresolved

   The issue affecting AI Assistant functionality in the US-W1 region has been resolved. Our team has verified that services are operating normally, and monitoring has confirmed system stability. Thank you for your patience while we resolved this issue.

1. 2026년 5월 18일 PM 6:37 UTCinvestigating

   We are currently investigating an issue with AI Assistant not functioning on all US deployments and are working to correct the issue as soon as possible. Please check back here or subscribe to updates for any changes.

2. 2026년 5월 18일 PM 7:27 UTCmonitoring

   We have implemented mitigation measures and are observing improved AI Assistant performance. We are continuing to monitor success rates and overall service stability.

3. 2026년 5월 18일 PM 8:30 UTCresolved

   The issue affecting AI Assistant functionality across all US deployments has been resolved, and full functionality has been restored. We will continue to monitor service performance to ensure stability.

1. 2026년 5월 13일 PM 1:22 UTCinvestigating

   We are currently investigating an issue causing limited authentication failures on DUO57 and are working to correct the issue as soon as possible.

2. 2026년 5월 13일 PM 2:52 UTCidentified

   We have identified the cause of the issue causing authentication failures on DUO57 with the Duo Core Authentication Service and are actively working to restore service.

3. 2026년 5월 13일 PM 3:37 UTCmonitoring

   We have implemented a fix for the issue causing authentication failures on DUO57 with the Duo Core Authentication Service. We will continue to monitor system performance and authentication success rates and will provide further updates once the incident is fully resolved.

4. 2026년 5월 13일 PM 4:03 UTCresolved

   The issue causing authentication failures on DUO57 with the Duo Core Authentication Service is now resolved and full functionality has been restored.

1. 2026년 5월 7일 PM 2:17 UTCinvestigating

   We are investigating an issue causing failures when administrators located in the EU attempt to access the AI Assistant for the first time. Any administrators who have used the Assistant before should still be able to at this time

2. 2026년 5월 7일 PM 4:30 UTCidentified

   We’ve identified a problem impacting first-time access to the AI Assistant for administrators in the EU and are actively working to restore normal access. We’ll continue to share updates as progress is made.

3. 2026년 5월 7일 PM 10:40 UTCresolved

   The issue affecting first-time access to the AI Assistant for administrators in the EU has been fully resolved, and normal functionality has been restored.

1. 2026년 4월 30일 PM 5:54 UTCidentified

   We are currently addressing an issue impacting desktop authenticator enrollments .A subset of customers may experience intermittent delays or failures during desktop authenticator enrollment. Mitigation actions are in progress and are helping to reduce impact. Core authentication, SSO, and delivery services  remain fully operational.

2. 2026년 4월 30일 PM 5:57 UTCmonitoring

   Mitigation actions have been implemented. The service has stabilized and recovery is underway. We will continue to monitor to ensure full resolution.

3. 2026년 4월 30일 PM 6:07 UTCresolved

   The issue has been resolved. Services have returned to normal operation, and customers should no longer experience delays or failures. We apologize for any inconvenience caused and thank you for your patience.

1. 2026년 4월 28일 PM 2:22 UTCinvestigating

   We are investigating elevated authentication latency affecting customers . Authentication requests are continuing to succeed at this time, but customers may experience slower-than-normal response times and degraded performance during login attempts. Our team is actively working to identify the cause and restore normal performance as quickly as possible.

2. 2026년 4월 28일 PM 2:56 UTCmonitoring

   We have applied mitigation steps and are seeing improved authentication performance for customers in the deployment. Authentication latency has decreased, and the service is recovering. Our team is continuing to monitor the environment closely to ensure stability while we investigate the underlying cause.

3. 2026년 4월 28일 PM 3:26 UTCresolved

   The issue causing elevated authentication latency has been fully resolved. Authentication performance has returned to normal levels, and customers should no longer experience delays during login attempts.

1. 2026년 4월 18일 AM 4:30 UTCinvestigating

   We are currently detecting an issue with carriers in the US region for SMS delivery. We are currently working with providers to restore the service. In the meantime, please use other means of authentication such as Duo Push.

2. 2026년 4월 18일 AM 5:39 UTCinvestigating

   We are continuing to investigate the SMS delivery issue impacting all customers in the US region. Push notifications will continue to work as expected. Further updates will be provided as more information becomes available.

3. 2026년 4월 18일 AM 7:09 UTCresolved

   The SMS delivery issue impacting customers in the US region has been resolved. SMS passcode authentication is now functioning normally. We apologize for any inconvenience caused and thank you for your patience.

1. 2026년 3월 22일 PM 10:13 UTCinvestigating

   We are currently observing a recurrence of the issue causing VOIP authentication failures in all deployments in the US region. We are working to correct the issue as soon as possible. While we work to resolve this, please use alternative authentication methods such as Duo Mobile Push, SMS passcodes etc. A full list of available authentication methods can be found here: https://duo.com/docs/policy#authentication-methods

2. 2026년 3월 23일 AM 12:01 UTCmonitoring

   We are now observing high rates of success of VoIP authentication. The issue appears to be resolved but we will continue to monitor the situation.

3. 2026년 3월 23일 AM 2:23 UTCresolved

   VoIP authentication has returned to normal operation with sustained success rates observed.

1. 2026년 3월 22일 PM 5:18 UTCinvestigating

   We are currently investigating an issue causing VOIP authentication failures in all the deployments in the US region and are working to correct the issue as soon as possible. At this time please use alternative authentication methods such as Duo Mobile Push, or SMS passcodes. A list of all other alternative authentication methods can be found here: https://duo.com/docs/policy#authentication-methods Please check back here or subscribe to updates for any changes.

2. 2026년 3월 22일 PM 7:13 UTCinvestigating

   We are observing a recovery in voice call processing across the U.S. and are continuing to investigate the root cause of this impact.

3. 2026년 3월 22일 PM 8:00 UTCmonitoring

   We are now observing high rates of success of VoIP authentication. We will continue to monitor the situation.

4. 2026년 3월 22일 PM 8:33 UTCresolved

   VoIP authentication has returned to normal operation with sustained success rates observed.

1. 2026년 3월 2일 AM 8:30 UTCinvestigating

   We are currently investigating a service degradation on our DUO81 deployment and are working to restore services as soon as possible.

2. 2026년 3월 2일 AM 8:31 UTCinvestigating

   We are continuing to investigate this issue.

3. 2026년 3월 2일 AM 9:01 UTCidentified

   We have isolated the issue to a regional outage and are working with our partners to alleviate the degraded performance.

4. 2026년 3월 2일 PM 2:00 UTCidentified

   We are continuing to work on a fix for this issue.

5. 2026년 3월 2일 PM 2:44 UTCidentified

   Our cloud provider is responding to a power outage at a primary data center that continues to cause elevated error rates and connectivity issues across the region. While early signs of recovery were observed, the provider has since confirmed the impact is ongoing. Full restoration and environment stability are expected to take several hours. We will provide our next update within 6 hours.

6. 2026년 3월 2일 PM 8:27 UTCidentified

   We remain in active coordination with our cloud service provider as they work to resolve a power outage affecting a primary data center. Connectivity issues and elevated error rates across the region will persist until power has been fully restored. The provider has indicated that full restoration and environment stabilization may take a day or more.

7. 2026년 3월 3일 PM 7:18 UTCidentified

   As previously communicated, a major power and connectivity outage at a third-party data center in the UAE operated by a third-party vendor has impacted Cisco Duo functionality in the region. Our engineering teams have worked continuously to implement local mitigations, but the persistent instability of the regional infrastructure has required more comprehensive measures to ensure business continuity. To fully resolve these issues and restore service stability, Cisco Duo has officially initiated its Disaster Recovery (DR) process. This involves transitioning critical service dependencies and user data away from the affected UAE region to secure, high-availability data centers.

8. 2026년 3월 3일 PM 11:46 UTCresolved

   The service outage affecting DUO81 has been resolved. All impacted customers have been successfully migrated to an alternative Duo deployment. This migration has restored full service functionality for all affected users.

1. 2026년 3월 1일 PM 1:29 UTCinvestigating

   We are currently investigating a service degradation on our DUO81 deployment and are working to restore services as soon as possible.

2. 2026년 3월 1일 PM 2:13 UTCidentified

   We have isolated the issue to a regional outage and are rerouting our traffic to alleviate the degraded performance.

3. 2026년 3월 1일 PM 3:33 UTCidentified

   We are continuing to work to reroute traffic in order to alleviate the degraded performance.

4. 2026년 3월 1일 PM 4:09 UTCmonitoring

   We have rerouted traffic and are monitoring the behavior for stable, positive results.

5. 2026년 3월 1일 PM 4:39 UTCresolved

   The issue was resolved by rerouting traffic.

1. 2026년 2월 23일 PM 3:04 UTCinvestigating

   We are currently investigating an issue where 504 Gateway timeouts are present when accessing Entra when Duo is called for MFA. We are working to resolve as quickly as possible.

2. 2026년 2월 23일 PM 4:20 UTCinvestigating

   We are continuing to investigate an issue where 504 Gateway timeouts are present when accessing Entra when Duo is called for MFA. We are working with our respective partners to resolve as quickly as possible.

3. 2026년 2월 23일 PM 4:38 UTCmonitoring

   We have deployed a fix for the gateway timeouts and are now seeing successful authentications. We are currently monitoring for any further issues.

4. 2026년 2월 23일 PM 5:52 UTCresolved

   The issue causing 504 gateway timeouts during Microsoft Entra logins when Duo MFA was called has been fully resolved. Authentication services are operating normally at this time. We will share a root cause analysis once it becomes available.

5. 2026년 2월 25일 PM 3:01 UTCpostmortem

   # Summary On February 23, 2026, US-based users utilizing the Duo integration with Azure Identity management experienced increased authentication failures and timeouts. These errors were occurring specifically when accessing applications using Microsoft Entra and Duo as the MFA provider. Duo engineering was alerted to the issues via monitoring at 9:15 AM and promptly began investigating various factors that could have caused this error to occur, including checking the health of the different third-party services involved. Finally, the root cause was identified to be increased CPU usage resulting from a code change that added additional validation for security operations. The incident was remediated by scaling the backend services that support this integration. During the incident, updates were posted to the status page as new information became available. At the beginning of the incident, the impact of the issue was underestimated, and thus notifications were not set to go out along with these status updates. When the impact assessment changed, notifications should have been set to go out, but this was missed. Processes have been improved to help prevent this from happening in the future. # Timeline of Events | **Time \(in EST\) 2/23/2026** | **Event** | | --- | --- | | 9:15 am | Engineering begins investigating AzureAuth errors | | 9:27 am  | Duo begins receiving reports of 504 gateway timeout errors for customers accessing applications using Microsoft Entra and Duo.   | | 10:02 am   | First status page from Duo is put up regarding investigating the root cause. This status page is posted without notifications due to assumed scale of incident.   | | 11:10 am  | Duo scales resources for the Azureauth US deployment.  | | 11:19 am   | Errors begin to subside.  | | 11:24 am | The incident is moved to monitoring.  | | 11:30 am  | The incident is resolved.  | # Root Cause\(s\) The Duo team follows a regular and disciplined release process across all services. Changes are routinely deployed to improve functionality, security posture, and overall reliability. All releases undergo comprehensive validation, including: * Unit testing * Integration testing * Frontend testing * Performance testing This layered testing strategy is designed to identify functional defects, regressions, and performance impacts prior to production deployment. In the days leading up to the incident, a software update was deployed to the backend service that supports the Duo integration with Azure Identity management. This update was part of a broader security improvement effort. The change successfully passed all standard validation processes and was not expected to introduce downtime or performance degradation. During the Monday morning traffic peak, this backend service experienced elevated CPU utilization, resulting in service degradation. The following points all contributed to the service interruption: * **Increased CPU Utilization Per Request** * The deployed change approximately doubled CPU consumption per request within this service. * This materially increased overall compute demand during peak traffic. * **Performance Test Coverage Gaps** * Existing performance tests did not identify the increased CPU usage introduced by the change. * The specific workload characteristics that amplified CPU consumption under peak conditions were not fully represented in pre-production testing. * **Monitoring Gaps** * A recent migration of our monitoring services missed CPU monitors for this service, leading to an alerting gap when CPU consumption started trending upward. * Alerts were triggered only after peak load conditions were reached. # Remediation and Future Work To address all the causes for the incident, Duo is committed to doing the following: * **Addressing increased CPU Utilization per Request** * **This backend service has been scaled up \(Completed\)** * Additional resources have been added to the infrastructure powering this backend service to provide additional compute headroom. * This ensures sufficient capacity to absorb higher per-request of CPU costs. * **Connection Efficiency Improvements \(In Progress\)** * Investigate using persistent connection pooling, and/or caching mechanism to reduce connection overhead and lower per-request for CPU consumption. * **Graceful Degradation Enhancements \(In Progress\)** * Improvements being explored to ensure systems degrade gracefully under high CPU load, with the goal of protecting critical authentication flows and reducing user impact during resource saturation events. * **Addressing Performance Testing Coverage Gaps** * **Expanded Performance Test Coverage \(In Progress\)** * Performance testing will be updated to explicitly model this failure scenario. * Peak-load and stress conditions will be more accurately simulated. * **Performance-Based Release Gates \(In Progress\)** * Performance tests will become release-gating criteria. * Changes that materially impact resource consumption will be identified before production deployment. * **Addressing Monitoring Gaps** * **Capacity Monitoring Enhancement \(Completed\)** * Missing capacity monitors \(inadvertently omitted during a recent migration\) have been restored. * Monitoring coverage for this service has been expanded. * **Comprehensive Monitoring Audit \(Completed\)** * Full audit of all capacity monitors underway to ensure complete monitoring coverage and eliminate similar gaps across services. * **Customer Communication Improvements** * **Improved Status Page Notifications** * Status page processes have been updated to ensure faster and clearer communication. * Commitment to timely, transparent, and regular updates to all customers during future incidents, with notifications.

1. 2026년 2월 23일 PM 2:02 UTCinvestigating

   We are currently investigating an issue causing an abnormal phone call authentication success rate for the US region. Please check back here or subscribe to updates for any changes.

2. 2026년 2월 23일 PM 5:26 UTCmonitoring

   The issue causing an abnormal phone call authentication success rate is tied to a third-party carrier. We are monitoring the third-party's communications at this time and can confirm that Duo telephony services are functioning as expected at this time.

3. 2026년 2월 23일 PM 6:20 UTCresolved

   The issue causing an abnormal phone call authentication success rate has been resolved by the third-party carrier.

1. 2026년 2월 19일 PM 2:46 UTCinvestigating

   We are investigating an issue causing the AI Assistant to be unavailable for administrators in India. Please check back here or subscribe to updates for any changes.

2. 2026년 2월 19일 PM 5:47 UTCmonitoring

   We have deployed a fix for the issue causing the AI Assistant to be unavailable for administrators in India and are monitoring the results of the fix at this time.

3. 2026년 2월 19일 PM 6:01 UTCresolved

   The issue causing the AI Assistant to be unavailable for administrators in India has been resolved.