Socure outage history

SSA is currently experiencing a service-impacting event affecting multiple applications and services, including the Electronic Consent Based Social Security Number Verification (eCBSV) service. A small number of transactions are affected as the disruption was first noted at approximately 4:47 PM (ET) on October 23, 2026. We will share another update once we receive notice that service has been restored, and eCBSV is operating normally again.

At approximately 4:56 (ET), service has been restored and eCBSV is operating normally at this time. We apologize for any inconveniences this may have caused.

We are currently investigating an issue that is causing intermittent latency with displaying and downloading DocV Images for a small percentage of completed transactions. There is no impact to DocV nor evaluations at this time but images may take additional time to appear.

Our team has taken steps to mitigate the latency experienced for DocV image display and downloads. There continues to be no impact to DocV evaluations at this time and the team will continue to monitor the situation. We will provide another update once all impact has been resolved.

This issue has been resolved, and we’re monitoring performance to ensure the services remain stable. We apologize for any inconvenience this may have caused.

SSA experienced a service-impacting event that affected multiple applications and services, including the Electronic Consent Based Social Security Number Verification (eCBSV) service. A small number of transactions were affected as the disruption began at approximately 2:50PM (ET) on October 18, 2026, and was resolved at approximately 3:10PM (ET). Service has been restored, and eCBSV is operating normally at this time.

Starting at 3:00 EDT, some customers may experience intermittent errors or failed requests when using the RiskOS Evaluate APIs. We’ve identified a database issue and are actively working to resolve it. We apologize for the disruption and appreciate your patience while we investigate.

We’ve taken an interim measure, and the elevated error rate has subsided. We haven’t observed related errors in the past 15 minutes. We’re moving the incident to monitoring while we continue work on a full resolution. We’ll share another update when the situation changes.

We appreciate your patience as our team is still monitoring the situation closely. Error rates have subsided as we continue to work towards a full resolution.

The incident has been resolved, and we are no longer seeing errors. Services are operating normally, and we’ll continue to monitor performance. We appreciate your patience while we worked through this.

Socure RiskOS and ID+ services have been restored. An error in a rollout caused a temporary spike in error rates from our services between 18:12 and 18:56 Eastern time. The error has been corrected and services are stable now. We understand this outage is unacceptable and a full RCA will be available as soon as full analysis of the issue is complete.

## **Root Cause Analysis** **Incident:** Socure APIs Outage **Date Range:** 15-Oct-2026 **Impact:** Interruption in service for Socure APIs **Status:** Resolved ### **1. Summary** On 15-Oct-2026, between 18:12 and 19:00 Eastern time Socure experienced an interruption to its RiskOS and ID\+ APIs, causing errors and timeouts to users. The first phase ran from 18:12 to 18:20, and was a major interruption for virtually every service. The second phase of the outage ran from 18:20 to 19:00, and was a partial outage for services that rely on KYC data, where customers saw elevated error rates and latencies for some services, but traffic was still being served. ### **2. Timeline** **Date / Time \(ET\)**  |  Event **18:12** | A routine cleanup change is merged, moving a source repo from one code repository to another. A bug in our review tool hides the scope of the change. **18:13** | ArgoCD restarts most services in a major Kubernetes cluster all at once. **18:20** | All service pods are running. Partial API service restored, although KYC services are still seeing an elevated error rate. **18:21** |  The flood of startup traffic overwhelms our KYC Search service. Error rates spike. **18:42** |  SRE diagnoses the cascading failure impacting KYC Search and blocks traffic to it so it can come up. **18:56** |  The KYC Search service is fully up, and SRE returns traffic to it. Downstream services stabilize.  **19:00** | All traffic running as normal, incident declared stable. ### **3. Root Cause** * **Primary Root Cause:** As a part of an ongoing migration, an on-call SRE made a change to ArgoCD, the system we use to manage the kubernetes clusters Socure runs its services on. The change was scoped to a single cluster, was reviewed carefully before pushing, caused no issues in other environments it was tested in, and should have had no visible impact. Unfortunately, the change updated a legacy setting that did not appear in the change review. That legacy setting governed what services ArgoCD managed, and removed many live services from ArgoCD. In consequence, ArgoCD removed those services from production briefly, before the setting was corrected. * **Contributing Factors:** The KYC Search service is a backend piece of Socure infrastructure that provides services to several of our internal products. When that service restarted from scratch at the same time as all of the clients of the service did, a small number of pods in the service crashed under load. This increased the load on the remaining pods, some of which crashed in turn. This cascading failure prevented the service from coming back on its own the way the rest of Socure’s infrastructure did. Diagnosing this separate failure mode took additional time, and remediating it fixed the remaining errors after the initial incident. ### **4. Resolution** The ArgoCD change was reverted immediately, causing services to come back to their prior state. The KYC Search service required manual effort to block traffic from reaching it until a majority of pods in the service could come up. After the service was healthy, we could restore traffic to it and resolve the overall incident. ### **5. Corrective and Preventive Actions** ### **6. Lessons Learned** * Socure’s tooling for viewing the changes it makes to production did not work correctly. We are reviewing these tools to ensure they operate correctly under all use cases and fix any cases like this where the full scope of a change is not clear. * Augment the existing change review process with agentic support. While in this case, the specific change was not visible in the plan, having an extra reviewer that is supported by an extensive training model will help ensure issues are caught before submission. * Even changes that appear to have no impact, when they are made to the governance system for our production jobs, can be risky, and should be shifted further out of US business hours. * Our KYC Search service is vulnerable to a “thundering-herd” problem when it receives full traffic flow when it is not fully online. We need to strengthen our circuit breakers to prevent this issue recurring. * We run too many of our core API services in a single kubernetes cluster. Even though that cluster is distributed across multiple availability zones, the shared control plane creates too high a blast radius for changes to that cluster. ### **7. Next Steps & Ongoing Commitment** Socure understands our customers rely on us as a key provider in critical business flows, and that this service interruption is not acceptable. We apologize for the impact this issue caused, and commit to ensuring it does not recur. To that end, we will deliver details on our plans to re-architect our application layout by November, as well as improve the resilience of the KYC Search service in Q1 of next year.

Socure RiskOS and ID+ services have been restored. An error in a rollout caused a temporary spike in error rates from our services between 18:12 and 18:56 Eastern time. The error has been corrected and services are stable now. We understand this outage is unacceptable and a full RCA will be available as soon as full analysis of the issue is complete.

We are currently experiencing an active incident that is impacting service availability. We are currently investigating the issue and will provide updates as soon as we have additional information.

Socure RiskOS and ID+ services have been restored. An error in a rollout caused a temporary spike in error rates from our services between 18:20 and 18:56 Eastern time. The error has been corrected and services are stable now. We understand this outage is unacceptable and a full RCA will be available as soon as full analysis of the issue is complete.

Socure RiskOS and ID+ services have been restored. An error in a rollout caused a temporary spike in error rates from our services between 18:12 and 18:56 Eastern time. The error has been corrected and services are stable now. We understand this outage is unacceptable and a full RCA will be available as soon as full analysis of the issue is complete.

Socure RiskOS and ID+ services have been restored. An error in a rollout caused a temporary spike in error rates from our services between 18:12 and 18:56 Eastern time. The error has been corrected and services are stable now. We understand this outage is unacceptable and a full RCA will be available as soon as full analysis of the issue is complete.

## **Root Cause Analysis** **Incident:** Socure APIs Outage **Date Range:** 15-Oct-2026 **Impact:** Interruption in service for Socure APIs **Status:** Resolved ### **1. Summary** On 15-Oct-2026, between 18:12 and 19:00 Eastern time Socure experienced an interruption to its RiskOS and ID\+ APIs, causing errors and timeouts to users. The first phase ran from 18:12 to 18:20, and was a major interruption for virtually every service. The second phase of the outage ran from 18:20 to 19:00, and was a partial outage for services that rely on KYC data, where customers saw elevated error rates and latencies for some services, but traffic was still being served. ### **2. Timeline** **Date / Time \(ET\)**  |  Event **18:12** | A routine cleanup change is merged, moving a source repo from one code repository to another. A bug in our review tool hides the scope of the change. **18:13** | ArgoCD restarts most services in a major Kubernetes cluster all at once. **18:20** | All service pods are running. Partial API service restored, although KYC services are still seeing an elevated error rate. **18:21** |  The flood of startup traffic overwhelms our KYC Search service. Error rates spike. **18:42** |  SRE diagnoses the cascading failure impacting KYC Search and blocks traffic to it so it can come up. **18:56** |  The KYC Search service is fully up, and SRE returns traffic to it. Downstream services stabilize.  **19:00** | All traffic running as normal, incident declared stable. ### **3. Root Cause** * **Primary Root Cause:** As a part of an ongoing migration, an on-call SRE made a change to ArgoCD, the system we use to manage the kubernetes clusters Socure runs its services on. The change was scoped to a single cluster, was reviewed carefully before pushing, caused no issues in other environments it was tested in, and should have had no visible impact. Unfortunately, the change updated a legacy setting that did not appear in the change review. That legacy setting governed what services ArgoCD managed, and removed many live services from ArgoCD. In consequence, ArgoCD removed those services from production briefly, before the setting was corrected. * **Contributing Factors:** The KYC Search service is a backend piece of Socure infrastructure that provides services to several of our internal products. When that service restarted from scratch at the same time as all of the clients of the service did, a small number of pods in the service crashed under load. This increased the load on the remaining pods, some of which crashed in turn. This cascading failure prevented the service from coming back on its own the way the rest of Socure’s infrastructure did. Diagnosing this separate failure mode took additional time, and remediating it fixed the remaining errors after the initial incident. ### **4. Resolution** The ArgoCD change was reverted immediately, causing services to come back to their prior state. The KYC Search service required manual effort to block traffic from reaching it until a majority of pods in the service could come up. After the service was healthy, we could restore traffic to it and resolve the overall incident. ### **5. Corrective and Preventive Actions** ### **6. Lessons Learned** * Socure’s tooling for viewing the changes it makes to production did not work correctly. We are reviewing these tools to ensure they operate correctly under all use cases and fix any cases like this where the full scope of a change is not clear. * Augment the existing change review process with agentic support. While in this case, the specific change was not visible in the plan, having an extra reviewer that is supported by an extensive training model will help ensure issues are caught before submission. * Even changes that appear to have no impact, when they are made to the governance system for our production jobs, can be risky, and should be shifted further out of US business hours. * Our KYC Search service is vulnerable to a “thundering-herd” problem when it receives full traffic flow when it is not fully online. We need to strengthen our circuit breakers to prevent this issue recurring. * We run too many of our core API services in a single kubernetes cluster. Even though that cluster is distributed across multiple availability zones, the shared control plane creates too high a blast radius for changes to that cluster. ### **7. Next Steps & Ongoing Commitment** Socure understands our customers rely on us as a key provider in critical business flows, and that this service interruption is not acceptable. We apologize for the impact this issue caused, and commit to ensuring it does not recur. To that end, we will deliver details on our plans to re-architect our application layout by November, as well as improve the resilience of the KYC Search service in Q1 of next year.

We are currently experiencing an unplanned outage affecting the eCBSV module due to SSA responses, resulting in 5xx errors. Socure has contacted the vendor to investigate the issue and determine when service is expected to be restored.

Error rates have subsided over the last few minutes. We are monitoring these rates, and pending our vendor response.

SSA experienced a service-impacting event that affected multiple applications and services, including the Electronic Consent Based Social Security Number Verification (eCBSV) service. The disruption began at approximately 10:19AM (ET) on October 15, 2026, and was resolved at approximately 10:41AM (ET). Service has been restored, and eCBSV is operating normally at this time.

We experienced elevated transaction failures affecting Watchlist Plus and Watchlist Premier. The issue has been resolved, and we’re monitoring performance to ensure the services remain stable. We apologize for any inconvenience this may have caused.

We are investigating an issue causing delays in case updates being reflected in the RiskOS Case Management UI. Our Engineering team is actively working to identify the root cause and restore normal update behavior. We will provide further updates as more information becomes available. This is not impacting API evaluation requests.

We have identified the root cause of the issue affecting updates in the RiskOS Case Management UI. This issue is limited to a subset of cases and specifically affects the Case Search functionality, which may temporarily show outdated case states. Please note that case status data displayed within the Case List and individual Case Details pages is accurate and reflects the latest state. Our Engineering team has deployed a fix for all newly updated cases and is actively working to re-sync the remaining affected cases to ensure data consistency across search. We will provide another update once normal behavior is fully restored.

The issue affecting case status updates has been resolved. All the new cases are being processed correctly. We sincerely apologize for any inconvenience this may have caused and appreciate your patience while our team worked to resolve it.

Between 6:28 AM and 6:57 AM ET on September 21, an issue during an operational activity caused some webhook notifications associated with older evaluations from the past month to be delivered again. Current evaluations and webhook processing were not impacted. The issue has been resolved. We apologize for any confusion or additional noise this may have caused.

We are investigating delays affecting batch processing on the dashboard. Customers may experience delayed processing while we review the issue.

This incident has been resolved.

We are observing increased latency affecting some customers using our Fraud and Synthetic products. Our team is actively investigating the issue. API requests are completing successfully, and we are not observing any API failures at this time. We will provide further updates as more information becomes available.

We’re seeing an improvement in latency affecting some customers using our Fraud and Synthetic products. We’ll continue to monitor the situation closely.

The issue impacting Fraud and Synthetic services has been resolved, and service has returned to normal operation. Customers should no longer experience increased latency or delays when accessing Fraud and Synthetic services

Between  8:30 AM and 2:00 PM EST, we observed an increase in 5xx error rates, which resulted in intermittent errors or failed requests for some customers. Overall impact was minimal, less than 0.01%. The error rate has since returned to normal. This issue is fully resolved at this time. We apologize for any disruption and appreciate your patience.

Between 7:00 AM and 7:20 AM EST, we observed an increase in 5xx error rates, which may have resulted in intermittent errors or failed requests for some customers. The error rate has since returned to normal, and we are investigating the root cause to ensure we have fully resolved the issue. We apologize for any disruption and appreciate your patience.

**Root Cause Analysis** **Incident:** Intermittent elevated 5xx errors across services **Date Range:** September 9, 2026, 7:00 AM – 9:20 AM EDT \(approximately 2 hours 20 minutes of intermittent impact\) **Impact:** Some requests submitted to the EmailAuthScore API, and related services, intermittently returned error responses. Impact was intermittent rather than continuous and was limited to defined windows described in the timeline below. **Status:** Resolved ### **1. Summary** On September 9, 2026, a planned increase in backend processing capacity led to elevated connection load on an internal infrastructure component used to support downstream processing for several services. This component reached an internal capacity threshold, which introduced processing delays and resulted in intermittent error responses on the EmailAuthScore API and related services for a subset of requests. The issue was identified through a combination of internal monitoring and reports during the impact window. Socure engineering correlated the errors with the recent capacity change and reverted it, which stopped the initial increase in errors. A second, related increase in errors then occurred when a separate capacity change, made earlier for a different internal service, placed additional load on the same shared component; engineering applied further mitigations, after which all affected services returned to normal, stable operation. Section 5 outlines the corrective and preventive actions Socure is committing to, with priority on improving visibility into infrastructure capacity and preventing delays in any single internal component from affecting customer-facing request processing. ### **2. Timeline** | **Date / Time \(EDT\)** | **Event** | | --- | --- | | **September 9, 2026 07:00 AM** | Following a planned increase in backend processing capacity, the EmailAuthScore API began intermittently returning error responses. | | **September 9, 2026 07:20 AM** | The recent capacity change was reverted; error rates returned to normal. | | **September 9, 2026 08:45 AM** | A separate capacity change made earlier for a different internal service placed additional load on the same shared internal component, and the EmailAuthScore API again began intermittently returning error responses. | | **September 9, 2026 09:20 AM** | Additional mitigations were applied; error rates returned to normal and remained stable. | | **September 9, 2026 01:13 PM** | As a precaution, Socure temporarily rerouted a portion of internal processing away from the affected internal component to safely complete the remaining capacity increase. No further errors were observed after this change. | ### **3. Root Cause** * **Primary Root Cause:** As part of planned work to increase system capacity ahead of anticipated demand, Socure increased the processing capacity of backend services. This change increased the number of connections to an internal messaging component used to support downstream processing for these services. That component reached an internal connection capacity limit, which resulted in connection and processing delays. These delays introduced additional latency and, in some cases, backpressure into request processing, which contributed to intermittent elevated error rates on the EmailAuthScore API and related services for a subset of requests during the impact windows described above.. * **Contributing Factors:** * A sudden increase in infrastructure capacity demand exceeded the capacity that was immediately not available within the internal messaging component. ### **4. Resolution** Upon identifying that the elevated error rates were linked to the recent capacity change, Socure engineering reverted the change, which stopped the initial increase in errors. A second, related increase in errors then occurred when a separate capacity change, made earlier for a different internal service, placed additional load on the same shared component. Engineering applied additional mitigations, including temporarily rerouting a portion of internal processing away from the affected component for the impacted services. Following these actions, all affected services returned to normal, stable operation, which Socure confirmed through continued monitoring. ‌ ### **5. Corrective and Preventive Actions** | **Action** | **Description** | **ETA / Status** | | --- | --- | --- | | Optimize infrastructure capacity utilization | Adjusted backend service capacity and connection usage to ensure processing remains within supported infrastructure limits and to reduce the risk of similar capacity-related issues. | Done | | Increase messaging infrastructure capacity | Increased the capacity of the underlying messaging infrastructure to provide additional headroom and support future increases in traffic volume. | Done | | Enhance capacity monitoring and alerting | Add proactive monitoring and alerts to identify potential infrastructure capacity constraints before they affect service performance. | In progress. ETA 30 Sep 2026. | ### **6. Lessons Learned** * Proactive monitoring of infrastructure health needs to be in place ahead of capacity changes, not only application-level monitoring. ### **7. Next Steps & Ongoing Commitment** We are committed to: * Completing the corrective and preventive actions listed in Section 5. * Improving visibility into the health and capacity of critical internal infrastructure ahead of future scaling activity. * Providing a follow-up update once all corrective actions are complete.

We are currently investigating an issue that created an impact on DocV and Watchlist monitoring services where errors were observed today September 8th, from 09:05 AM EST to 09:30 AM EST. System has recovered, we are working on determining a root cause at this time.

The issue impacting DocV and Watchlist monitoring services on September 8 has been resolved. No customer impact has been observed since 10:35 AM EST. We apologize for any disruption this may have caused.

#### **Root Cause Analysis** **Incident:** Service Interruption Affecting Account Data Access **Date Range:** September 8, 2026 \(10:04 AM to 10:31 AM EDT\) **Impact:** DocV and Watchlist Monitors **Status:** Resolved ### 1. Summary An unexpected depletion of backend database capacity temporarily prevented our platform from retrieving account information for a subset of client requests. During the incident window \(10:04 AM to 10:31 AM EDT of 8-Sep-2026\), some of the client requests experienced elevated error rates when attempting to query account details. Following a routine maintenance update, concurrent connections from backup environments exceeded normal thresholds, saturating available database capacity. Engineering teams safely disconnected the redundant environments and refreshed active connections, restoring full system responsiveness and stability. ### 2. Timeline | **Date / Time \(ET\)** | **Event** | | --- | --- | | First error  | 10:04 AM EST, 8 Sep 2026  | | Incident reported | 10:17 AM EST, 8 Sep 2026  | | Redundant environment connection pool isolated | 10:30 AM EST, 8 Sep 2026  | | Incident resolved | 10:31 AM EST, 8 Sep 2026  | ### 3. Root Cause * **Primary Root Cause:** During a scheduled system update, elevated connection retention between deployment environments led to database connection pool saturation. Under peak traffic, this caused processing delays and request timeouts for dependent platform services. * **Contributing Factors:** * Rigid connection timeout parameters delayed the timely release of idle database allocations by passive environment. * Inadequate database node headroom hindered seamless processing of extra connections generated by the standby environment. * **Non-contributing Factors:** * None ### 4. Resolution Full service integrity was restored through the following immediate actions: 1. Isolating non-active environment connections to immediately free database pool capacity. 2. Performing a rolling reset of active service nodes to re-establish clear database connection baselines. ‌ ### 5. Corrective and Preventive Actions | **Action** | **Description** | **ETA / Status** | | --- | --- | --- | | Improvements to data access patterns | Improve caching strategy and data access patterns in internal services that rely on accounts data. | 15 Oct 2026 | | Automated Connection Management | Enhance connection lifecycle management and automated cleanup routines for inactive background tasks. | 15 Oct 2026 | ### 6. Lessons Learned * **Enhancing Deployment Isolation:** We are updating our deployment procedures to ensure complete separation of resources across active and passive environments, eliminating cross-environment resource contention during upgrades. * **Optimizing Database Resource Allocation:** To ensure peak performance, we are rebalancing our workload and increasing overall system capacity and efficiency. ### 7. Next Steps & Ongoing Commitment We are committed to: * Resolve the identified root causes by the timelines shared * Implement proactive safety guards and monitoring thresholds while permanent application and infrastructure enhancements are finalized.

Our Vendor, SSA, has reported that there is currently an elevated latency issue affecting eCBSV module. As a result, you may experience elevated response times. Our team is actively working with them to resolve the issue as quickly as possible. We apologize for any inconvenience this may cause and appreciate your patience and understanding.

SSA resolved the elevated latency affecting the eCBSV module as of 2:00 AM ET. Response times have returned to normal. Thank you for your patience and understanding.

We are investigating a slight degradation on the Watchlist product due to latency. Affecting both ID+ and RiskOS. The team is actively investigating this issue at this time.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

Between 1:21 AM and 2:15 AM EDT on August 4, 2026, a subset of DocV webhook delivery notifications experienced elevated 401 response rates. The issue is fully resolved and webhook delivery is now operating normally. A full Root Cause Analysis (RCA) will be shared once the investigation concludes.

We are currently investigating reports of increased latency for submitted transactions, which may result in processing delays. Our team is actively looking into the root cause and will provide updates as more information becomes available. We apologize for any inconvenience and appreciate your patience.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

A fix has been implemented and confirmed effective. All systems are operating normally. Thank you for your patience while we worked through this.