Cornerstone outage history

This swimlane is experiencing some functional issues accessing the xAPI Statement viewer and may have issues with xAPI statement processing. Clients with portals in this swimlane may experience delays or issues executing some tasks. This is our top priority and we are working to resolve the problem as soon as possible. Please check back periodically for additional updates, which will be posted as they become available.

We are continuing to investigate this issue and will share an update at least every 2 hours or until we have more.

Cornerstone Engineering has identified the issue and is currently working on a fix. We will update once the issue is Resolved.

We are still working on a fix and will confirm once it has been successfully deployed. This remains a top priority for our teams and we will continue sharing updates as they become available or at least every 2 hours.

A fix has been deployed and we are seeing no errors. We will monitor for 1 hour before placing this in a Resolved status.

After a period of monitoring with no repeat occurrences we are marking this incident Resolved. A formal Root Cause Analysis will be available 7-10 business days from Resolution.

A production incident occurred in the EU Production environment hosted in Frankfurt (FRA PRD), resulting in the Careersite service becoming completely unavailable. This incident has been classified as a high-severity issue. Resolving this problem is our top priority, and we are working diligently to restore service as quickly as possible.

Career Site is now accessible. We'll keep monitoring the portals.

We are continuing to monitor for any further issues.

This incident has been resolved.

The CSOD Technology Team observed an issue affecting Edge Import on FRA and CDG swimlanes. The problem began on September 2nd 2026, at 10:00 PM Pacific Time and was resolved on September 2nd 2026, at 11:30 PM Pacific Time. During this time, clients may have experienced issues or delays with Edge Import functionalities.

The swimlane is experiencing some latency issues. Clients with portals across US Swimlanes may experience delays or issues accessing certain pages in the application. This is our top priority and we are working to resolve the problem as soon as possible. Please check back periodically for additional updates, which will be posted as they become available.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

Clients on US SL5 may encounter errors or issues when attempting to log in to Production. Our Engineering team has identified the root cause on the backend and is actively working on a fix as the highest priority. We will continue to provide regular updates as they become available.

We have received confirmation from the CSOD Engineering team that the issue has been fixed and the service has been fully restored. Our Engineering team is currently monitoring the fix to ensure its reliability and stability. We will provide further updates if there are any additional developments.

The CSOD Technology Team observed a service disruption on US SL5. The issue began at 6:47 AM Pacific Time and service was restored at 6:56 AM Pacific Time. During this time, clients with portals on this swimlane were unable to access the application. Following a brief period of monitoring, the fix remains stable and the service continues to operate as expected.

A production incident occurred in the EU Production environment hosted in Frankfurt (FRA PRD), resulting in the Careersite service becoming completely unavailable. This incident has been classified as a high-severity issue. Resolving this problem is our top priority, and we are working diligently to restore service as quickly as possible. Please check back periodically for additional updates, which will be posted as they become available.

We are continuing to investigate this issue.

A fix has been implemented and we are monitoring the results.

After a period of monitoring and no recurrence, this is resolved.

**Root Cause Analysis** ‌ **Incident Summary:** On August 28th, 2026, users in the FRA SL1 Production environment experienced intermittent issues accessing Career Site functionalities. **Impact:** Intermittent failures while accessing Career Site functionality, resulting in a degraded user experience and partial service disruption during the incident window. **Root Cause:** The issue was caused by a sudden increase in traffic that temporarily exceeded the available service capacity. During the resulting scale-up activity, newly provisioned application capacity required additional time to become fully operational and could not immediately accommodate the increased demand. The limited capacity buffer combined with the application startup time resulted in intermittent service failures until sufficient capacity was available to handle the traffic. **Resolution:** The service capacity and scaling configuration were adjusted to maintain an appropriate capacity buffer and better accommodate sudden increases in traffic. The affected service was also redeployed to restore stability. **Preventive Measures:** * Application startup and initialization processes will be optimized to reduce the time required for newly provisioned capacity to become fully operational * Baseline capacity and scaling thresholds will be reviewed and optimized to maintain sufficient buffer for sudden increases in traffic * Capacity and application health will continue to be monitored during traffic fluctuations to ensure sufficient resources are available to maintain service stability

A production incident occurred in the EU production environment, impacting Career Sites hosted in CDG SL1. As of 3:35 AM PT the portals are back to function.

**Root Cause Analysis**   **Incident Summary:** On August 27th, 2026, users in the CDG SL1 Production environment experienced intermittent issues accessing Career Site functionalities. **Impact:** Intermittent failures while accessing Career Site functionality, resulting in a degraded user experience and partial service disruption during the incident window. **Root Cause:** The issue was caused by a sudden increase in traffic that temporarily exceeded the available service capacity. During the resulting scale-up activity, newly provisioned application capacity required additional time to become fully operational and could not immediately accommodate the increased demand. The limited capacity buffer combined with the application startup time resulted in intermittent service failures until sufficient capacity was available to handle the traffic. **Resolution:** The service capacity and scaling configuration were adjusted to maintain an appropriate capacity buffer and better accommodate sudden increases in traffic. The affected service was also redeployed to restore stability. **Preventive Measures:** * Application startup and initialization processes will be optimized to reduce the time required for newly provisioned capacity to become fully operational * Baseline capacity and scaling thresholds will be reviewed and optimized to maintain sufficient buffer for sudden increases in traffic * Capacity and application health will continue to be monitored during traffic fluctuations to ensure sufficient resources are available to maintain service stability

Production environment on AU, JP, FR, US swimlanes were experiencing a service disruption when accessing from CHINA region. Issue lasted around 50 minutes. The CSOD Technology Team observed a service disruption on these swimlanes. Our engineering team has resolved the issue and services are back to normal for users in China.

**Root Cause Analysis**   **Incident Summary:** On August 25th, 2026, an issue was identified affecting select customers using the China Content Delivery Network \(CDN\). The issue impacted requests for specific web resources across US SL1, US SL5, FRA SL1, JP SL1, and AU SL 1 Production environments. **Impact:** Affected users experienced errors when accessing certain resources through the China CDN during the incident window. The impact was limited to customers using the China CDN. **Root Cause:** The issue was caused by a configuration optimization implemented to improve resource caching behavior. The change resulted in an unexpected routing condition, which caused an incorrect SSL certificate to be presented for affected requests. As a result, requests for the impacted resources were rejected and returned errors to users. **Resolution:** The configuration was reverted to the previous stable state, which removed the unexpected routing condition and restored the correct SSL configuration. Following the change, the affected resources were validated and service was restored for the impacted customers. **Preventive Measures:** * Additional validation will be performed for CDN configuration changes to confirm correct routing and SSL behavior before and after implementation * Configuration changes impacting CDN caching and routing will undergo additional review prior to implementation

The US SL5 swim lane is experiencing some latency issues. Clients with portals in this swimlane may experience delays or issues accessing certain pages in the application. This is our top priority and we are working to resolve the problem as soon as possible. Please check back periodically for additional updates, which will be posted as they become available.

The CSOD Technology Team observed a performance degradation affecting the US SL5 swimlane. The problem began at 4:25 AM Pacific Time and was resolved at 6:35 AM Pacific Time. During this time, clients with portals on this swim lane may have experienced slow performance or intermittent errors while accessing the application.

**Issue Summary:** On August 24, 2026, users in the US SL5 Production environment experienced significant latency across multiple portals. The issue resulted in elevated response times and degraded portal responsiveness for affected users. **Root Cause:** The issue was caused by a degraded execution plan associated with a high-frequency database operation. The change in execution behavior caused the operation to consume significantly more database resources than expected, resulting in elevated CPU utilization and increased latency across the environment. Concurrent high-volume workloads further contributed to resource contention during the incident and amplified the overall performance impact. **Corrective Action:** Cornerstone cleared the problematic cached execution plans and recompiled the affected database procedure to generate a new execution plan. Additional workload was temporarily controlled to reduce resource consumption while the environment recovered. Following these actions, database utilization decreased, portal response times improved, and the environment returned to normal performance levels. **Preventive Measures:** To reduce the risk of a similar issue, Cornerstone has taken the following actions: * Reviewing execution plan behavior for the identified high-frequency database operation to reduce the likelihood of future plan degradation. * Reviewing high-volume workloads and their resource consumption to reduce the risk of excessive database contention during periods of elevated activity.

The UK Swimlane is experiencing issues while accessing the Edge Imports and Feeds. Clients may experience delays or issues while accessing the Edge imports. This is our top priority and we are working to resolve the problem as soon as possible. Please check back periodically for additional updates, which will be posted as they become available.

Edge Imports and Feeds are now accessible on the UK swim lane. The problem began at 6:00 AM Pacific Time and was resolved at 7:03 AM Pacific Time. During this time, clients may have experienced issues while accessing the Edge Imports and Feeds.

**Issue Summary:** Starting August 20, 2026, users in the UK Production, Pilot, and Stage environments, including Metpol Production and Pilot, experienced disruption affecting Edge Import functionality. **Root Cause:** The issue occurred during a scheduled cleanup activity when the associated cleanup jobs failed across the affected environments. The failed jobs contributed to instability within the supporting services, which impacted Edge Import availability and processing. **Corrective Action:** Cornerstone reviewed the affected service environments and reset the associated capacity to restore service stability. Once the services had recovered, the cleanup jobs were successfully re-run. The affected environments were then monitored to confirm that service health and Edge Import functionality had returned to normal. **Preventive Measures:** To reduce the risk of a similar issue, Cornerstone has taken the following action: * Strengthened post-maintenance validation to confirm service health before and after scheduled cleanup activities.

Error when creating vILT session (Microsoft Teams VILT Integration): "Error while creating Schedule: Exception of type 'System.ArgumentException' was thrown." Cause has been found and we are working on the fix.

This incident has been resolved.

**Root Cause Analysis** ‌ **Incident Summary:** On August 20th, 2026, users in the UK Production and Pilot environments, across all swimlanes, experienced issues creating VILT sessions. **Impact:** Users were unable to create VILT sessions, resulting in temporary disruption to VILT-related functionality. **Root Cause:** The issue was related to the availability of the required messaging queues used by the VILT integration service. The queues were not available as expected, resulting in exceptions and preventing the service from processing VILT requests successfully. **Resolution:** The VILT integration service was redeployed in the UK Production and Pilot environments. Following the redeployment, the required queues were successfully established and the exceptions stopped. VILT session creation functionality was restored. **Preventive Measures:** * Queue availability and service startup dependencies are being reviewed to ensure the required queues are available during service initialization * Startup and redeployment sequencing are being reviewed to prevent similar queue initialization issues in the future

The CSOD Technology Team identified and resolved an issue impacting access to Imports and Feeds in Edge, which caused loading failures, navigation errors, and, in some cases, automatic disabling of the CHR > Employee Load Enablement setting. The problem observed at August 19, 2026, 9:00 AM PT, and full functionality was restored as of August 19, 2026, 12:10 PM PT.

FRA SL1 swim lane is experiencing issues with the Career Site. Clients may experience delays or issues accessing the Career Site. This is our top priority and we are working to resolve the problem as soon as possible. Please check back periodically for additional updates, which will be posted as they become available.

The Career Sites should now be accessible. Our Engineering team investigated promptly and have implemented the necessary corrective measures. The problem began at 4:40 AM Pacific Time and the service was restored at 5:58 AM Pacific Time.

**Root Cause Analysis**   **Incident Summary:** On August 19th, 2026, users in the EU Production environment experienced intermittent issues accessing Career Site functionalities. **Impact:** Users experienced intermittent failures while accessing Career Site functionality, resulting in degraded user experience and partial service disruption during the incident window. **Root Cause:** The issue was caused by a sudden increase in traffic that temporarily exceeded the available service capacity during scaling activity. Due to the service's sensitivity to startup time, the newly provisioned capacity was unable to stabilize quickly enough to handle the increased traffic. This resulted in intermittent service failures. **Resolution:** The service capacity and scaling configuration were adjusted to maintain sufficient baseline capacity and better accommodate sudden increases in traffic. The affected service was also redeployed to restore stability. **Preventive Measures:** * Scaling configuration has been optimized to better respond to sudden traffic increases * Additional capacity and startup behavior are being monitored to reduce the likelihood of recurrence

User accounts configured with Local Authentication and Multi-Factor Authentication (MFA) are currently unable to log in to the Production portal. After users enter valid credentials and submit the login request, a blank error modal is displayed and the MFA challenge is not initiated. The issue occurs across multiple browsers, including InPrivate/Incognito sessions. That said, end users with the ability to login via SSO, can login successfully. Our Engineering Team is actively looking into this concern, we will keep you posted of any significant updates. Thank you!

Our Engineering Team has confirmed that the issue has been fixed following the restoration of the MFA service. We will continue to monitor the service for a period of time to ensure stability and reliability.

Following a brief monitoring period, the issue has been successfully resolved, and the service has remained stable. The issue began at 6:45 AM PT and was resolved at 7:26 AM PT on August 17, 2026. The Root Cause Analysis (RCA) will be provided within 7–10 business days. Thank you for your patience and understanding!

Root Cause Analysis ‌ **Incident Summary:** On August 17th, 2026, customers across US Production environments experienced issues logging in due to Multi-Factor Authentication \(MFA\) failures. Affected users were unable to complete the authentication process and access the application. **Impact:** Users across US Production environments experienced intermittent login failures due to unavailability of the MFA service. **Root Cause:** The issue was caused by insufficient memory allocation for the MFA service. Service tasks became unstable and were repeatedly terminated due to memory exhaustion, resulting in no healthy endpoints being available to process authentication requests. **Resolution:** Additional capacity was provisioned; however, the service continued to experience memory-related instability. The memory allocation for the service was subsequently increased, after which the service tasks stabilized and healthy endpoints became available. Login functionality was restored. **Preventive Measures:** * Service resource utilization and memory requirements were reviewed to ensure appropriate capacity is maintained * Additional health checks will be incorporated to help identify potential service degradation before it impacts customer authentication * Resource requirements will be reviewed as part of future capacity planning and scaling activities

Admins are currently unable to edit user records in the Production environment. When attempting to open a user record for editing through Admin > Tools > Core Functions > Users, the request fails and an error message is displayed ("An error occurred while processing your request"). The issue has been observed across multiple US SL4 (AWS) portals. Our Engineering Team is actively looking into this concern. We will keep you posted on any significant updates.

UPDATE: The issue seems to be impacting the Core User Management functionality (Example: unable to view or edit user records, View Universal profile, API services related to User record management) Our Engineering Team has confirmed the issue to be resolved. We will continue to closely monitor the situation to check its reliability.

Following a period of monitoring, the implemented fix has remained stable, and this issue has been confirmed as resolved. Thank you for your patience and understanding!

**Incident Summary** Starting August 10th, 2026, users in the US RES PRD environment experienced intermittent issues with user management functionality and other portal functions dependent on core user services.   **Impact** Affected users were intermittently unable to edit user records and may have experienced timeouts or errors when accessing other functionality dependent on core user services. The impact was limited to requests routed through the affected capacity, while healthy capacity continued to serve requests normally.   **Root Cause Analysis \(RCA\)** The issue was related to instability affecting a subset of application capacity. Multiple services became unhealthy on the affected nodes, resulting in intermittent request failures and timeouts for functionality dependent on those services. The remaining healthy capacity continued to serve requests normally.   **Resolution** The affected nodes were identified and recycled to restore service health. Following the recovery actions, the affected services returned to a healthy state and intermittent timeout issues were resolved.   **Preventive Measures** * Monitoring and alerting for unhealthy application capacity will continue to be reviewed to ensure timely identification and remediation of similar conditions. * Post-recovery validation will be performed to confirm that affected capacity has returned to a healthy state before normal traffic handling is resumed.

Swimlanes in EU are experiencing issues with the Career Site. Some Clients may experience delays or issues accessing the Career Site. The issue reported earlier today(https://status.csod.com/incidents/m5ppwktbx4rt) has resurfaced. This is our top priority and we are working to resolve the problem as soon as possible. Please check back periodically for additional updates, which will be posted as they become available.

The Career Sites should now be accessible. Our Engineering team investigated promptly and has implemented the necessary corrective measures. We will continue to closely monitor the environment to ensure the issue remains resolved and the career sites remains stable.

Following a period of monitoring, the implemented fix has remained stable, and this issue has been confirmed as resolved. Thank you for your patience and understanding!

**Incident Summary** Starting August 10th, 2026, users experienced intermittent issues accessing the Career Site in the Frankfurt \(EU\), FR EXT\(CDG\) , FR RES\(CDG\) and UK \(all swimlanes\) Production environment.   **Impact** Users experienced intermittent access failures and degraded responsiveness while accessing the Career Site during the incident window.   **Root Cause Analysis \(RCA\)** The issue was related to an environment change that resulted in reduced available application capacity during EMEA region business hours. The newly provisioned capacity was not available as expected, which placed increased load on the existing application capacity. This resulted in instability and intermittent request failures affecting the Career Site.   **Resolution** The environment was stabilized by restoring the required application capacity and reverting the changes contributing to the instability. Following the recovery actions, application stability was restored and Career Site access returned to normal.   **Preventive Measures** * Post-change validation will be strengthened to confirm that newly provisioned capacity is fully available before changes are considered complete. * Capacity and application health will be monitored following environment changes, particularly during business hours, to identify similar conditions earlier.

FRA SL1 swim lane is experiencing issues with the Career Site. Clients may experience delays or issues accessing the Career Site. This is our top priority and we are working to resolve the problem as soon as possible. Please check back periodically for additional updates, which will be posted as they become available.

The CSOD Technology Team observed an issue on FRA SL1 while accessing Career Site. The problem began at 1:05 AM Pacific Time and service was restored at 2:52 AM Pacific Time. During this time, clients with portals on FRA SL1 were not able to access the Career Site.

**Incident Summary** Starting August 10th, 2026, users experienced intermittent issues accessing the Career Site in the Frankfurt \(EU\), FR EXT\(CDG\) , FR RES\(CDG\) and UK \(all swimlanes\) Production environment.   **Impact** Users experienced intermittent access failures and degraded responsiveness while accessing the Career Site during the incident window.   **Root Cause Analysis \(RCA\)** The issue was related to an environment change that resulted in reduced available application capacity during EMEA region business hours. The newly provisioned capacity was not available as expected, which placed increased load on the existing application capacity. This resulted in instability and intermittent request failures affecting the Career Site.   **Resolution** The environment was stabilized by restoring the required application capacity and reverting the changes contributing to the instability. Following the recovery actions, application stability was restored and Career Site access returned to normal.   **Preventive Measures** * Post-change validation will be strengthened to confirm that newly provisioned capacity is fully available before changes are considered complete. * Capacity and application health will be monitored following environment changes, particularly during business hours, to identify similar conditions earlier.

This swimlane is experiencing functional issues accessing Edge Import. Clients with portals in this swimlane are unable to access or use the import. This is our top priority, and we are working to resolve the problem as soon as possible. Please check back periodically for additional updates, which will be posted as they become available.

This incident has been resolved.

**Issue Summary:** On July 27, 2026, beginning at approximately 7:58 a.m. Pacific Time, customers in the US SL4 environment experienced issues accessing Edge Import. Affected customers encountered prolonged page load times and intermittent errors, temporarily impacting the availability and usability of Edge Import functionality. **Root Cause:** The issue was caused by an application configuration inconsistency introduced during a recent deployment. This inconsistency prevented the affected service from communicating normally with a dependent component, resulting in delayed request processing and intermittent errors. **Corrective Action:** Cornerstone corrected the affected application configuration and recycled the application service to restore normal functionality. Following these actions, the team validated Edge Import availability and confirmed that the service was operating as expected. **Preventive Measures:** To reduce the risk of a similar issue, Cornerstone has taken the following actions: * Enhanced deployment validation procedures to verify application configurations after deployment activities. * Strengthened post-deployment health checks to confirm successful service initialization and connectivity before services are made available.

Some customers in UK, France and Japan swimlanes are experiencing issues while logging into the system. This is our top priority and we are working to resolve the problem as soon as possible. Please check back periodically for additional updates, which will be posted as they become available.

The fix has been successfully deployed to all Production instances across all regions. We are currently working on deploying the fix to the lower environments. In the meantime, we will continue to closely monitor the Production environments to ensure the solution remains stable and no further issues are observed.

The fix has been successfully deployed to all the environments (Production and lower environments). We will continue to closely monitor the environments to ensure the solution remains stable and no further issues are observed.

Some customers in UK, France and Japan swim lanes experienced issues while logging into the system. The problem began on 23rd July 7:10 PM Pacific Time and was resolved by 8:20 PM Pacific Time.

**Root Cause Analysis \(RCA\)** ‌ **Incident Summary:** On July 23rd, 2026, users across the UK, JP, and EU Production environments experienced intermittent login failures. The issue primarily affected users authenticating using Multi-Factor Authentication \(MFA\), resulting in temporary portal access issues. **Impact:** Temporarily impacted authentication and portal accessibility for affected users. **Root Cause:** The issue was related to transient instability within the authentication processing flow, which impacted the successful processing of a subset of login requests. This resulted in intermittent authentication failures for affected users until normal service processing was restored. **Resolution:** As an immediate mitigation, the affected authentication application components were recycled across the impacted regions to restore normal request processing. Following the recovery actions, authentication services stabilized and successful user login was validated. **Preventive Measures:** The following measures have been implemented to reduce the likelihood of recurrence: * Application monitoring has been reviewed to improve visibility into authentication service health and login processing. * Operational procedures have been updated to enable faster identification and recovery of authentication service interruptions.

Customers across all swimlanes may experience delayed or stuck processing of queued events and background jobs (i.e., Content Uploader, Emails, Reporting, Roster Submissions etc.). We are actively investigating this issue as a top priority, and will provide regular updates until it is fully resolved.

We are continuing to investigate this issue.

We have taken action to expedite the processing of the backlog of queued events, and most swimlanes have now fully recovered. We will provide a final update once processing has fully returned to normal across all swimlanes.

The backlog of queued events has largely cleared across all Production environments. We are now reviewing the queues in all Pilot and Stage environments and taking action to expedite processing of any remaining backlog.

Queues have been caught up across all environments and this issue is now Resolved.

**Root Cause Analysis** ‌ **Issue Summary:** There are backend processes in place that are designed to handle course and subscription updates reliably, without causing delays in downstream processing. During the incident, a very large update covering many courses created a backlog in the processing system, which led to delayed updates from reaching downstream services. **Root Cause:** The disruption happened because an oversized bulk update was allowed to proceed, and the available safeguards were not strong enough to prevent it from overwhelming the processing queues. As a result, the system spent more time working through the backlog than processing new changes, delaying course update propagation downstream. The incident recurred because user behavior alone was not a reliable control, and enforcement of prior guidance was not sufficient. **Corrective Action:** On July 21st, 2026, Cornerstone Engineering placed a block to prevent further incoming events of a specific type from entering the queue. Following that action, Cornerstone Engineering ran an emergency data fix to stop processing already queued items, helping the system recover faster and reducing the backlog’s impact. **Preventive Action:**  * Implemented stronger permission controls so only super admin users can perform high-impact actions such as approving, bulk approving, bulk uploading, and publishing subscriptions * Updated the emergency script to support running within a specific date range and made it more widely available across our teams * Updated our emergency runbook for queue overload incidents, so response is faster and more consistent next time