Sap Conversational Ai outage history and incident timeline

US DC - Outage on SAP Conversational AI platform

Started September 10, 2021 at 4:00 AM UTC 0m

Pending

September 10, 2021 at 6:16 AM UTCresolved
There was an incident that caused SAP Conversational AI in the US datacenter to be intermittently unavailable on 9-Sep-2021 from 0409 UTC until 0538 UTC. During this interval both dialog and designtime were giving fluctuating responses. The issue has been addressed and we apologise for the inconvenience this may have caused you. SAP Conversational AI Team

Disruption on SAP Conversational AI platform

Started August 31, 2021 at 1:21 PM UTC 7h 37m

OutageMajor incident

Affected components

[EU10] Channels ConnectorSAP Conversational AI[EU10] Design Time[EU10] NLU Analysis[EU10] Dialog

August 31, 2021 at 1:21 PM UTCidentified
Dear Users, A problem has been identified by the SAP Business Technology Platform team and the issue resolution is in progress. This affects several critical functionalities of the SAP Conversational AI platform. We are working closely with the BTP team to resolve the issue. We thank you for your patience. You may also follow the progress at the SAP BTP status page here : https://sapcp.statuspage.io/incidents/2c071ngwz7gf Regards, The SAP Conversational AI team
August 31, 2021 at 1:38 PM UTCmonitoring
Dear Users, The service has been restored since 13:28 UTC on 31 Aug 2021 and we are monitoring to ensure stability. Regards, SAP Conversational AI
August 31, 2021 at 8:58 PM UTCresolved
This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Disruption on SAP Conversational AI platform

Started August 31, 2021 at 10:33 AM UTC 2h 48m

OutageMajor incident

Affected components

[EU10] Channels Connector[EU10] Design Time[EU10] NLU Analysis[EU10] Dialog

August 31, 2021 at 10:33 AM UTCinvestigating
Dear Users, We are currently experiencing disruption on our bot connector. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
August 31, 2021 at 11:04 AM UTCmonitoring
Dear Users, We have deployed a fix in production and the system is back to normal. We are monitoring the stability of the platform. Regards, The SAP Conversational AI team
August 31, 2021 at 1:20 PM UTCresolved
This incident has been resolved.

Disruption on SAP Conversational AI platform

Started August 30, 2021 at 9:24 AM UTC 12h 27m

OutageMajor incident

Affected components

[US10] Dialog[EU10] Channels ConnectorSAP Conversational AI[EU10] Design Time[US10] NLU Analysis[EU10] NLU Analysis[US10] Channels Connector[EU10] Dialog[US10] Design Time

August 30, 2021 at 9:24 AM UTCinvestigating
Dear Users, We are currently experiencing disruption on our platform. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
August 30, 2021 at 10:17 AM UTCidentified
Dear Users, The root cause has been identified by the SAP Business Technology Platform team and the issue resolution is in progress. We thank you for your patience. You may also follow the progress at the SAP BTP status page here : https://sapcp.statuspage.io/incidents/ms9lg7vk9mng Regards, The SAP Conversational AI team
August 30, 2021 at 12:44 PM UTCmonitoring
Dear Users, The issue is still being addressed, with the help of the SAP BTP core team, at a fundamental level to prevent reoccurrence. However the platform has been stabilised and is operational for the last two hours. We are continuing to closely monitor the situation and provide the best service to you. Your patience is highly appreciated. You may also follow the progress at the SAP BTP status page here : https://sapcp.statuspage.io/incidents/ms9lg7vk9mng Regards, The SAP Conversational AI team
August 30, 2021 at 3:41 PM UTCidentified
Dear Users, We are currently experiencing disruption on platform. The root cause has been identified by the SAP Business Technology Platform team and the issue resolution is in progress. We thank you for your patience. You may also follow the progress at the SAP BTP status page here : https://sapcp.statuspage.io/incidents/ms9lg7vk9mng Regards, The SAP Conversational AI team
August 30, 2021 at 5:27 PM UTCmonitoring
Dear Users, The issue is still being addressed, with the help of the SAP BTP core team, at a fundamental level to prevent reoccurrence. However the platform has been stabilised and is operational for the last one hour. We are continuing to closely monitor the situation and provide the best service to you. Your patience is highly appreciated. You may also follow the progress at the SAP BTP status page here : https://sapcp.statuspage.io/incidents/ms9lg7vk9mng Regards, The SAP Conversational AI team
August 30, 2021 at 9:51 PM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational for last two hours. We thank you for your patience. Regards, The SAP Conversational AI team

Disruption on SAP Conversational AI platform

Started August 26, 2021 at 7:30 AM UTC 42m

OutageMajor incident

Affected components

[EU10] Channels Connector[EU10] Design Time[EU10] NLU Analysis[EU10] Dialog

August 26, 2021 at 7:30 AM UTCinvestigating
Dear Users, We are currently experiencing disruption on our platform. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
August 26, 2021 at 8:12 AM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Outage on SAP Conversational AI platform

Started August 24, 2021 at 7:50 AM UTC 50m

OutageCritical incident

Affected components

[EU10] Channels Connector[EU10] Design Time[EU10] NLU Analysis[EU10] Dialog

August 24, 2021 at 7:50 AM UTCinvestigating
Dear Users, We are currently experiencing outages on our platform due to an issue with one of our backing services. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
August 24, 2021 at 7:58 AM UTCinvestigating
We are continuing to investigate this issue.
August 24, 2021 at 7:59 AM UTCidentified
Dear Users, We have identified the incident's root cause and we are working with the relevant platform team to get the backing service up again. The fix will be implemented as soon as possible. Thank you for your patience. Regards, The SAP Conversational AI team
August 24, 2021 at 8:24 AM UTCmonitoring
Dear Users, The system is back to normal. We are monitoring the stability of the platform. We regret the inconvenience caused to you and your users. Regards, The SAP Conversational AI team
August 24, 2021 at 8:40 AM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Outage on /dialog endpoints

Started August 22, 2021 at 12:45 AM UTC 45m

OutageCritical incident

Affected components

[EU10] Channels Connector[EU10] Design Time[EU10] NLU Analysis[EU10] Dialog

August 22, 2021 at 12:45 AM UTCinvestigating
Dear Users, We are currently experiencing outages on our /dialog endpoints. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
August 22, 2021 at 1:04 AM UTCmonitoring
Dear Users, We have deployed a fix in production and the system is back to normal. We are monitoring the stability of the . Regards, The SAP Conversational AI team
August 22, 2021 at 1:30 AM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Outage on SAP Conversational AI platform

Started July 13, 2021 at 12:33 PM UTC 3h 3m

OutageMajor incident

Affected components

SAP Conversational AI

July 13, 2021 at 12:33 PM UTCidentified
Dear Users, We have identified that an issue with one of the backing services of SAP Business Technology Platform, which is causing an issue on CAI platform. We are working with the support team to resolve the issues as soon as possible. Thank you for your patience. Regards, The SAP Conversational AI team
July 13, 2021 at 12:53 PM UTCmonitoring
Dear Users, The backing service status is healthy again and CAI's health checks are also reported to be normal. We will continue to monitor the situation. Regards, The SAP Conversational AI team
July 13, 2021 at 3:36 PM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Service degradation on SAP Conversational AI platform

Started July 1, 2021 at 3:47 AM UTC 6h 40m

OutageMajor incident

Affected components

SAP Conversational AI

July 1, 2021 at 3:47 AM UTCinvestigating
Dear Users, We are currently experiencing service degradation on our platform. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
July 1, 2021 at 4:21 AM UTCinvestigating
We are continuing to investigate this issue.
July 1, 2021 at 5:13 AM UTCinvestigating
We are continuing to investigate the issue. Apologies for the inconvenience. Regards, SAP Conversational AI Team
July 1, 2021 at 5:42 AM UTCidentified
Dear Users, We have identified the incident's root cause and we are working on a fix. The fix will be implemented as soon as possible. Thank you for your patience. Regards, The SAP Conversational AI team
July 1, 2021 at 7:01 AM UTCidentified
We are still working on the issue. Thank you for your patience. Regards, SAP Conversational AI Team
July 1, 2021 at 8:50 AM UTCidentified
Our team is still working on the resolution of the issue. Thank you for your patience. Regards, SAP Conversational AI Team.
July 1, 2021 at 9:21 AM UTCmonitoring
Dear Users, We have deployed a fix in production and the system is back to normal. We are monitoring the stability of the platform. Regards, The SAP Conversational AI team
July 1, 2021 at 10:27 AM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Outage on SAP Conversational AI platform

Started June 10, 2021 at 8:43 PM UTC 9h 22m

OutageCritical incident

Affected components

[US10] Dialog[EU10] Channels ConnectorSAP Conversational AI[EU10] Design Time[US10] NLU Analysis[EU10] NLU Analysis[US10] Channels Connector[US10] Design Time

June 10, 2021 at 8:43 PM UTCinvestigating
Dear Users, We are currently experiencing outages on our platform. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
June 10, 2021 at 9:53 PM UTCinvestigating
We are continuing to investigate this issue which seems related to an infrastructure issue (https://status.aws.amazon.com/#EU_block).
June 10, 2021 at 11:47 PM UTCidentified
The issue has been identified and partly fixed. The platform is back and can be used. We are still experience issues in our runtime and the dialog endpoint does not reply consistently. We are working on this last issue.
June 11, 2021 at 3:35 AM UTCidentified
SAP Business Technology Platform team is working to restore the service. An update will be provided regularly until the service is fully restored. Bot runtime is responding in most cases with exception of slow response time occasionally. We regret the inconvenience caused to you. Regards, SAP Conversational AI Team
June 11, 2021 at 4:30 AM UTCmonitoring
Dear Users, SAP BTP services have reported recovery, we do not see any alert in our quality of service. We expect that the services are back up and running. We will continue to monitor the platform. Regards, The SAP Conversational AI team
June 11, 2021 at 6:04 AM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Poor performance / Intermittent outage on SAP Conversational AI platform /dialog endpoints

Started June 8, 2021 at 7:32 AM UTC 3h 14m

IssuesMinor incident

Affected components

SAP Conversational AI

June 8, 2021 at 7:32 AM UTCinvestigating
Dear Users, We are currently experiencing poor performance and intermittent outages on our /dialog endpoints. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
June 8, 2021 at 7:53 AM UTCmonitoring
Dear Users, The platform is back to normal performance and we are monitoring for stability. We apologize for the inconvenience caused by the temporary issue. Regards, The SAP Conversational AI team
June 8, 2021 at 10:46 AM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Outage on SAP Conversational AI platform

Started May 21, 2021 at 5:53 PM UTC 2h 57m

OutageMajor incident

Affected components

[US10] Dialog[EU10] Channels ConnectorSAP Conversational AI[EU10] Design Time[US10] NLU Analysis[EU10] NLU Analysis[US10] Channels Connector[US10] Design Time

May 21, 2021 at 5:53 PM UTCinvestigating
Dear Users, We are currently experiencing outages on our platform. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
May 21, 2021 at 6:07 PM UTCmonitoring
Dear Users, We have fixed the issue in production and the system has returned to normalcy. We are monitoring the stability of the platform. Regards, The SAP Conversational AI team
May 21, 2021 at 8:50 PM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Degraded performance on Bot Runtime

Started April 29, 2021 at 11:51 AM UTC 25m

IssuesMinor incident

Affected components

[US10] Dialog[EU10] Channels Connector[US10] NLU Analysis[EU10] NLU Analysis[US10] Channels Connector

April 29, 2021 at 11:51 AM UTCinvestigating
Dear Users, We are currently experiencing degraded performance with intermittent outages on our /dialog endpoints. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
April 29, 2021 at 12:10 PM UTCmonitoring
Dear Users, We have deployed a fix in production and the system is back to normal. We are monitoring the stability of the platform. Regards, The SAP Conversational AI team
April 29, 2021 at 12:16 PM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Intermittent outage on /dialog endpoint

Started April 12, 2021 at 2:29 PM UTC 5h 45m

IssuesMinor incident

Affected components

[US10] Dialog[EU10] Channels Connector[US10] NLU Analysis[EU10] NLU Analysis[US10] Channels Connector

April 12, 2021 at 2:29 PM UTCinvestigating
Dear Users, Starting 9:51 am CET, we are experiencing intermittent outages on our /dialog endpoints. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
April 12, 2021 at 4:03 PM UTCmonitoring
Dear Users, We have identified the intermittent outages in our /dialog endpoint to be caused by an update of the underlying infrastructure that caused individual instances to be unavailable for short period of time, leading to some requests running into a timeout. Only a small amount of the overall /dialog requests were impacted. We are monitoring the stability of the /dialog endpoints and are further analysing the root cause for the unavailability of the instances. Regards, The SAP Conversational AI team
April 12, 2021 at 8:14 PM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Outage on /dialog endpoints

Started March 24, 2021 at 4:09 PM UTC 2h 28m

OutageCritical incident

Affected components

SAP Conversational AI

March 24, 2021 at 4:12 PM UTCmonitoring
Dear Users, We have deployed a fix in production and the system is back to normal. We are monitoring the stability of the /dialog endpoints. Regards, The SAP Conversational AI team
March 24, 2021 at 4:14 PM UTCinvestigating
Dear Users, We are currently experiencing outages on our /dialog endpoints. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
March 24, 2021 at 4:22 PM UTCidentified
Dear Users, We have identified the root cause of the outage and we are working on a fix. Fix will be implemented as soon as possible. Thank you for your patience. Regards, The SAP Conversational AI team
March 24, 2021 at 4:42 PM UTCmonitoring
Dear Users, We have deployed a fix in production and the system is back to normal. We are monitoring the stability of the dialog endpoints. Regards, The SAP Conversational AI team
March 24, 2021 at 6:37 PM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Design Time Login Outage

Started March 15, 2021 at 7:16 PM UTC 2h 34m

OutageMajor incident

Affected components

SAP Conversational AI[EU10] Design Time

March 15, 2021 at 7:16 PM UTCinvestigating
We are currently investigating an issue impacting login to our SAP Conversational AI design time. Login operations are currently not possible. Bot runtime is fully operational in the meantime. We will report back with updates at least every 30 minutes until the issue is resolved.
March 15, 2021 at 7:51 PM UTCinvestigating
We are still investigating the issue impacting our bot design time with our experts. We will post an update once the issue is identified or every 30 minutes to report on the progress of the investigation. The bot runtime remains fully operational in the meantime.
March 15, 2021 at 8:43 PM UTCmonitoring
We have addressed the issue and design time authentication is fully operational again. Our teams have identified an issue with a previously aborted database migration that prevented our design time authentication service from restarting. After addressing this issue, we were able to restart the affected service instances. We will conduct a proper root cause analysis to identify the reason for the crash and prevent similar situations in the future.
March 15, 2021 at 9:50 PM UTCresolved
Our systems remain fully operational, so we're marking this issue as resolved. As noted earlier, our teams will conduct a proper root cause analysis to identify the reason for the crash and prevent similar situations in the future. Thank you for being a valued user of SAP Conversational AI and your patience while the design time was not accessible.

Outage on SAP Conversational AI platform related to login

Started March 11, 2021 at 11:05 AM UTC 8h 7m

OutageMajor incident

Affected components

[EU10] Channels ConnectorSAP Conversational AI[EU10] Design Time

March 11, 2021 at 11:05 AM UTCinvestigating
Dear Users, We are currently experiencing issues with the login to our platform. Our team is working on identifying and fixing the issue. We apologize for the inconvenience. Regards, The SAP Conversational AI team
March 11, 2021 at 1:12 PM UTCinvestigating
Dear Users, we're still investigating the issue. Only the community edition of SAP Conversational AI is affected, as the problem is related to the login with SAP accounts. There is no impact on the bot runtime or to the enterprise edition of SAP Conversational AI. Regards, The SAP Conversational AI team
March 11, 2021 at 1:33 PM UTCidentified
Dear Users, We have identified the root cause of the incident and we are working on a fix. The fix will be implemented as soon as possible. Thank you for your patience. Regards, The SAP Conversational AI team
March 11, 2021 at 1:43 PM UTCmonitoring
Dear Users, the SAP Business Technology Platform deployed a fix in production and the login to the SAP Conversational AI community edition works correctly now. We are monitoring the stability of the platform and specifically the login to the community edition. Regards, The SAP Conversational AI team
March 11, 2021 at 1:50 PM UTCmonitoring
We are continuing to monitor for any further issues.
March 11, 2021 at 7:11 PM UTCresolved
Dear Users, This incident has been resolved and the platform is fully operational. We thank you for your patience. Regards, The SAP Conversational AI team

Outage reported during hotfix deployment

Started March 2, 2021 at 5:00 PM UTC 0m

Pending

March 3, 2021 at 2:36 AM UTCresolved
Dear Customers, At 5:05 pm UTC, our operations team triggered the deployment of a hotfix on the platform. An outage on our /dialog endpoint was reported by QoS health checks. The team analyzed the incident and fix the issue shortly. The outage lasted about 10 min, during which end users were unable to converse with their bots. Our team is conducting a Root Cause Analysis to determine the actions required to make our deployment process more robust. We apologize for the inconvenience caused. Regards, SAP Conversational AI Team

Issue with SAP IDP basic auth logins

Started February 24, 2021 at 5:41 AM UTC 6h 10m

IssuesMinor incident

Affected components

[EU10] Design Time[US10] Design Time

February 24, 2021 at 5:41 AM UTCinvestigating
Dear Customer, Due to an ongoing issue with the SAP Identity Provider, developer login to the SAP Conversational AI design time maybe affected if you use the SAP IDP and do not have a certificate based login mechanism. Developers that use username/password for login into the SAP IDP may face an error. We are closely monitoring the issue and will update this incident with the progress. We deeply regret the inconvenience caused due to this issue. Regards, SAP Conversational AI Team
February 24, 2021 at 7:26 AM UTCmonitoring
Dear Customer, The SAP IDP service has been restored and we are monitoring for stability. Bot developers should now be able to use username/password for login into the SAP Conversational AI design time. Regards, SAP Conversational AI Team
February 24, 2021 at 11:51 AM UTCresolved
Dear Customers, This issue has been resolved and the platform is stable for design time logins that use SAP IDP. We thank you for your support and patience. Regards, SAP Conversational AI team

Outage on SAP Conversational AI platform /dialog endpoints

Started February 19, 2021 at 12:00 PM UTC 3h 17m

OutageCritical incident

Affected components

SAP Conversational AI

February 19, 2021 at 12:27 PM UTCinvestigating
We are currently experiencing outages on our /dialog endpoints. Our team is working on identifying and fixing the issue. We apologize for the inconvenience.
February 19, 2021 at 12:33 PM UTCidentified
We have identified the root cause of the outage and we are working on a fix. Fix will be implemented as soon as possible.
February 19, 2021 at 12:41 PM UTCidentified
We are still working to resolve the issue. We regret the inconvenience caused. Regards, SAP Conversational AI Team
February 19, 2021 at 12:49 PM UTCmonitoring
We have deployed a fix in production and the system is back to normal. We are monitoring the stability of the platform.
February 19, 2021 at 3:17 PM UTCresolved
This incident has been resolved and the platform is fully operational.