53 Askcody incidents · 2022年10月 — official updates, affected components, duration and resolution details.
今天11:40到13:15 CEST,10月13日,我们欧盟区域的一些AskCody客户可能已经看到一些更新的信息. 这可能表现为: - 缺少或过时的会议室预订 - 缺少餐饮订单 - 无法进出的客人 我们找到了原因:在今天的更新中出现了一个配置问题。 我们以13:15的CEST恢复了更新,AskCody现在再次显示当前信息。 11:40至13:15 CEST,10月13日的更改,如新订入,订入或来宾入住,可能不明显. 感谢您的耐心.
此事已决.
自动翻译自官方事件更新。
信息仪表板和会议室显示器上的房间可能会间歇地显示出错误("错误的未经授权")而不是会议,其他通过前提上交换连接读取日历的功能也会被间歇地影响,包括保存或更新交换连接证书. 在线交换(微软365)连接的客户不受影响。 会议数据本身是完好无损的——会议,预订,订单没有丢失. 下一次更新将会如出一辙.
已部署一个固定装置,服务正在恢复。 信息仪表板,会议室显示器,以及欧盟地区在前提上阅读的其他功能交换日历被确认再次工作,出错率已经恢复正常. 保存和更新Exchange连接证书也再次起作用. 我们将继续进行监测,然后才能解决这个问题.
这一事件已经得到解决.
自动翻译自官方事件更新。
受影响用户: AskCody 服务 区域:欧洲 我们查明了AskCody服务公司部分停工的原因,并开始寻找解决办法。 用户可能会遇到这一问题,因为交付没有跟踪,无法订购服务,包括管理门户服务部分的问题。 下一次更新将尽快提供
在监测了AskCody Services的已实施解决方案之后,与部分停电有关的问题已经解决,受影响的部分现已投入使用.
自动翻译自官方事件更新。
Affected Users: Workplace Central, Maps, All Bookings Region: Global We are currently experiencing issues with Workplace Central, Maps, All Bookings and are now investigating the impact. Users may experience this issue as locations not loading and slow response times. As a temporary workaround, users can reload the page or open it in a new browser window. The next update will be provided as soon as possible.
We have identified the issue being a faulty DB configuration. A fix has been applied and we are monitoring the platform.
The incident has been resolved and the affected components are operational again.
Affected Users: AskCody Insights Customers Region: Global Since 2 Jan 2026, AskCody insights has not been collecting data for meetings, due to a database misconfiguration. All other data in AskCody is intact and safe otherwise. We are actively working on a fix and all AskCody Insights data will be restored after the fix is applied.
Affected Users: AskCody Insights Customers Region: Global We have restored all AskCody Insights missing data and the module is operational again.
We are seeing unstable connections to the Exchange Web Services integration that powers our add-ins and other services dealing with Exchange meetings directly. Currently, it seems to only affect EU customers, but we are investigating the extent and impact as well as the root cause, and will keep you updated.
We have identified a small set of customers affected by this issue. We have hardened our systems to handle the effect of unintentional misconfigurations, but load-based side effects can still occasionally affect other customers. We are monitoring the platform and will provide updates when additional actions are taken and/or the impact has subsided.
After monitoring the implemented solution, the issue related to the EWS connectivity and the degraded performance has been resolved, and the incident will now be closed.
Affected Users: All Region: All customers in all regions We are currently experiencing a major outage on all components of the platform and are now investigating the cause. Users may experience this issue as errors saying the site cannot be reached. The next update will be provided at 8 pm CEST, unless cause is found before then, in which case we will update immediately. Major Outage Definition: A component is unavailable for all users. Affects the up-time calculation 100%.
Affected Users: All Region: All customers in all regions Microsoft Azure appears to be heavily impacted and everything points to this as the cause of the outage: https://azure.status.microsoft/en-us/status. We will follow any development we can while this persist. The next update will be provided at 9 pm CEST, unless cause is found before then, in which case we will update immediately. Major Outage Definition: A component is unavailable for all users. Affects the up-time calculation 100%.
The platform remains unavailable at this time, and we are continuing to monitor closely until full functionality is restored. Microsoft has deployed a fix for the ongoing Azure outage, and they are reporting gradual recovery across their services: https://azure.status.microsoft/en-us/status The next update will be provided at 10 pm CEST, unless the issue is resolved before then.
Microsoft has deployed a fix for the ongoing Azure outage and reports continued recovery across their services: https://azure.status.microsoft/en-us/status. Our platform is now operational again, but users may still experience disturbances or degraded performance while Microsoft's services stabilize. We continue to monitor the situation closely. Next update will be provided when more relevant information is available.
Affected Users: All Users - Worldwide After monitoring the implemented solution by Microsoft, the major outage has been resolved. AskCody has been operational ever since the initial solution was deployed by Microsoft. More information can be found on Microsoft's incident history page: https://azure.status.microsoft/en-us/status/history.
Affected Users: AskCody Insights Region: Europe We are currently experiencing a major outage in AskCody Insights Reports and are now investigating the impact. The data collections is still running and only the reports are affected. The next update will be provided as soon as possible.
Affected Users: AskCody Insights Region: Europe After monitoring the implemented solution, the major outage in AskCody Insights has been resolved and is now operational once again. The issue was connectivity issue of the AskCody Insights service. All data is intact and unaffected.
Affected Users: AskCody Insights Region: North America AskCody Customers using Insights will experience missing data from 09 Sept 2025 till 01 Oct 2025. We have identified the issue in our data extraction and have fixed it, so new data is available and intact. We are currently working on getting the data for the affected dates as well. Next update will be provided as soon as possible.
Affected Users: AskCody Insights Region: North America We have extracted the missing data and reports should show data again. We are monitoring the Insights module to ensure that everything continues to work.
Affected Users: AskCody Insights Region: North America After monitoring and verifying that the Insights module is working as intended, we are closing the incident.
Affected Users: All users in Central and Maps Region: All Customers We have an outage and have immediately identified the cause of the major outage in AskCody Bookings, Central and Maps. We are working on implementing a solution. Users may experience this issue as errors stating they do not have access to one of these three modules. The next update will be provided at latest 11 am UTC+1. Major Outage Definition: A component is unavailable for all users. Affects the up-time calculation 100%.
We misinformed in that the Bookings Add-in is fully operational and not impacted. We are still working towards a solution to solve the outage on Central and Maps, where you as a user will experience the outage in the form of denied access.
Affected Users: Central, Maps Region: Europe We have implemented a solution to the major outages in Central and Maps. We are currently monitoring the status.
After monitoring the implemented solution, the major outage has been resolved and is now operational once again.
Affected Users: [Add-in Users, Management Portal Users,.] Region: [Europe] We are currently experiencing a partial outage in the management portal and Outlook Add-ins and are investigating the issue. Users may experience this issue as timeouts and slow response times. As a temporary workaround, users can retry their requests after a few seconds. The next update will be provided as soon as possible.
We are continuing to investigate this issue.
Affected Users: [Add-in Users, Management Portal Users] Region: [Europe] We have identified and fixed the issue and we are actively monitoring the platform.
Affected Users: [Add-in Users, Management Portal Users,.] Region: [Europe] We can see that the issue persists and we are working on a solution. You will as a user continue to experience long loading times and timeouts. It will, in most cases help if you refresh the page/add-in. The next update will be provided as soon as we know more.
Affected Users: [Add-in Users, Management Portal Users,.] Region: [Europe] We can see that the issue persists and we are working on a solution. You will as a user continue to experience long loading times and timeouts. It will, in most cases help if you refresh the page/add-in. We have a support case going with Microsoft to try and resolve this and will continue to work on the issue until it has been resolved. The next update will be provided as soon as we know more.
Affected Users: [Add-in Users, Management Portal Users,.] Region: [Europe] We are still trying to solve this. You will as a user continue to experience long loading times and timeouts. It will, in most cases help if you refresh the page/add-in. We are in dialogue with Microsoft and will continue to work on the issue until it has been resolved.
Affected Users: [Add-in Users, Management Portal Users,.] Region: [Europe] We are aware that access to platform disappeared entirely and expect it to be back shortly.
Affected Users: [Add-in Users, Management Portal Users,.] Region: [Europe] The platform currently appears to be fully operational again and we are monitoring if this continues to be the case. You should not experience timeouts and loading errors currently.
We can see that the issue persists and we are working on a solution. You will as a user continue to experience long loading times and timeouts. It will, in most cases help if you refresh the page/add-in. The next update will be provided as soon as we know more.
The issue persists, and we are continuing to investigate possible causes. Users may still experience long loading times and timeouts. In many cases, refreshing the page or add-in can help temporarily. We are working to restore normal performance as soon as possible and will provide the next update once we have more information.
We are unfortunately still not able to resolve this issue and while we work towards a solution, we are also trying to help mitigate the situation while we solve it by making the platform available to you without errors. We hope to install a small fix that should help provide access but with slow loading times (no timeouts) soon. Users may still experience long loading times. In many cases, refreshing the page or add-in can help temporarily. We are working to restore normal performance as soon as possible and will provide the next update once we have more information.
We are still working to fully resolve this issue. In the meantime, we are taking steps to mitigate the impact and ensure you can continue using the platform without errors. A small fix will soon be deployed to improve access. While this may result in slower loading times, it should prevent timeouts. Please note that users may still experience delays. Refreshing the page or add-in often helps temporarily. We are committed to restoring normal performance as quickly as possible and will share another update as soon as we have more information.
We have identified and isolated the issue affecting the platform, and it should now be operating normally. While the platform is accessible normally again, some users may continue to experience slower loading times. In these cases, refreshing the page or add-in will help. We are continuing to monitor performance closely and will provide further updates if needed.
Affected Users: [Add-in Users, Management Portal Users] Region: Europe After monitoring the implemented solution, the issue related to the slowness/timeouts and the degraded performance has been resolved, and the incident will now be closed. The identified faulty endpoint has been isolated and the platform is operational again.
# \[EU\] Latency and timeouts from secondary services - Post Mortem This Postmortem and technical timeline follows the announcement of degraded performance on the AskCody platform, beginning on Wednesday, August 13th. ## Summary On August 13th at 13:12 CEST, parts of the AskCody platform became unresponsive with very long response times and timeout errors. The errors were periodic and were not caused by any changes made by AskCody. This report describes the timeline and remediation steps taken to resolve the issue. ## Incident overview On August 13th, 2025, at 13:15 CEST, our team was alerted to a severe drop in availability on the [app.onaskcody.com](http://app.onaskcody.com/) domain, serving both our management portal, add-ins, and the check-in kiosks to our EU customers. The alert was in response to a steep increase in response times that had manifested a few minutes earlier. ## Investigation and actions taken ### Initial response August 13, at 13:15 CEST: Detected drop in availability and influx in response times. August 13, at 13:15 CEST: Extended team investigation into the cause and effects of the incident. No conclusive evidence. ### Investigation, temporary solutions, and monitoring May 14 - 15: Investigations extended to include Microsoft Azure services as PaaS resources seemed likely to be involved. Working with Microsoft to profile latency through their network and services. ### Critical discovery August 18, 13:00 CEST: Tracking led to the identification of periodically failing calls to a third-party service that had made changes to their ingestion on the 13th. ### Resolution August 18,. 14:00 CEST: The issue was mitigated, and the incident status was set to “monitoring”. ## Root cause analysis The root cause of the incident was a misconfigured request to a third-party service. We had received timely advice to alter the configuration, but the fact that the endpoint still periodically accepted requests made the troubleshooting more complex. ## Preventative and supplementary measures To prevent similar incidents in the future, we have: * Extended alerting systems for availability tests. ## Conclusion Though this incident was not classified as **critical**, we still consider the impact to be serious. We apologize for any inconvenience caused by the service degradation and assure you that our platform is fully operational again. We remain committed to maintaining the highest security standards, following Microsoft best practices, and will continue to take proactive measures to safeguard our services. If you have any questions or need further information, please do not hesitate to contact our support team. Thank you for your understanding and continued trust in AskCody. Sincerely, **AskCody**
Affected Users: Global We are currently experiencing a major outage in the AskCody platform and are now investigating the impact. Users may experience this issue as Service unavailable errors. The next update will be provided as soon as possible
North America is operational again after a small down time. We are continuing to investigate this issue.
Affected Users: EU A temporary disturbance in our database caused the outage. All data is secure, safe and intact. We have implemented a solution to the major outages in all AskCody components and are currently monitoring the status.
The incident has been resolved.
Region: EU Customers We are currently experiencing issues with Cost Centers being unavailable. Users may experience this issue as Cost Centers either being unavailable or appear as "Hidden". The next update will be provided at latest 16:00 CET
The issue has been identified and a fix is being implemented.
We are continuing to work on a fix for this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
Affected Users: Delegate users Region: All customers We are currently experiencing issues with an error that occurs while booking services in delegate scenarios through the AskCody Services add-in, and we are investigating the cause. Users may experience this issue as being issued an error that states the meeting does not exist, but it may also occur in other formats. It is, however, specific for delegate bookings where you want to add services. As a workaround we recommend reserving a booking to order services without the delegate scenario or to contact the service provider to have them aid with ad-hoc bookings for urgent matters. The next update will be provided as soon as we know more, Degraded Performance Definition: The affected component is working but is slow or otherwise impacted in a minor way. Does not affect downtime.
Affected Users: Delegate users Region: All customers The issue related to the error stating the meeting does not exist and the degraded performance has been resolved, and this incident will now be closed.
We are currently investigating an issue causing partial outage in North America. Users might experience slow load times and failures.
Region: [North America] Customers We are currently experiencing a partial outage in North America and are investigating the issue. Users may experience this issue as slow response times and failures. The next update will be provided at 2 hour intervals or as soon as we have identified the issue.
Region: [North America] Customers We have implemented a solution for a faulty network configuration. Users might experience slow response times and failures while the fix propagates.
Region: [North America] Customers After monitoring the implemented solution the issue related to the partial outage has been resolved, and the affected components are now operational.
Affected Users: Azure AD synced users Region: Europe We have identified an ongoing issue with some of our Azure AD synchronization's and are investigating how to resolve the issue. For some customer on the European platform, this means that user changes are currently not being synced. As a temporary workaround, admin users can still manually create and add users and hosts, apply memberships and roles, to provide the necessary access while we work to resolve this. Everything will be restored once the sync is resolved, which will also help overwrite the manual added users during this period. The next update will be provided as soon as possible, Degraded Performance Definition: The affected component is working but is slow or otherwise impacted in a minor way. Does not affect downtime.
Affected Users: Azure AD synced users Region: Europe We have implemented a solution for the issue and are seeing most syncs are working as intended again. We do have a few we are awaiting and will monitor them, to ensure they also come back on track. We will continue to monitor all syncs over the next hours and days to ensure everything stays operational. The next update will be provided as soon as we have more. Degraded Performance Definition: The affected component is working but is slow or otherwise impacted in a minor way. Does not affect downtime.
Affected Users: Azure AD synced users Region: Europe We did not find resolution in our implemented fix and will continue to work to resolve this matter. This means the Azure AD sync will not function properly and may not push your changes through. You can still resolve to manuel edits and all will be sorted once the Azure AD integration is up and running again. We apologise for the inconvenience and extra effort it puts on all and will work as fast as we can to resolve the matter. Degraded Performance Definition: The affected component is working but is slow or otherwise impacted in a minor way. Does not affect downtime.
Affected Users: Azure AD synced users Region: Europe After monitoring the implemented solution, the issue related to the Azure AD sync and the degraded performance has been resolved, and the incident will now be closed. Degraded Performance Definition: The affected component is working but is slow or otherwise impacted in a minor way. Does not affect downtime.
Microsoft is experiencing worldwide issues with Exchange online, which can affect the ability to book meetings and see resource availability. AskCody is following the incident updates and we urge our customers to do the same. All AskCody services are up and operational, but can be affected by the Exchange online outage. More info: Only for exchange admins: https://portal.office.com/adminportal/home?#/servicehealth X: https://x.com/MSFT365Status/status/1860973220662280356
All AskCody systems are still operational, however the issue with Microsoft Exchange is ongoing and the following update was shared by Microsoft: Users may be unable to access Exchange Online using the following impacted connection methods: - Outlook on the web - Outlook desktop client Users may be unable to use the following features within Microsoft Teams: - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. Scope of impact: Impact is specific to users who are served through the affected infrastructure. Preliminary root cause: A portion of infrastructure which supports mailbox and calendar functionality isn't operating as expected, resulting in impact.
A fix by Microsoft has been implemented and should be applied to the affected users. For more info: https://x.com/MSFT365Status/status/1861047970143019048 As before, AskCody is up and operational.
This incident has been resolved.
Affected Users: All users who access AskCody Central, Maps or the mobile app. We are currently experiencing a partial outage in AskCody Central, Maps and the mobile app. and are investigating the issue. Users may experience this issue as a error "Code 400", when trying to change the selected location. The next update will be provided as soon as we know more, or latest Tuesday 22nd 8 am CEST
This incident has been resolved. We were able to identify the issue to be related to an update released to the AskCody Platform, which we therefore rolled back. All components are again fully operational.
Affected Users: Add-in and AskCody Management Portal users Region: Europe We are currently experiencing a major outage in accessing AskCody add-ins, check-in kiosks and the management portal and are now investigating the impact and cause. Users may experience this issue as being unable to login. The next update will be provided at latest: 2 pm CEST
We are continuing to investigate the issue. From a user perspective, the issue still persists, but we are making progress in identifying the root cause. Next update will latest be at 3.30 PM CEST. Should we see a reliable improvement in stability before 3.30 PM CEST, we will make sure to post it here.
We identified the root cause of the incident, and a fix was implemented immediately. We are currently monitoring the results. For now, all components are once again stabil and fully operational.
The solution to the incident worked as expected, and is therefore to be considered as resolved.
Affected Users: Add-in Users, Management Portal Users Region: Europe We have identified a degraded performance in accessing add-ins as well as the management portal. It can also appear on check-in kiosks and Displays Users may experience this issue as receiving af 500 internal error message. The next update will be provided as soon as possible Degraded Performance Definition: The affected component is working but is slow or otherwise impacted in a minor way. Does not affect downtime.
We are continuing to work on a fix for this issue.
Affected Users: Add-in Users, Management Portal Users Region: Europe The affected areas have been stable and we are currently monitoring the performance. The next update will be provided, should the situation change, at the earliest time possible. Degraded Performance Definition: The affected component is working but is slow or otherwise impacted in a minor way. Does not affect downtime.
Affected Users: Add-in Users, Management Portal Users Region: Europe The degraded performance has been resolved, and the incident will now be closed. Degraded Performance Definition: The affected component is working but is slow or otherwise impacted in a minor way. Does not affect downtime.
