Mindtickle 中断历史

1. 2026年10月24日 UTC 13:19resolved

   内容提要 • 2026年10月21日,PHP运行时间的两个关键弱点被公开披露。 ^ Mindtickle平台,包括客户学习站点和自定义域部署,主要在Go,JavaScript,TypeScript,Python,和Java等平台上被构建而不受影响. 受影响部分 · 一个辅助部分使用了PHP:应用中的“报告一个问题”反馈表的后端服务。 它与同一支持面板中的帮助文章搜索和聊天功能是分开的,这些功能运行于其他服务上而不受影响. • 它提供反馈表,并在用户提交报告时收到报告。 该服务将报告转交给我们的支援小组的服务台。 • 它与Mindtickle核心平台分开运行。 它不是提供学习网站、用户如何签字、学习内容和租户数据如何储存或处理的一部分。 它无法进入掌握这些数据的系统。 补救 • 2026年10月23日,我们在准备升级时,采用了PHP安全咨询中建议的减轻风险的配置变化。 • 2026年10月24日,我们把部件升级为PHP 8.3.36,即固定放行,覆盖所有生产区域,并核实了固定放行。 • 我们还增加了一项请求过滤控制,作为深入防卫的措施。 该构成部分一直服务,没有停工时间。 调查 • 我们在2026年10月20日,即公开披露的前一天,通过2026年10月24日的补救,审查了日志。 • 我们没有发现任何剥削证据,也没有发现任何客户数据被取用或披露的证据。 无需采取行动 • 不需要旋转密码、API密钥、SSO设置或集成证书。 • 如果有任何变化,我们将继续监测并更新这一页。 • 问题、佐证证据或正式证明,电子邮件:[email protected].

自动翻译自官方事件更新。

1. 2026年9月21日 UTC 13:13investigating

   目前用户无法登录到Mindtickle平台. 已经登录的用户没有受到影响,可以继续使用该平台。 我们的小组正在积极调查这个问题.

2. 2026年9月21日 UTC 13:53identified

   我们已查明登入问题的根源,目前正在研究补丁.

3. 2026年9月21日 UTC 15:10monitoring

   有些用户现在应该可以登录到Mindtickle平台。 我们正在继续监测复苏情况,并通过其余受影响的用户开展工作.

4. 2026年9月21日 UTC 15:46monitoring

   更新:现已恢复大多数用户的登录功能. 我们将继续密切监测该平台,以确保全面恢复和稳定.

5. 2026年9月21日 UTC 16:42resolved

   影响平台的登录问题已经解决,用户现在可以正常签名. 中断时间从04:52AM PT持续到9:42AM PT. 没有客户数据受到影响。 已经登录到Mindtickle的用户可以毫无顾虑地进入平台. 我们对此造成的破坏感到抱歉。 如果你仍然无法登录,请联系我们@[email protected]. 我们将完成一次内部尸检,并在接下来的几天里在这里公布RCA.

自动翻译自官方事件更新。

1. 2026年9月16日 UTC 13:33identified

   我们知道一个影响销售力量相关工作流程的问题,这是销售力量方面持续停工造成的。 这不是Mindtickle内部问题的结果。 影响是什么: 使用销售力量集成的客户可能会看到以下的故障或延误: 1. 销售力量整合(SFDC):客户关系管理数据同步、用户提供/同步、模块/资产建议、学习者业绩同步到销售力量 2. 数字销售室:同步机会、联系人、自动创建销售室机会以及销售室账户,并将房间活动写回销售室。 DSR的核心功能(创建和共享房间,内容,分析)仍然可用. 3. 致电AI:将录音映射到销售力量记录,并在会议可用后显示销售力量机会/账户栏目,将调用见解推回销售力量,以及销售力量迪亚勒呼叫捕捉. 核心调用人工智能功能(记录、抄录、洞察)仍然可用。 不受影响者:非销售力量工作流程,核心产品功能,以及不使用销售力量集成的客户均正常运行. 我们正在积极监测销售力量的停产情况并评估其影响.

2. 2026年9月16日 UTC 17:01resolved

   已解决了销售力量方面的停用问题,目前所有受影响的工作流程都正常运行.

自动翻译自官方事件更新。

1. 2026年8月31日 UTC 11:08investigating

   由于Azure数据中心的下游断电支持这一使用案例,Seller Copilot出现了高误差率. 另外,Mindtickle聊天支持在生成对您的询问的回复时可能遇到故障.

2. 2026年8月31日 UTC 11:40identified

   这一问题已经确定,一个解决办法正在实施之中.

3. 2026年8月31日 UTC 12:10monitoring

   一项固定措施已经执行,我们正在监测局势.

4. 2026年8月31日 UTC 13:06resolved

   该事件已得到解决,所有系统现已投入使用.

自动翻译自官方事件更新。

1. 2026年8月19日 UTC 17:16investigating

   我们正在调查一个问题,一个子集的客户可能在某些分析报告中看到被扭曲的数据,具体来说是:SSR,自定义的Dashboards,经理和关系报告,以及Overview/通过/绩效报告. 其他报告和产品领域没有受到影响。 如果您的分析反映了当前数据,则不需要采取任何行动。 我们马上发布最新消息.

2. 2026年8月20日 UTC 02:07resolved

   这个问题已经得到解决。 所有受影响的报告现在都显示最新数据。 谢谢你的耐心.

自动翻译自官方事件更新。

1. 2026年6月13日 UTC 00:35investigating

   We are currently experiencing a partial service disruption on Mindtickle platform. Users may encounter errors, intermittent failures, or degraded performance across the platform. Our engineering team is actively investigating and mitigating the issue while closely monitoring recovery efforts.

2. 2026年6月13日 UTC 00:51identified

   We have identified the cause of the elevated error rates and are deploying mitigation measures. We are monitoring the platform closely and working to restore normal service.

3. 2026年6月13日 UTC 01:01monitoring

   Mitigation has been deployed and services are operating normally. We are continuing to monitor the platform closely to ensure stability.

4. 2026年6月15日 UTC 03:36resolved

   The incident has been resolved and services are operating normally. We will continue to monitor the platform to ensure stability. A detailed Root Cause Analysis will be shared shortly.

5. 2026年6月18日 UTC 12:24postmortem

   # **Incident Summary** On June 12, 2026, Mindtickle experienced a service disruption caused by an unexpected surge in processing activity within one of our backend workflows. The resulting increase in downstream traffic exceeded the capacity of a critical platform dependency, leading to service degradation and eventual unavailability. The issue was mitigated by reducing processing throughput and allowing affected systems to recover. **Impact Duration:** **3 hours and 15 minutes**. ## **Impact** During the incident window, customers may have experienced: * Delays in invitation-related workflows * Elevated latency and intermittent errors across affected services * Temporary degradation of platform functionality dependent on the impacted backend systems ## **Incident Timeline** * ‌**Jun 12, 03:00 PM**: A critical backend dependency becomes saturated, resulting in service degradation and elevated error rates. * **Jun 12, 03:15 PM**: The issue is detected through automated monitoring and customer reports. Investigation begins. * **Jun 12, 04:50 PM**: Mitigation measures are implemented to reduce processing throughput and stabilize affected systems. * **Jun 12, 05:46 PM**: Platform functionality is restored fully ## **Root Cause** A backend workflow generated a significantly higher volume of processing activity than historical norms. As the system worked through the resulting backlog, requests propagated through multiple downstream services, creating sustained load across the platform. While each component behaved as designed, the combined volume exposed capacity and protection gaps within the processing chain. The resulting amplification effect overwhelmed a critical backend dependency, causing service degradation and eventual unavailability. ## **Resolution** The team mitigated the incident by reducing the rate at which backlog processing occurred, allowing downstream systems to stabilize and recover. Once traffic levels returned to normal operating ranges, affected services resumed normal operation and platform functionality was fully restored. ## **Corrective and Preventive Actions** * Introduce rate limiting and additional safeguards for high-volume processing workflows to prevent a single workflow from generating excessive downstream load. * Implement backpressure, circuit-breaking, and load-shedding mechanisms across critical processing paths to better protect downstream dependencies during traffic spikes. * Review platform capacity limits, concurrency settings, and monitoring to improve resilience and provide earlier visibility into abnormal processing patterns.

1. 2026年6月3日 UTC 17:06resolved

   On June 2nd, between 7:19 AM PDT and 9:21 AM PDT, our third-party recording provider experienced a disruption that prevented the Call AI bot from joining meetings. A limited number of customers were impacted, and unfortunately, the affected recordings are non-recoverable. We sincerely apologize for the disruption and any impact to your workflows. The issue is fully resolved, and recordings are functioning normally. If your account was affected, your account team will reach out with specific details.

1. 2026年5月21日 UTC 15:29investigating

   Users may experience issues starting Two-Way Roleplay sessions. In addition admins may be unable to preview roleplays. The team is actively investigating the issue and working toward resolution.

2. 2026年5月21日 UTC 15:59identified

   We have identified the root cause as a service disruption at a third-party provider that powers parts of the Two-Way Roleplay workflow. We are actively working with the vendor to restore the service and recover impacted functionality as quickly as possible.

3. 2026年5月21日 UTC 17:07identified

   Recovery efforts are still ongoing, and we continue to work closely with our third-party partners to restore the impacted service. Our teams remain actively engaged on the issue.

4. 2026年5月21日 UTC 17:18identified

   We are seeing partial recovery in the impacted service, and some roleplay attempts are now completing successfully. We continue to work with the third-party provider toward full restoration.

5. 2026年5月21日 UTC 17:49monitoring

   The impacted systems are now operational again, and roleplay workflows are functioning successfully. We are continuing to monitor the systems closely to ensure stability.

6. 2026年5月21日 UTC 18:30resolved

   All impacted systems are now operating normally, and Two-Way Roleplay functionality has been fully restored. We will continue to monitor the systems closely. The incident is now resolved.

1. 2026年5月18日 UTC 13:17investigating

   The 'Featured Hubs' widget, wherever used on the Mindtickle platform, is not loading. The Hubs and individual media/assets are working properly. We are currently investigating this. All other capabilities of the Mindtickle platform are operating normally.

2. 2026年5月18日 UTC 13:21identified

   The issue has been identified and a fix is being implemented.

3. 2026年5月18日 UTC 13:31monitoring

   A fix has been implemented and we are monitoring the results.

4. 2026年5月18日 UTC 13:55resolved

   This incident has been resolved. The 'Featured Hubs' widget is now loading as expected, and the Mindtickle platform is fully operational.

1. 2026年5月11日 UTC 14:25investigating

   We are currently investigating an issue impacting call recordings in Call AI. Users may experience failures while recording calls. The team is actively working on identifying the root cause and restoring normal functionality. Further updates will be shared shortly.

2. 2026年5月11日 UTC 16:35resolved

   The issue has now been identified and resolved, and Call AI services are operating normally again. We will share a detailed Root Cause Analysis (RCA) shortly.

1. 2026年3月26日 UTC 11:50resolved

   Between 07:50 AM and 11:25 AM PT on March 24, 2026, the Mindtickle platform experienced a service disruption affecting Call Recordings. Full service was restored by 11:25 AM PT. We are currently conducting a thorough investigation and will publish a detailed Root Cause Analysis (RCA) shortly.

2. 2026年3月27日 UTC 03:50postmortem

   We experienced a service disruption impacting **Call AI bot functionality**. During this time, **select scheduled meetings were affected**, where Call AI bots were unable to join and record. The issue has been fully resolved, and services have returned to normal operation. ‌ **Impact** * Call AI bots were unable to join **select scheduled meetings** * Call recordings and transcriptions were not generated for impacted meetings ‌ **Timeline** * **07:50 AM PT:** Incident began; bots unable to join select meetings * **07:52 AM PT:** Alerts triggered; issue detected and investigation initiated * **09:00 AM PT:** Root cause identified within Call AI bot infrastructure * **10:15 AM PT:** Mitigation measures applied; service recovery in progress * **11:25 AM PT:** Service fully restored; bots resumed normal operation ‌ **Root Cause** This incident was caused by a **transient issue within the Call AI bot infrastructure**, which impacted service availability during the affected period. ‌ **Next Steps** * Exploring & implement enhanced failover mechanisms to improve system resilience * Introducing more granular monitoring and alerting for faster detection ‌ We apologize for the inconvenience caused and appreciate your patience.

1. 2026年2月19日 UTC 14:33investigating

   we are currently investigating an issue affecting the Analytics Platform. While out of the box dashboards remain functional, users may be unable to open existing custom reports or create new ones. Our engineering team is identifying the root cause, and we will provide an update as soon as more information is available. We apologize for the interruption to your workflow.

2. 2026年2月19日 UTC 14:50investigating

   We are continuing to investigate this issue.

3. 2026年2月19日 UTC 14:52resolved

   We have successfully resolved the issue affecting the Analytics Platform. All custom reports should now load correctly, and the ability to create new reports has been fully restored. Our team has verified the fix, and we are monitoring the system to ensure continued stability. Thank you for your patience while we worked to get this back online.

1. 2026年2月12日 UTC 03:29investigating

   We are observing high error rates for both admins and end users for select customer instances. We are investigating the issue and will share an update shortly.

2. 2026年2月12日 UTC 03:29identified

   The issue has been identified and a fix is being implemented.

3. 2026年2月12日 UTC 03:36monitoring

   We have implemented a fix and are monitoring the progress. We are seeing the error rates coming down.

4. 2026年2月12日 UTC 03:37resolved

   The incident has been resolved. All admins and end users should be able to access the platform without any concerns.

1. 2026年2月6日 UTC 13:23resolved

   [Resolved] A subset of customers (admins only) experienced intermittent failures when admins attempted to upload content to the platform. End users were not impacted and were able to view existing content without issues. The issue has been resolved, and all admin content upload workflows are operating as expected. Incident Timeline Start Time: 05 Feb 2026, 11:15 PM PT Resolved Time: 06 Feb 2026, 05:12 AM PT We will share a detailed RCA once the post-incident review is complete.

1. 2025年11月18日 UTC 14:45identified

   The Two-Way Roleplay Missions feature is currently experiencing an outage and is inaccessible. This is a direct consequence of the ongoing, widespread Cloudflare network outage. We are closely monitoring Cloudflare's recovery efforts and expect the service to be restored as soon as their network disruption is resolved.

2. 2025年11月18日 UTC 14:54monitoring

   Cloudflare has confirmed the deployment of a fix for the underlying network issue. We are now observing a significant decrease in error rates on our platform, with Two-Way Roleplay Missions starting to become accessible again. We will continue to monitor the platform closely as services stabilize, and will post a final resolution notice once all operations are confirmed to be fully back to normal.

3. 2025年11月18日 UTC 15:37monitoring

   Two-Way Roleplay Missions are operational. We will continue to monitor the platform closely to ensure stability before marking this incident as fully resolved.

4. 2025年11月18日 UTC 17:17resolved

   All services are normal. A detailed Root Cause Analysis (RCA) will be published shortly.

1. 2025年10月20日 UTC 16:30identified

   There has been a second surge of error rates and latencies reported due to an AWS outage in the US-East-1 region. The AWS team is actively working on this and we are monitoring the status closely. Though the platform is up and some features are working, select customers and users may face intermittent failues in accessing the platform. We will keep updating the status page as soon as we receive updates from AWS. We apologize for the inconvenience caused here. AWS Outage Status: https://health.aws.amazon.com/health/status

2. 2025年10月20日 UTC 17:51identified

   As per AWS, their internal subsystems are now showing early signs of recovering in a few Availability Zones (AZs) in the US-EAST-1 Region. They are now applying mitigations to the remaining AZs at which point we expect launch errors and network connectivity issues to subside. We will share an update once the AWS team has applied the mitigations and we have verified at our end.

3. 2025年10月20日 UTC 19:43monitoring

   Post the recovery and fix from the AWS team, we are observing a significant improvement in the Mindtickle platform performance. Most Mindtickle workflows are now functioning normally. Our teams continue to monitor the situation closely and will provide an update once the issue is fully resolved. AWS Outage Status: https://health.aws.amazon.com/health/status

4. 2025年10月20日 UTC 20:55resolved

   The incident has been resolved, and the Mindtickle platform is now operating normally. We continue to closely monitor performance to ensure everything remains stable.

1. 2025年10月20日 UTC 10:07monitoring

   Earlier today, some Mindtickle customers experienced high error rates and intermittent service disruptions due to an AWS outage in the US-East-1 region. AWS has since deployed a fix, and we’re observing steady recovery across our systems. Error rates are declining, and most services are returning to normal. Our teams continue to monitor the platform closely to ensure full stability. We’ll share a final update once all systems are fully restored. Thank you for your patience and understanding.

2. 2025年10月20日 UTC 11:03resolved

   The incident has been resolved, and the Mindtickle platform is operating normally.

1. 2025年10月16日 UTC 03:54investigating

   We’ve identified an issue where specific pages within the Analytics section are displaying stale or incorrect data for select customers. Our team is actively investigating the issue and will share further updates as soon as possible. Impacted Areas: 1. OData Routes (Partial Routes) – Stale data visible 2. Analytics Platform Pages (Series, Modules, Learner & Learner Groups) – Incorrect data displayed 3. Custom Reports shared with customers via Email (in the last 12 hours) – Incorrect data displayed

2. 2025年10月16日 UTC 04:29monitoring

   A fix has been implemented and we are running a sanity check at our end.

3. 2025年10月16日 UTC 04:50resolved

   This incident has been resolved. We will share an RCA once the postmortem is complete.

1. 2025年10月15日 UTC 11:19resolved

   Between October 14, 2025, 12:10 AM PT and October 15, 2025, 3:00 AM PT, select Call AI customers experienced a delay in the processing of call recordings. Impact: 1. Call recordings generated during this period were temporarily not visible on the Call Listing page. What Continued to Work: 1. All recordings captured before the incident window remained accessible on the Call Listing page. 2. For calls recorded during the incident, Recording Ready emails were successfully sent to end users, allowing access to the recordings via the email links. We are conducting a post-incident analysis to identify the root cause. A detailed Root Cause Analysis (RCA) report will be shared once the investigation is complete.

1. 2025年9月24日 UTC 09:01investigating

   We're currently experiencing an high error rate in our in-platform analytics and are actively investigating the root cause to restore full functionality.

2. 2025年9月24日 UTC 09:40identified

   The issue has been identified and a fix is being implemented.

3. 2025年9月24日 UTC 09:51monitoring

   A fix has been implemented and we are monitoring the results.

4. 2025年9月24日 UTC 10:09resolved

   A fix has been applied, and the incident has been resolved. We are continuing to monitor this closely.