52 Highspot incidents · 2022年10月 — official updates, affected components, duration and resolution details.
我们目前正在调查美国第二数据中心的业绩和延迟问题,包括504个错误。 由3PM PT再次更新.
工程组解决了美国S2数据中心的性能问题. 所有服务都全面投入运作.
自动翻译自官方事件更新。
We are currently investigating reports of performance and latency issues, including 504 errors, in the AMERICAS2 Data Center. We will provide another update by 1PM PT.
The Engineering Team has resolved the performance issues in the AMERICAS2 Data Center. All services are fully operational.
我们正在调查关于一个管理人员无法审查我们美洲统计1和美国统计2数据中心学习中心所提交材料的问题的报告。 我们正在积极调查这一问题,并将在太平洋时间上午11时30分提供有关症状和范围的更详细情况.
我们正在继续调查这一问题.
工程组已查明这一问题的根源,并正努力解决这个问题.
我们正在继续努力解决这个问题。
与管理人员有关的问题目前无法在学习中心内审查提交材料.
自动翻译自官方事件更新。
We are currently investigating reports of performance and latency issues, including 504 errors, in the AMERICAS2 Data Center. We will provide another update by 11AM PT
The Engineering Team has resolved the performance issues in the AMERICAS2 Data Center. Everything is fully operational again.
We are currently investigating reports of performance and latency issues in the AMERICAS2 Data Center. We will provide another update by 10AM PT.
The Engineering Team has resolved the performance issues in the AMERICAS2 Data Center. Everything is fully operational again.
We are investigating an issue where Pitches and Digital Rooms are failing to send across all data centers. Our Engineering Team is actively working on a resolution. We will provide another update at 10AM PT.
We have deployed a fix for this issue - all Digital Rooms and Pitches are now sending without issue. This is fully resolved.
We have received reports that a few users are experiencing server dizzy errors and intermittent performance degradation within Highspot. Our Engineering Team is investigating with the highest priority to identify and resolve the issue. We will provide another update by 12 PM PT.
We have remediated the issue and are currently monitoring system performance. Our Engineering Team will continue to observe stability and provide another update by 2 PM PT.
This incident has been resolved. System performance has returned to fully operational.
We are investigating reports regarding an issue the is impacting the Highspot Platform. This is impacting all our Scale Units and multiple features on the platform. We are actively investigating this issue and will provide further details regarding symptoms and scope at 05:00 Pacific Time.
The issue that impacted multiple features on all Data Centers has now been resolved. All customers with impacted customer have been notified.
We are currently experiencing performance degradation due to recurrent AWS service disruptions. Customers may notice varying levels of delays in item processing, CRM data synchronization, and overall platform latency. Our team is monitoring the situation closely and will provide the next update by 12 PM PT.
We are continuing to investigate this issue.
The recurrent AWS service disruptions are still causing intermittent performance degradation, item processing delays, and CRM data sync issues. We are continuing to investigate this issue as a top priority. We will provide another update by 2PM PT.
The ongoing AWS service disruptions continue to cause intermittent issues. Symptoms include performance degradation, item processing delays, CRM data sync issues, and gateway errors. We are continuing to investigate this with urgency. We will provide another update by 2PM PT.
We are continuing to experience intermittent service degradation due to the ongoing AWS outage. Customers may encounter intermittent login errors, item processing delays, CRM data sync issues, and general performance impacts across the platform. Our Engineering team continues to investigate this as the top priority. We will provide another update by 4PM PT.
We are continuing to work through service degradation issues resulting from the earlier AWS outage. Customers may experience intermittent login errors, item processing delays, CRM data sync issues, and general performance impacts across the platform. Our team continues to treat this as our highest priority and is actively working toward full recovery. We will provide another update by 6PM PT.
We continue to address ongoing service degradation following the earlier AWS outage. While AWS has reported recovery on their end, residual impacts are still affecting parts of the Highspot platform. Customers may continue to experience intermittent login errors, CRM data sync delays, item processing issues, and general performance slowdowns. Our team remains fully engaged and is treating this as a top priority. We’ll share the next update by 9PM PT.
We’re continuing to investigate and address ongoing service degradation following the earlier AWS outage. Our team is implementing changes intended to improve stability and performance. AMERICAS customers may continue to experience intermittent login errors, CRM data sync delays, item processing issues, and general performance impacts while this work is in progress. We’ll provide another update by 11 PM PST NOTE: EMEA & APAC services are fully functional.
We have made changes to alleviate an issue previously reported due to AWS service disruptions for AMERICAS. Our Engineers continue to work on this issue with high priority. All customers should be able to use Highspot. Error rates are decreasing but non-zero. However, if you experience errors with, please contact Highspot Support ([email protected]) for assistance. We will provide an update at 5 AM PST, Oct 21. NOTE: EMEA & APAC services are fully functional.
We have made changes to alleviate the issue previously reported due to AWS service disruptions for AMERICAS. Our Engineers continue to monitor and no new issues have been reported. However, if you experience errors, please contact Highspot Support ([email protected]) for assistance. We will provide an update at 10 AM PST, Oct 21. NOTE: EMEA & APAC services are fully functional.
We’ve completed recovery efforts related to the earlier AWS outage, and the Highspot platform is now fully operational. End users can access Highspot without issue, and all services are performing as expected. Our team will continue to monitor system stability closely, but no further impact is anticipated. Thank you for your patience while we worked through this incident.
We are investigating reports regarding an issue related to Items not processing correctly on all our Data Centers. We are actively investigating this issue and will provide further details regarding symptoms and scope at 08:00 Time.
We are continuing to investigate this issue.
We have identified an issue related to Item Processing on all our Data Centers. Our Engineers are actively working on this issue with high priority. Symptoms for this issue be be items are stuck in a processing state. We will provide an update at 09:00 Pacific Time.
The issue related to Item Processing on our Data Centers has now been resolved. All items that had been 'stuck' in processing will now show as completed.
We are investigating reports regarding an issue related to Item Processing, which is impacting all our Scale Units We are actively investigating this issue and will provide further details regarding symptoms and scope at 08:00 Pacific Time.
We have identified the issue related to Item Processing affecting our Data Centers. Our Engineers are actively working on this issue with high priority. Symptoms for this issue may include files failing to process correctly with uploaded to Highspot. We will provide an update at 09:00 Pacific Time.
We are investigating reports regarding an issue related to Item Processing, which is impacting all our Data Centers. Currently, only Americas1 is still impacted. We are actively investigating this issue and will provide further details regarding symptoms and scope at 10:00 Pacific Time.
We are investigating reports regarding an issue related to Item Processing, which is impacting all our Data Centers. Currently, only Americas1 is still impacted. We are actively investigating this issue and will provide further details regarding symptoms and scope at 11:00 Pacific Time.
We have identified the issue impacting Item Processing in all Data Centers. As of right now, we have fixed the Item Processing issues taking place in the AMERICAS2, EMEA1, EMEA2, and APAC1 Data Centers. We are currently resolving the Item Processing issues taking place in the AMERICAS1 Data Center. We will provide another update at 12PM PT.
We have deployed a fix for the AMERICAS1 Data Center & are currently reprocessing all content in said Data Center. We expect the reprocess to be done in the next 2-3 hours. We will provide another update by 3PM PT.
We continue to reprocess the AMERICAS1 Data Center, which is the only remaining Data Center affected. The only outstanding impact is that PowerPoint animations may not function correctly for users in this Data Center. We will provide another update by 5:00 PM PT.
We continue to reprocess the AMERICAS1 Data Center. We expect this to be fully resolved within the next 1 hour. We will provide another update by 7:00 PM PT.
This incident has been resolved, all reprocessing in the AMERICAS1 Data Center has caught up. End-users will no longer experience any issues when viewing items within Highspot.
We are currently investigating reports of degraded performance in Highspot where Server Dizzy Errors are being returned when using Search. We are investigating this with the highest priority possible. We will provide another update at 4PM PT.
We have deployed a fix for the degraded performance issue impacting Search in Highspot. This is now fully resolved.
We are currently investigating an issue where users are intermittently returned an error when accessing the Engagement Center. We will provide another update at 10AM PT.
We have deployed a fix for the degraded performance issue impacting the Engagement Center. This is now fully resolved.
We are currently investigating reports of degraded performance in Highspot, including slow load times, latency, and dizzy errors. We are actively investigating this issue and will provide another update at 4PM PT.
The Engineering Team identified and resolved a DNS issue. All Highspot systems are back to normal.
We are currently investigating reports of degraded performance in the AMERICAS1 & AMERICA2 Data Centers, including slow load times & server errors. We are actively investigating this issue and will provide another update at 12PM PT.
We are continuing to investigate the performance degradation issues occurring in the AMERICAS1 & AMERICA2 Data Centers. We will provide another update at 1PM PT.
We continue to investigate the reports of degraded performance in the AMERICAS1 & AMERICAS2 Data Centers. The Engineering Team is actively working on a fix at the moment. We will provide another update at 3PM PT.
The Engineering Team has deployed a fix for the performance issues in the AMERICAS1 & AMERICAS2 Data Centers. Performance in the AMERICAS1 & AMERICAS2 Data Centers is back to normal.
We are currently investigating intermittent reports of login issues across multiple Data Centers. The Engineering Team is currently investigating with the highest priority possible. We will provide another update at 6PM PT.
Our Engineering team continues to investigate intermittent login issues. We have identified the root of the issue & are actively working on a fix. We will provide another update at 8PM PT.
The Engineering Team deployed a fix for the intermittent login issues. This issue is now fully resolved.
We are investigating reports regarding an issue related to Training and Coaching on all our Data Centers. We are actively investigating this issue and will provide further details regarding symptoms and scope at 09:00 Pacific Time.
We are actively investigating this issue and will provide further details regarding symptoms and scope at 10:00 Pacific Time.
We have identified the issue related to Training and Coaching in our Data Centers. Our Engineers are actively working on this issue with high priority. Symptoms for this issue may include videos views not being tracked & course submissions not registering. We will provide an update at 11:00 Pacific Time.
The issue related to Training and Coaching has now been resolved.
We are currently investigating the Highspot Marketplace being down. Our Engineering Team is investigating this with the highest priority at the moment. We will provide another update by 3PM PT.
The Highspot Marketplace is back up & available.
Analytics Data in the AMERICAS2 Data Center is currently delayed. We expect the data to be caught up in 24 hours. We will provide another update by 4PM PT.
We continue to work through the data delay, with the data gradually catching up. Our current target is to have the data fully caught up by 11 PM PT. We will provide another update by 8 PM PT.
The data delay is continuing to progress, and we expect it to be fully caught up by the end of the night. We will provide another update once the data has completely caught up.
The analytics data delay in AMERICAS2 Data Center has been fully resolved, and all data is now up to date
On Tuesday, October 15th we experienced an issue with additional notification emails being sent in error. We've identified and mitigated the core problem, and have disabled some notification emails as part of the investigation. We apologize for any inconvenience this may have caused.
A hotfix has been deployed to address the notification error event, and we are continuing to monitor the situation and will provide an update as available. You may begin to receive email notifications as the system sends out emails from the queue.
This incident has been resolved.
