История сбоев Incident Iq

1. 27 октября 2026 г. в 13:33 UTCmonitoring

   Мы знаем, что некоторые районы имеют проблемы с доступом к платформе IQ. Нет проблем с платформой Incident IQ, а скорее с более крупной интернет-инфраструктурой, где некоторые брандмауэры 3-й стороны блокируют интернет-трафик, проходящий через Cloudflare. Об этом сообщает Cloudflare и активно работает над восстановлением. https://www.cloudflarestatus.com/incidents/dk2obcyw2aew

2. 27 октября 2026 г. в 15:22 UTCresolved

   Cloudflare устранила инцидент со сторонней сетью, влияющий на доступ к району. Все службы работают в штатном режиме, и теперь районы должны иметь доступ к IQ без проблем. Подробнее об инциденте см. в https://www.cloudflarestatus.com/incidents/dk2obcyw2aew.

Автоматический перевод официального обновления инцидента.

1. 11 августа 2026 г. в 18:09 UTCinvestigating

   We want to inform you that we are currently experiencing a performance issue that may impact your experience on our platform. Our engineering team is actively investigating the issue, and we are working to resolve it as quickly as possible. We apologize for any inconvenience this may cause and appreciate your patience. We will keep you updated on our progress and notify you as soon as the platform is performing at optimal levels.

2. 11 августа 2026 г. в 18:15 UTCmonitoring

   Our engineering team has identified the root cause and applied a fix. We are actively monitoring the results, and services are recovering.

3. 11 августа 2026 г. в 19:03 UTCresolved

   Our internal monitoring and testing confirm that the platform and all services are fully restored and operating normally. We apologize for any disruption this issue may have caused and appreciate your patience while our team worked on a fix. If you continue to experience any issues, please don't hesitate to reach out to our support team.

1. 4 августа 2026 г. в 15:39 UTCinvestigating

   We are experiencing intermittent site responsiveness issues, causing slow page loads and feature performance. Our engineering team is currently investigating and working on a solution. We will provide updates here as soon as they become available. We apologize for the inconvenience and appreciate your patience.

2. 4 августа 2026 г. в 16:15 UTCmonitoring

   Our engineering team has identified the root cause and applied a fix. We are actively monitoring the results and expect services to recover over the next 30 minutes.

3. 4 августа 2026 г. в 16:45 UTCresolved

   Our internal monitoring and testing confirm that the platform and all services are fully restored and operating normally. We apologize for any disruption this issue may have caused and appreciate your patience while our team worked on a fix. If you continue to experience any issues, please don't hesitate to reach out to our support team.

4. 6 августа 2026 г. в 19:50 UTCpostmortem

   On Tuesday morning, 08/04/2026, a performance update caused temporary and intermittent slowness across parts of the platform, particularly during batch check-in and check-out. Our team isolated the cause, reversed the update, and fully restored normal performance by early afternoon. No data was impacted. We have paused these types of updates during the busy back to school season and are updating our internal monitoring to ensure faster detection moving forward.

1. 8 июня 2026 г. в 17:53 UTCinvestigating

   We are currently experiencing an outage from our phone vendor. To contact Support please put in a ticket via the in application messenger, or email us at [email protected]. We will provide an update here as soon as the phones are working again.

2. 8 июня 2026 г. в 18:30 UTCresolved

   Resolved: Phone Services Restored The issues affecting our phone systems have been fully resolved, and all lines are back up and running. We apologize for the inconvenience and appreciate your patience while we worked to restore functionality.

1. 11 мая 2026 г. в 12:33 UTCidentified

   We are investigating reports of an issue affecting the use of certain characters in the ticket comments fields. Workaround: You can type up notes in another field or text editor, such as Word or Notepad, and copy the text into the comments field. The rest of the platform remains operational. We are working to identify the problem and will update you soon.

2. 11 мая 2026 г. в 13:51 UTCmonitoring

   We are pleased to share that a resolution has been implemented, and our team is currently monitoring the system to ensure continued stability. If you are still seeing any irregularities, please refresh your session by closing and restarting your browser.

3. 11 мая 2026 г. в 15:03 UTCresolved

   The underlying root cause has been addressed and our monitoring confirms that all systems are functioning as expected. If you continue to see any local cached issues, we recommend a final browser restart.

1. 30 октября 2025 г. в 17:46 UTCinvestigating

   We are aware of an issue affecting the loading of tickets. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience as we work to resolve this issue.

2. 30 октября 2025 г. в 18:21 UTCmonitoring

   We are pleased to confirm that the issue has been resolved, and full functionality is restored. Our team is actively monitoring the system to ensure continued stability. Should you experience any further issues, please contact our support team immediately. We apologize for any inconvenience this caused and thank you for your patience.

3. 30 октября 2025 г. в 21:22 UTCresolved

   We are pleased to report that the service issue reported earlier today has been resolved. Following a thorough review, our internal monitoring and testing have confirmed that all systems are fully operational. At approximately 1:30 PM ET, our support team was made aware of an issue preventing certain pages from loading within the platform. Our Engineering team, which was already investigating, confirmed that full system functionality was restored at 2:10 PM ET. We sincerely apologize for the disruption and appreciate your patience as we verified system health. If you are still experiencing any issues, please contact our support team directly for further investigation.

4. 12 ноября 2025 г. в 23:22 UTCpostmortem

   At separate times on October 27th and 30th, IncidentIQ experienced brief periods of degraded platform performance. During these windows, users experienced slow page loads, timeouts, and overall sluggish response times across several areas of the application. The issues were remediated by adjusting, but not fundamentally changing, cloud database configurations. Initial investigation by the engineering team identified issues with a patch deployed by one of our upstream cloud vendors. While working with the vendor to confirm the cause and roll back the change, iiQ engineers increased capacity and deployed additional resources to ensure performance was not impacted between October 30th and November 7th. On November 7, 2025 the provider’s engineers rolled back the patch overnight. No similar incidences of these symptoms have occurred since the rollback, and we expect the rollback will fully resolve this class of issues. We know that platform reliability is critical for your team's operations. Thank you for your patience as we resolved this with the vendor.

1. 27 октября 2025 г. в 18:36 UTCinvestigating

   We are aware of intermittent issues affecting page loads and login issues. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience as we work to resolve this issue.

2. 27 октября 2025 г. в 19:07 UTCmonitoring

   We are pleased to report that all systems are stable and operating normally. Our team will continue to closely monitor performance to ensure continued stability. Thank you for your patience.

3. 27 октября 2025 г. в 21:09 UTCresolved

   Following a review, our internal monitoring and testing show that all systems are fully operational. We appreciate your patience as we verified system health. If you are still experiencing any issues, please contact our support team directly so we can investigate further.

4. 12 ноября 2025 г. в 23:22 UTCpostmortem

   At separate times on October 27th and 30th, IncidentIQ experienced brief periods of degraded platform performance. During these windows, users experienced slow page loads, timeouts, and overall sluggish response times across several areas of the application. The issues were remediated by adjusting, but not fundamentally changing, cloud database configurations. Initial investigation by the engineering team identified issues with a patch deployed by one of our upstream cloud vendors. While working with the vendor to confirm the cause and roll back the change, iiQ engineers increased capacity and deployed additional resources to ensure performance was not impacted between October 30th and November 7th. On November 7, 2025 the provider’s engineers rolled back the patch overnight. No similar incidences of these symptoms have occurred since the rollback, and we expect the rollback will fully resolve this class of issues.

1. 2 сентября 2025 г. в 13:27 UTCinvestigating

   We are experiencing some intermittent site responsiveness issues that are causing slow page loading times. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience.

2. 2 сентября 2025 г. в 13:35 UTCmonitoring

   A fix has been implemented and we are monitoring the results.

3. 2 сентября 2025 г. в 14:10 UTCresolved

   Apologies for the inconvenience and thank you for your patience This incident has been resolved. If you experience any additional problems please reach out to Support.

1. 24 июля 2025 г. в 14:14 UTCinvestigating

   We are aware of an intermittent issue affecting the loading of tickets and assets views. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience as we work to resolve this issue.

2. 24 июля 2025 г. в 15:36 UTCmonitoring

   We have deployed a fix to address the intermittent issue with tickets and assets views loading and are currently monitoring the situation. We will provide additional updates as they become available. In the meantime, please contact our support team if you have any additional questions or concerns during this time.

3. 24 июля 2025 г. в 21:11 UTCresolved

   Our internal monitoring and testing indicates that the platform and all services are functioning as expected. If you have any additional questions or concerns, please reach out to support. We apologize for any disruption this may have caused, and thank you for your understanding during this time.

1. 11 июня 2025 г. в 18:01 UTCinvestigating

   We are aware of an intermittent issue affecting the loading of assets on certain pages. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience as we work to resolve this issue.

2. 11 июня 2025 г. в 18:35 UTCmonitoring

   We have deployed a fix to address the intermittent asset loading issue and are currently monitoring the situation. We will provide additional updates as they become available. In the meantime, please contact our support team if you have any additional questions or concerns during this time.

3. 11 июня 2025 г. в 20:33 UTCinvestigating

   We are aware of intermittent issues with check-in/out and verifications. Our Engineering Team is currently working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience as we work to resolve this issue.

4. 11 июня 2025 г. в 20:46 UTCmonitoring

   We have deployed a fix to address the issues with check-in/out and verifications and are currently monitoring the situation. We will provide additional updates as they become available. In the meantime, please contact our support team if you have any additional questions or concerns during this time.

5. 11 июня 2025 г. в 21:48 UTCresolved

   Our internal monitoring and testing indicates that the platform and all services are functioning as expected. If you have any additional questions or concerns, please reach out to support. We apologize for any disruption this may have caused, and thank you for your understanding during this time.

1. 17 мая 2025 г. в 16:30 UTCidentified

   We've identified intermittent issues with fee tracker app, as well as, displaying and filtering on certain elements in assets explorer. There is no loss or impact to the underlying data. Thanks for your patience as our Engineering team works to resolve the issue as soon as possible.

2. 17 мая 2025 г. в 23:28 UTCmonitoring

   A fix has been implemented and we are monitoring the results.

3. 18 мая 2025 г. в 01:12 UTCresolved

   This incident has been resolved.

1. 14 мая 2025 г. в 16:34 UTCidentified

   We are aware of an issue intermittently preventing users from accessing the Incident IQ site or seeing blank pages load. Our engineering team is aware of this and is working to restore full system functionality as soon as possible. We will provide further updates as they become available.

2. 14 мая 2025 г. в 16:43 UTCmonitoring

   We have deployed a fix to address the login and page-loading issues and are currently monitoring the situation. We will provide additional updates as they become available. In the meantime, please contact our support team if you have any additional questions or concerns during this time.

3. 14 мая 2025 г. в 17:30 UTCresolved

   This incident has been resolved. We deeply regret any inconvenience this may have caused and appreciate your understanding as we worked to resolve the issue. If you have any additional questions or concerns please contact our support team for further assistance.

1. 13 мая 2025 г. в 11:58 UTCinvestigating

   We are currently aware of an issue causing delays in job processing at the moment. Our team is working on identifying and resolving the underlying cause as soon as possible. We will continue to provide updates here as information becomes available.

2. 13 мая 2025 г. в 12:42 UTCmonitoring

   Our team has identified the underlying cause of this issue, and a fix has been deployed. The system is currently processing all delayed jobs, and we hope to have full functionality restored shortly.

3. 13 мая 2025 г. в 13:28 UTCresolved

   All delayed jobs have been processed, and the system is once again performing as expected. We appreciate your patience as our team worked to resolve this issue. If you have any additional questions or concerns, please contact our support team for further assistance.

1. 16 апреля 2025 г. в 20:20 UTCidentified

   Our phone provider is currently experiencing an outage affecting our ability to receive inbound calls. Please use the in-system support ticketing or email ([email protected]) to reach out for assistance. We will provide updates as functionality is restored, and we appreciate your understanding during this time.

2. 16 апреля 2025 г. в 20:41 UTCmonitoring

   Our phone support is back online. We will continue to monitor the situation for the time being. If you continue to experience any issues during this time, please reach out through the in-system support ticketing or email ([email protected]).

3. 16 апреля 2025 г. в 21:57 UTCresolved

   After monitoring the situation, everything appears stable and functioning as expected. If you have any additional questions or concerns, please reach out to support. We apologize for any disruption this may have caused, and thank you for your understanding during this time.

1. 13 января 2025 г. в 15:08 UTCinvestigating

   We are aware of an issue preventing users from accessing the Incident IQ site or causing blank loading pages. Our engineering team is aware of this and is working to restore system functionality as soon as possible. We will provide further updates as they become available.

2. 13 января 2025 г. в 18:17 UTCmonitoring

   We have deployed a fix to address the slowness and page-loading issues and are currently monitoring the situation. We will provide additional updates as they become available. In the meantime, please contact our support team if you have any additional questions or concerns during this time.

3. 14 января 2025 г. в 13:49 UTCresolved

   This incident has been resolved. Our Engineering team is completing an investigation into the cause and will provide an update here as soon as the postmortem is complete. We deeply regret any inconvenience this may have caused and appreciate your understanding as we worked to resolve the issue. If you have any additional questions or concerns please contact our support team for further assistance.

1. 2 декабря 2024 г. в 14:46 UTCinvestigating

   We are experiencing some intermittent site responsiveness issues that are causing slow page loading times and intermittent login problems. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience.

2. 2 декабря 2024 г. в 15:16 UTCmonitoring

   We have identified the issue and implemented a resolution. We are continuing to monitor the platform for stability and responsiveness. Additional updates regarding the cause of the issue and remediation steps will be posted here as soon as they are available.

3. 2 декабря 2024 г. в 15:55 UTCmonitoring

   During our continued platform monitoring, we have identified additional intermittent loading and responsiveness issues. Our Engineering team is focused 100% on returning the platform to optimal operational status. We will post updates here as they become available.

4. 2 декабря 2024 г. в 16:20 UTCidentified

   We are experiencing some intermittent site responsiveness issues that are causing slow page loading times and intermittent login problems. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience.

5. 2 декабря 2024 г. в 16:49 UTCmonitoring

   A fix has been implemented and we are monitoring the results.

6. 2 декабря 2024 г. в 22:09 UTCresolved

   This incident has been resolved and will be updated with a postmortem as soon as our investigation is complete. We understand how essential our platform is for your schools’ day-to-day activities, and we are committed to ensuring its reliability. We deeply regret any inconvenience this may have caused and appreciate your understanding as we worked to resolve the issue.

7. 9 декабря 2024 г. в 13:53 UTCpostmortem

   On Monday December 2, Incident IQ experienced a service interruption that caused degradation of services for several customers. ‌ **Timeline of events:** * 9:39 AM ET - Began receiving customer reports of slowness/degradation of services for portions of the platform to include login authentication. * 11:41 AM ET - All services and platform responsiveness levels restored **Cause of Incident:** * As part of normal operating procedures, Incident IQ scales platform capacity to meet the needs of our current customer and predictive demands. * Incident IQ uses the Microsoft Azure platform to  automatically scale infrastructure as part of normal operating procedures.  The process to scale encountered errors on Monday December 2nd and took five times longer than normal to complete. * The combination of infrastructure not scaling on normal schedules and the larger load than the available infrastructure caused the outage. **Remediation:** * Services were ultimately restored by a combination of manually adding infrastructure resources and the automated scaling job completion * Incident IQ is working alongside Microsoft and have identified that unforeseeable abnormally high transactions on the SQL environment during scaling caused the process to run longer. * We are continuing to work with Microsoft to prevent a recurrence of this issue. The reliability of our platform remains of the utmost importance to us. We understand the impact these moments have on our customers. The remediations put in place to prevent a recurrence of this particular incident, as well as the processes we have in place to continuously improve the platform, provide us with a level of certainty that we are able to stay ahead of unexpected surges in traffic. As before, we do sincerely apologize for this disruption and want to thank you for your patience and partnership as we worked through this issue.

1. 12 ноября 2024 г. в 16:26 UTCinvestigating

   We want to inform you that we are currently experiencing a performance issue that may impact your experience on our platform. You may notice delays in loading ticket pages, performing searches, taking actions, or creating new tickets. Our engineering team is actively investigating the issue, and we are working to resolve it as quickly as possible. We apologize for any inconvenience this may cause and appreciate your patience. We will keep you updated on our progress and notify you as soon as the platform is performing at optimal levels. Thank you for your understanding and partnership.

2. 12 ноября 2024 г. в 17:13 UTCmonitoring

   The engineering team has reported that the platform is now stable, and they are actively investigating the root cause of the event to prevent any recurrence. I’ll keep you posted on any further findings or actions. Thank you for your patience and understanding.

3. 12 ноября 2024 г. в 19:51 UTCresolved

   This incident has been resolved.

1. 1 октября 2024 г. в 15:48 UTCinvestigating

   We are aware of an issue preventing users from accessing the Incident IQ site or causing blank loading pages. Our engineering team is aware of this and is working to restore system functionality as soon as possible. We will provide further updates as they become available.

2. 1 октября 2024 г. в 16:16 UTCinvestigating

   We have identified the issue and have deployed a resolution to restore access to affected sites are currently monitoring the situation. If you have any additional questions or concerns during this time please contact our support team.

3. 1 октября 2024 г. в 20:41 UTCresolved

   We encountered a platform-wide issue between 11:31 am - 12:10 pm EDT which degraded performance and in some cases prevented login access. Since the remediation deployment this afternoon, we have been monitoring all platform metrics and continuing with extensive testing. We're sorry for any delays or confusion this issue has caused and appreciate everyone's patience as we worked to resolve this issue. If you have any additional questions or concerns please contact our support team for further assistance.

1. 30 сентября 2024 г. в 13:03 UTCinvestigating

   We're aware of an issue preventing users from closing tickets. Our engineering team is working on resolving the issue. We will continue to provide updates as they become available.

2. 30 сентября 2024 г. в 16:55 UTCresolved

   We identified an issue where only adding a resolution to actions that were not visible to requestors would prevent users from closing tickets if resolution actions were required. A fix has been deployed and this issue has been resolved. We're sorry for any delays this may have caused in closing tickets out this morning. If you have any further questions or concerns please reach out to our support team for assistance.

1. 9 сентября 2024 г. в 13:52 UTCinvestigating

   We are aware of an issue preventing users from accessing the Incident IQ site or causing blank loading pages. Our engineering team is working to restore system functionality as soon as possible. We will provide further updates as they become available.

2. 9 сентября 2024 г. в 15:27 UTCinvestigating

   We are continuing to investigate this issue.

3. 9 сентября 2024 г. в 15:30 UTCmonitoring

   We have restored system functionality and will continue to monitor the situation to ensure stability. We will continue to provide updates as they become available, but if you have any additional questions or concerns please contact our support team.

4. 10 сентября 2024 г. в 20:00 UTCresolved

   On September 9th, at approximately 9:49 AM ET, we identified an issue affecting critical infrastructure, which resulted in a temporary service outage. Our team immediately took action, and by 11:04 AM ET, all services were fully restored. There was no data loss during this time. Reason for Outage (RFO): The outage was caused by an unexpected issue in our infrastructure, temporarily impacting system availability. Our engineering team promptly resolved the problem to restore full functionality. Next Steps: To ensure the continued reliability of our platform, we have taken the following steps: Enhanced Monitoring: We have added additional monitoring and alert systems to detect and resolve issues before they impact your service. Infrastructure Optimization: Our systems have been further optimized and scaled to handle even greater demands, providing more stability for critical operations. We understand how essential our platform is for your schools’ day-to-day activities, and we are committed to ensuring its reliability. We deeply regret any inconvenience this may have caused and appreciate your understanding and trust. Incident Resolved: 11:04 AM ET

5. 10 сентября 2024 г. в 20:01 UTCpostmortem

   On September 9th, at approximately 9:49 AM ET, we identified an issue affecting critical infrastructure, which resulted in a temporary service outage. Our team immediately took action, and by 11:04 AM ET, all services were fully restored. There was no data loss during this time. Reason for Outage \(RFO\): The outage was caused by an unexpected issue in our infrastructure, temporarily impacting system availability. Our engineering team promptly resolved the problem to restore full functionality. Next Steps: To ensure the continued reliability of our platform, we have taken the following steps: * Enhanced Monitoring: We have added additional monitoring and alert systems to detect and resolve issues before they impact your service. * Infrastructure Optimization: Our systems have been further optimized and scaled to handle even greater demands, providing more stability for critical operations. We understand how essential our platform is for your schools’ day-to-day activities, and we are committed to ensuring its reliability. We deeply regret any inconvenience this may have caused and appreciate your understanding and trust.