Incident Iq 장애 기록

We are currently experiencing an outage from our phone vendor. To contact Support please put in a ticket via the in application messenger, or email us at support@incidentiq.com. We will provide an update here as soon as the phones are working again.

Resolved: Phone Services Restored The issues affecting our phone systems have been fully resolved, and all lines are back up and running. We apologize for the inconvenience and appreciate your patience while we worked to restore functionality.

We are investigating reports of an issue affecting the use of certain characters in the ticket comments fields. Workaround: You can type up notes in another field or text editor, such as Word or Notepad, and copy the text into the comments field. The rest of the platform remains operational. We are working to identify the problem and will update you soon.

We are pleased to share that a resolution has been implemented, and our team is currently monitoring the system to ensure continued stability. If you are still seeing any irregularities, please refresh your session by closing and restarting your browser.

The underlying root cause has been addressed and our monitoring confirms that all systems are functioning as expected. If you continue to see any local cached issues, we recommend a final browser restart.

We are aware of an issue affecting the loading of tickets. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience as we work to resolve this issue.

We are pleased to confirm that the issue has been resolved, and full functionality is restored. Our team is actively monitoring the system to ensure continued stability. Should you experience any further issues, please contact our support team immediately. We apologize for any inconvenience this caused and thank you for your patience.

We are pleased to report that the service issue reported earlier today has been resolved. Following a thorough review, our internal monitoring and testing have confirmed that all systems are fully operational. At approximately 1:30 PM ET, our support team was made aware of an issue preventing certain pages from loading within the platform. Our Engineering team, which was already investigating, confirmed that full system functionality was restored at 2:10 PM ET. We sincerely apologize for the disruption and appreciate your patience as we verified system health. If you are still experiencing any issues, please contact our support team directly for further investigation.

At separate times on October 27th and 30th, IncidentIQ experienced brief periods of degraded platform performance. During these windows, users experienced slow page loads, timeouts, and overall sluggish response times across several areas of the application. The issues were remediated by adjusting, but not fundamentally changing, cloud database configurations. Initial investigation by the engineering team identified issues with a patch deployed by one of our upstream cloud vendors. While working with the vendor to confirm the cause and roll back the change, iiQ engineers increased capacity and deployed additional resources to ensure performance was not impacted between October 30th and November 7th. On November 7, 2025 the provider’s engineers rolled back the patch overnight. No similar incidences of these symptoms have occurred since the rollback, and we expect the rollback will fully resolve this class of issues. We know that platform reliability is critical for your team's operations. Thank you for your patience as we resolved this with the vendor.

We are aware of intermittent issues affecting page loads and login issues. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience as we work to resolve this issue.

We are pleased to report that all systems are stable and operating normally. Our team will continue to closely monitor performance to ensure continued stability. Thank you for your patience.

Following a review, our internal monitoring and testing show that all systems are fully operational. We appreciate your patience as we verified system health. If you are still experiencing any issues, please contact our support team directly so we can investigate further.

At separate times on October 27th and 30th, IncidentIQ experienced brief periods of degraded platform performance. During these windows, users experienced slow page loads, timeouts, and overall sluggish response times across several areas of the application. The issues were remediated by adjusting, but not fundamentally changing, cloud database configurations. Initial investigation by the engineering team identified issues with a patch deployed by one of our upstream cloud vendors. While working with the vendor to confirm the cause and roll back the change, iiQ engineers increased capacity and deployed additional resources to ensure performance was not impacted between October 30th and November 7th. On November 7, 2025 the provider’s engineers rolled back the patch overnight. No similar incidences of these symptoms have occurred since the rollback, and we expect the rollback will fully resolve this class of issues.

We are experiencing some intermittent site responsiveness issues that are causing slow page loading times. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience.

A fix has been implemented and we are monitoring the results.

Apologies for the inconvenience and thank you for your patience This incident has been resolved. If you experience any additional problems please reach out to Support.

We are aware of an intermittent issue affecting the loading of tickets and assets views. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience as we work to resolve this issue.

We have deployed a fix to address the intermittent issue with tickets and assets views loading and are currently monitoring the situation. We will provide additional updates as they become available. In the meantime, please contact our support team if you have any additional questions or concerns during this time.

Our internal monitoring and testing indicates that the platform and all services are functioning as expected. If you have any additional questions or concerns, please reach out to support. We apologize for any disruption this may have caused, and thank you for your understanding during this time.

We are aware of an intermittent issue affecting the loading of assets on certain pages. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience as we work to resolve this issue.

We have deployed a fix to address the intermittent asset loading issue and are currently monitoring the situation. We will provide additional updates as they become available. In the meantime, please contact our support team if you have any additional questions or concerns during this time.

We are aware of intermittent issues with check-in/out and verifications. Our Engineering Team is currently working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience as we work to resolve this issue.

We have deployed a fix to address the issues with check-in/out and verifications and are currently monitoring the situation. We will provide additional updates as they become available. In the meantime, please contact our support team if you have any additional questions or concerns during this time.

Our internal monitoring and testing indicates that the platform and all services are functioning as expected. If you have any additional questions or concerns, please reach out to support. We apologize for any disruption this may have caused, and thank you for your understanding during this time.

We've identified intermittent issues with fee tracker app, as well as, displaying and filtering on certain elements in assets explorer. There is no loss or impact to the underlying data. Thanks for your patience as our Engineering team works to resolve the issue as soon as possible.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We are aware of an issue intermittently preventing users from accessing the Incident IQ site or seeing blank pages load. Our engineering team is aware of this and is working to restore full system functionality as soon as possible. We will provide further updates as they become available.

We have deployed a fix to address the login and page-loading issues and are currently monitoring the situation. We will provide additional updates as they become available. In the meantime, please contact our support team if you have any additional questions or concerns during this time.

This incident has been resolved. We deeply regret any inconvenience this may have caused and appreciate your understanding as we worked to resolve the issue. If you have any additional questions or concerns please contact our support team for further assistance.

We are currently aware of an issue causing delays in job processing at the moment. Our team is working on identifying and resolving the underlying cause as soon as possible. We will continue to provide updates here as information becomes available.

Our team has identified the underlying cause of this issue, and a fix has been deployed. The system is currently processing all delayed jobs, and we hope to have full functionality restored shortly.

All delayed jobs have been processed, and the system is once again performing as expected. We appreciate your patience as our team worked to resolve this issue. If you have any additional questions or concerns, please contact our support team for further assistance.

Our phone provider is currently experiencing an outage affecting our ability to receive inbound calls. Please use the in-system support ticketing or email (support@incidentiq.com) to reach out for assistance. We will provide updates as functionality is restored, and we appreciate your understanding during this time.

Our phone support is back online. We will continue to monitor the situation for the time being. If you continue to experience any issues during this time, please reach out through the in-system support ticketing or email (support@incidentiq.com).

After monitoring the situation, everything appears stable and functioning as expected. If you have any additional questions or concerns, please reach out to support. We apologize for any disruption this may have caused, and thank you for your understanding during this time.

We are aware of an issue preventing users from accessing the Incident IQ site or causing blank loading pages. Our engineering team is aware of this and is working to restore system functionality as soon as possible. We will provide further updates as they become available.

We have deployed a fix to address the slowness and page-loading issues and are currently monitoring the situation. We will provide additional updates as they become available. In the meantime, please contact our support team if you have any additional questions or concerns during this time.

This incident has been resolved. Our Engineering team is completing an investigation into the cause and will provide an update here as soon as the postmortem is complete. We deeply regret any inconvenience this may have caused and appreciate your understanding as we worked to resolve the issue. If you have any additional questions or concerns please contact our support team for further assistance.

We are experiencing some intermittent site responsiveness issues that are causing slow page loading times and intermittent login problems. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience.

We have identified the issue and implemented a resolution. We are continuing to monitor the platform for stability and responsiveness. Additional updates regarding the cause of the issue and remediation steps will be posted here as soon as they are available.

During our continued platform monitoring, we have identified additional intermittent loading and responsiveness issues. Our Engineering team is focused 100% on returning the platform to optimal operational status. We will post updates here as they become available.

We are experiencing some intermittent site responsiveness issues that are causing slow page loading times and intermittent login problems. Our Engineering Team is currently investigating the cause and working on a solution. We will provide updates here as soon as they are available. Apologies for the inconvenience and thanks for your patience.

A fix has been implemented and we are monitoring the results.

This incident has been resolved and will be updated with a postmortem as soon as our investigation is complete. We understand how essential our platform is for your schools’ day-to-day activities, and we are committed to ensuring its reliability. We deeply regret any inconvenience this may have caused and appreciate your understanding as we worked to resolve the issue.

On Monday December 2, Incident IQ experienced a service interruption that caused degradation of services for several customers. ‌ **Timeline of events:** * 9:39 AM ET - Began receiving customer reports of slowness/degradation of services for portions of the platform to include login authentication. * 11:41 AM ET - All services and platform responsiveness levels restored **Cause of Incident:** * As part of normal operating procedures, Incident IQ scales platform capacity to meet the needs of our current customer and predictive demands. * Incident IQ uses the Microsoft Azure platform to  automatically scale infrastructure as part of normal operating procedures.  The process to scale encountered errors on Monday December 2nd and took five times longer than normal to complete. * The combination of infrastructure not scaling on normal schedules and the larger load than the available infrastructure caused the outage. **Remediation:** * Services were ultimately restored by a combination of manually adding infrastructure resources and the automated scaling job completion * Incident IQ is working alongside Microsoft and have identified that unforeseeable abnormally high transactions on the SQL environment during scaling caused the process to run longer. * We are continuing to work with Microsoft to prevent a recurrence of this issue. The reliability of our platform remains of the utmost importance to us. We understand the impact these moments have on our customers. The remediations put in place to prevent a recurrence of this particular incident, as well as the processes we have in place to continuously improve the platform, provide us with a level of certainty that we are able to stay ahead of unexpected surges in traffic. As before, we do sincerely apologize for this disruption and want to thank you for your patience and partnership as we worked through this issue.

We want to inform you that we are currently experiencing a performance issue that may impact your experience on our platform. You may notice delays in loading ticket pages, performing searches, taking actions, or creating new tickets. Our engineering team is actively investigating the issue, and we are working to resolve it as quickly as possible. We apologize for any inconvenience this may cause and appreciate your patience. We will keep you updated on our progress and notify you as soon as the platform is performing at optimal levels. Thank you for your understanding and partnership.

The engineering team has reported that the platform is now stable, and they are actively investigating the root cause of the event to prevent any recurrence. I’ll keep you posted on any further findings or actions. Thank you for your patience and understanding.

This incident has been resolved.

We are aware of an issue preventing users from accessing the Incident IQ site or causing blank loading pages. Our engineering team is aware of this and is working to restore system functionality as soon as possible. We will provide further updates as they become available.

We have identified the issue and have deployed a resolution to restore access to affected sites are currently monitoring the situation. If you have any additional questions or concerns during this time please contact our support team.

We encountered a platform-wide issue between 11:31 am - 12:10 pm EDT which degraded performance and in some cases prevented login access. Since the remediation deployment this afternoon, we have been monitoring all platform metrics and continuing with extensive testing. We're sorry for any delays or confusion this issue has caused and appreciate everyone's patience as we worked to resolve this issue. If you have any additional questions or concerns please contact our support team for further assistance.

We're aware of an issue preventing users from closing tickets. Our engineering team is working on resolving the issue. We will continue to provide updates as they become available.

We identified an issue where only adding a resolution to actions that were not visible to requestors would prevent users from closing tickets if resolution actions were required. A fix has been deployed and this issue has been resolved. We're sorry for any delays this may have caused in closing tickets out this morning. If you have any further questions or concerns please reach out to our support team for assistance.

We are aware of an issue preventing users from accessing the Incident IQ site or causing blank loading pages. Our engineering team is working to restore system functionality as soon as possible. We will provide further updates as they become available.

We are continuing to investigate this issue.

We have restored system functionality and will continue to monitor the situation to ensure stability. We will continue to provide updates as they become available, but if you have any additional questions or concerns please contact our support team.

On September 9th, at approximately 9:49 AM ET, we identified an issue affecting critical infrastructure, which resulted in a temporary service outage. Our team immediately took action, and by 11:04 AM ET, all services were fully restored. There was no data loss during this time. Reason for Outage (RFO): The outage was caused by an unexpected issue in our infrastructure, temporarily impacting system availability. Our engineering team promptly resolved the problem to restore full functionality. Next Steps: To ensure the continued reliability of our platform, we have taken the following steps: Enhanced Monitoring: We have added additional monitoring and alert systems to detect and resolve issues before they impact your service. Infrastructure Optimization: Our systems have been further optimized and scaled to handle even greater demands, providing more stability for critical operations. We understand how essential our platform is for your schools’ day-to-day activities, and we are committed to ensuring its reliability. We deeply regret any inconvenience this may have caused and appreciate your understanding and trust. Incident Resolved: 11:04 AM ET

On September 9th, at approximately 9:49 AM ET, we identified an issue affecting critical infrastructure, which resulted in a temporary service outage. Our team immediately took action, and by 11:04 AM ET, all services were fully restored. There was no data loss during this time. Reason for Outage \(RFO\): The outage was caused by an unexpected issue in our infrastructure, temporarily impacting system availability. Our engineering team promptly resolved the problem to restore full functionality. Next Steps: To ensure the continued reliability of our platform, we have taken the following steps: * Enhanced Monitoring: We have added additional monitoring and alert systems to detect and resolve issues before they impact your service. * Infrastructure Optimization: Our systems have been further optimized and scaled to handle even greater demands, providing more stability for critical operations. We understand how essential our platform is for your schools’ day-to-day activities, and we are committed to ensuring its reliability. We deeply regret any inconvenience this may have caused and appreciate your understanding and trust.

Dear Valued Customer, We wanted to make you aware that our Incident IQ platform is currently experiencing a service outage that began at approximately 2:05 PM EST. At this time, users are unable to load or access support tickets within the platform. Our engineering team is fully engaged and has prioritized this issue. They are actively investigating the root cause and working to restore full functionality as quickly as possible. While we do not have an estimated time for resolution yet, we will continue to provide timely updates as more information becomes available. Please know that resolving this matter is our highest priority, and we are committed to minimizing the impact on your operations. We understand the importance of our platform to your workflow and sincerely apologize for the disruption this may be causing. Thank you for your patience and understanding during this time. We will keep you updated with progress and provide the next update as soon as possible. If you have any immediate concerns or need assistance, please don't hesitate to contact our support team.

A fix has been implemented and we are monitoring the results.

We are continuing to monitor for any further issues.

Dear Valued Customer, We are pleased to inform you that our engineering team has identified the issue affecting the Incident IQ platform and has successfully applied a fix. The system is now operational, and we are closely monitoring for stability to ensure everything is functioning as expected as of 2:17pm EST. Our team is also conducting a thorough root cause analysis to prevent future occurrences. If you are still experiencing any issues or notice any irregularities, please reach out to our support team as soon as possible so we can assist you promptly. Thank you for your continued patience and understanding as we worked to resolve this issue.

This incident has been resolved.

We are aware of an issue preventing users from accessing the Incident IQ site or causing blank loading pages. Our engineering team is aware of this and is working to restore system functionality as soon as possible. We will provide further updates as they become available.

Site access and responsiveness should be restored to normal operation. If you encounter any ongoing issues, please report them via email to support@incidentiq.com. Root cause investigation is ongoing and we are actively monitoring all internal performance logs and dashboards to ensure site performance remains optimal. We will continue to provide additional updates here as information is available and validated. Apologies for this interruption in service.

At approx. 10:30 AM EDT today users began to experience issues accessing Incident IQ. The engineering team engaged and identified an issue that was affecting the ability for users to access the IQ platform. Access was restored at approx. 11:15 AM EDT. Though all systems have been restored, we will continuing to monitor all platform KPIs to ensure there are no additional disruptions. As always, we appreciate everyone's patience as we worked to resolve this situation. If you have any additional questions or concerns please feel free to contact the support team.

We are experiencing an issue that is causing outages and degraded services. Our engineering is investigating we will provide updates as they become available.

All systems have been brought back online and we are currently monitoring the situation as a precaution. We will continue to provide additional updates as they become available. If you have any questions or concerns please contact our support team.

Starting at 3:10 PM EDT today user began to experience issues accessing Incident IQ. Our engineering team quickly identified the issue and deployed a fix to restore system service soon after. The outage lasted roughly 20 minutes. Though all systems have been restored we are continuing to monitor all platform KPIs to ensure there are no additional disruptions. As always we appreciate everyone's patience as we worked to resolve this situation and if you have any additional questions or concerns please contact our support team.