27 Testrail incidents · 2021年7月 — official updates, affected components, duration and resolution details.
我们知道一个影响TestRail的问题,一些用户可能会遇到较慢的性能和/或500个出错. 我们的团队正在努力恢复全部功能. 一俟我们有更多东西要分享,将立即提供最新情况.
已经执行了一项措施,我们正在监测恢复情况。 随着系统恢复,在此期间可能会出现一些暂时的不稳定. 一旦系统完全稳定,我们将进行更新.
影响TestRail的问题已经解决,所有服务现已全面运作. 谢谢你的耐心.
自动翻译自官方事件更新。
我们发现了一个DNS问题,它可能阻止了一些用户访问TestRail. 这个问题已经解决,所有服务现已全面运作。 我们为任何不便表示歉意并感谢你的耐心.
自动翻译自官方事件更新。
We identified a performance issue affecting a subset of TestRail Cloud accounts, resulting in slower page load times for some users. Our team identified the underlying cause and implemented a mitigation that has stabilized performance. We are continuing to closely monitor the environment while we work toward a permanent resolution. We appreciate your patience and understanding as we work to fully resolve the issue.
We continue to closely monitor performance for the affected Cloud instances following the mitigation applied earlier. Our team has identified a permanent resolution, which is currently undergoing review and testing. Thank you for your continued patience while we finalize this work.
The issue affecting a small subset of customers has been resolved and all services are now fully operational. Thank you for your patience.
We are currently investigating an issue affecting authentication for the Jira integration. Some users may experience failures when connecting to Jira or syncing data between TestRail and Jira. Our team is actively working to identify the root cause and will provide updates as more information becomes available.
We are continuing to investigate this issue.
We have identified the root cause of the issue affecting authentication for the Jira integration and are actively working to implement a fix. Some users may continue to experience failures when connecting to Jira or syncing data between TestRail and Jira during this time. We will provide further updates as remediation progresses.
We are continuing to implement a fix for the issue. Further updates will be provided as progress continues.
Work to implement a fix is ongoing. During this time, some users may encounter “500 server error” messages when attempting to authenticate with or use the Jira integration. We will share additional updates as they become available.
We are continuing to make progress on implementing the fix. Some customers may still experience issues, including “500 server error” messages when using the Jira integration. We appreciate your patience and will share further updates as they become available.
A fix has been implemented for the issue affecting authentication with the Jira integration. We are currently monitoring to ensure the issue is fully resolved. Users will need to reauthenticate with Jira within TestRail for the fix to take effect. After reauthenticating, normal functionality should resume. We will continue to monitor and provide further updates as needed.
Following the implementation of the fix, we have completed a period of monitoring and have confirmed that the issue affecting authentication with the Jira integration has been fully resolved, with no further errors observed. Users should ensure they have reauthenticated with Jira within TestRail to maintain normal functionality. Thank you for your patience while we worked to resolve this issue.
Some customers may be experiencing degraded performance. Our engineers are investigating.
Our team is actively working to resolve the incident. Customers may see their instances in maintenance mode intermittently, but for no longer than 15 minutes.
Some customers may experience maintenance windows longer than 15 minutes as our team continues to work to resolve the incident.
The incident has been resolved, and all customers should now have access to their instances. Please note that we have temporarily increased the API rate limit restrictions, and reduced the background task frequency while we monitor the system. We appreciate your patience during this time, and are committed to maintaining service stability.
Some customers are experiencing degraded performance. Our engineers are investigating.
We're pleased to report significant performance improvements across our platform, with decreased error rates and API rate limits restored to their normal levels. Our team continues to closely monitor the system to ensure ongoing stability and reliability. Thank you for your patience and support.
This incident has been resolved.
Some customers may be experiencing degraded performance. Our engineers are investigating.
Our engineers have identified the potential source and implemented a fix.
This incident has been resolved.
We identified degraded performance that may affect some customers in connection with the 14-Feb partial outage. Our team is actively investigating to ensure a complete resolution.
We're continuing to monitor for any further issues while the team is working to a full resolution.
We successfully resolved the issue, and normal service has been restored.
We're currently experiencing a partial Cloud outage which may result in error messages relating to failing API calls or attachment related nodes. We're working on the issue right now and expect to have a resolution shortly.
Customers should be seeing improvements. We will continue to monitor whilst services return to normal.
We are continuing to monitor this issue.
Monitoring continues while the rate of errors continues to reduce sharply.
The incident has been resolved. We are continuing to monitor for errors.
The TestRail Cloud Attachments service is currently experiencing performance issues affecting a subset of attachments functionality for a small number of TestRail Cloud users. Our DevOps team is deploying a fix and we expect to resolve this matter promptly.
The issue has been identified and a fix is being implemented.
We're still working to fix this issue as fast as we can. Thanks for your patience while we do so. We'll share updates as progress is made.
We are continuing to work on a fix for this issue which is affecting a small number of customers. Thanks for your patience. Our team is working hard to complete this activity as quickly as possible and we appreciate your bearing with us while we do so.
Our team is stabilising our hosted infrastructure after applying the necessary fixes. We are continuing to monitor the situation closely.
Our DevOps team has applied a fix for this issue and after a period of monitoring, has determined that users should no longer be experiencing issues related to the TestRail attachments service. We appreciate your patience while we dealt with this matter. If you're still experiencing errors that result in a "Call to a member function prepare()" message, please reach out to our support team and we'll be happy to assist.
One of our databases is not responding correctly and as a result, TestRail instances reliant on it are currently in maintenance mode. Our DevOps team is investigating the problem.
The cause of the database issue has been identified and our engineers are working to solve the problem. In the meantime, we are also building a new database from a backup with a view towards switching affected customers to the new database.
The database has been recovered and all affected TestRail cloud instances should now be back online. We are continuing to monitor the database for any further issues.
All systems are operational.
We are currently investigating this issue.
The issue has been identified and a fix implemented. We will be continuing to monitor the situation.
We are currently investigating this issue.
We have identified the cause of this issue and we are currently in the process of implementing a fix.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
Some customers are experiencing gateway errors, e.g. '502 Bad Gateway', '504 Gateway Time-out', or '403-Access Denied' when attempting to access their TestRail cloud instance. We are currently investigating this issue and will provide further updates shortly.
We are continuing to investigate this issue.
We have identified the issue in which some customers have experienced gateway errors (502 - Bad Gateway, 504 - Gateway Time-out, 403 - Access Denied) when attempting to access their TestRail Cloud instance. We have resolved the issue for several of our customer instances and expect to resolve the remaining instances shortly.
We have fixed the issue encountered by some customers earlier today in regards to Gateway Access issues. We are monitoring the situation.
This incident has been resolved.
Some customers are experiencing slower than expected TestRail performance. We are currently investigating this issue.
We have identified the cause of the instability and implemented a fix which we are currently monitoring.
This incident has been resolved.
Some customers are experiencing an error when attempting to add test results. We are currently investigating the cause of this issue and will provide an update shortly.
We are continuing to investigate this issue and will provide further updates shortly. In the meantime, users are able to workaround the problem by using the "Pass & Next" button to add a result instead of the "Add Result" button.
Our team has identified the issue and implemented a fix. All cloud customers will receive the fix shortly.
We have deployed the fix to all customers and are monitoring for any further issues.
This incident has been resolved.
Some customers are experiencing gateway errors, e.g. '502 Bad Gateway' or '504 Gateway Time-out', when attempting to access their TestRail cloud instance. We are currently investigating this issue and will provide further updates shortly.
This incident has been resolved.
Some customers are experiencing gateway errors, e.g. '502 Bad Gateway' or '504 Gateway Time-out', when attempting to access their TestRail cloud instance. We are currently investigating this issue and will provide further updates shortly.
Earlier today we had some further reports of further 504 and 502 gateway errors affecting a small number of customers. We've identified the cause of the errors and implemented a fix. Service to TestRail Cloud has been fully restored and all instances are showing as online and operational. We will continue to monitor closely to ensure there are no other issues. If you are still receiving any 502 or 504 gateway errors, please email us at [email protected] so that we can take a closer look at your specific instance.
This incident has been resolved.
TestRail Cloud is currently experiencing issues. We know this is frustrating and we’re working to resolve this as soon as possible. We have escalated this to our engineering team, and will provide an update as soon as more information becomes available.
We have identified and resolved the issue. All instances are operational, however we are continuing to closely monitor the situation.
This incident has been resolved.
TestRail Cloud is fully operational, but we are continuing to monitor and investigate reports of intermittent 504 and 502 gateway errors, including for the TestRail API, as well as some other issues related to timeouts. We will post an update as soon as we have more information. If you or a teammate is experiencing any issues, please notify our Support team at [email protected] so we can help triage.
Earlier today we had some further reports of further 504 and 502 gateway errors affecting a small number of customers. We've identified the cause of the errors and implemented a fix. Service to TestRail Cloud has been fully restored and all instances are showing as online and operational. We will continue to monitor closely to ensure there are no other issues. If you are still receiving any 502 or 504 gateway errors, please email us at [email protected] so that we can take a closer look at your specific instance.
This incident has been resolved.
Some customers are experiencing gateway errors, e.g. '502 Bad Gateway' or '504 Gateway Time-out', when attempting to access their TestRail cloud instance. We are currently investigating this issue and will provide further updates shortly.
We are continuing to investigate this issue. We do not have an ETA for resolution at the present time, but expect to provide one shortly.
We are continuing to investigate this issue.
TestRail Cloud continues to experience issues with 502 or 504 gateway errors. We know this is frustrating and we’re working to resolve this as soon as possible. We have escalated this to our engineering team, and will provide an update as soon as more information becomes available.
We've identified the cause of the gateway errors and implemented a fix. Service to TestRail Cloud has been fully restored and all instances are showing as online and operational. We will continue to monitor closely to ensure there are no other issues. If you are still receiving any 502 or 504 gateway errors, please email us at [email protected] so that we can take a closer look at your specific instance.
This incident has been resolved.