26 Helpjuice incidents · August 2016 — official updates, affected components, duration and resolution details.
We are currently running a DB maintenance. The app is in read-only mode. We will update the page here as we proceed with the maintenance.
The app is back on regular mode and accepting writes. We are monitoring to make sure all is working correctly after the DB maintenance
The DB maintenance is over and the app is back to normal
The application experienced intermittent slowness and 500 errors due to an unusually high request queue time.
We are currently investigating this issue.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are encountering a major issue that impact all search API calls, we are actively investigating the issue and will provide updates as soon as we have more information.
We've implemented a fix for the recent search issues and are monitoring. We're working with our provider to understand the root cause and prevent it from happening again.
This incident has been resolved.
We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating an issue with some Knowledge Bases experiencing issues with DNS connectivity We'll keep you posted with any updates.
The issue has been resolved, and we are currently monitoring the situation to ensure continued stability.
We are continuing to monitor for any further issues.
Some accounts are still reporting issues. We are continuing to monitor.
This incident has been resolved.
We observed a significant slowdown in application performance at approximately 3:07 PM UTC. A prompt investigation was initiated, leading to the identification of an edge case in one of our knowledge base routes as the root cause. A resolution was developed and subsequently deployed at 3:47 PM UTC. Following the deployment, the application's performance returned to expected levels.
We are investigating what's happening, and estimate a fix within the next 5-6 minutes.
We have successfully fixed this issue. It was due to a database error on Heroku. As a long-term fix, we are looking at making further changes to our infrastructure to avoid this from ever happening.
This incident has been resolved.
Our Hosting Infrastructure provider, Heroku.com had a hardware failure on one of our follower databases, we proceeded by switching to the backup database until the issue is resolved. We can confirm that no data has been lost. The outage lasted about 5 minutes.
Our Hosting Infrastructure provider, Heroku.com had a hardware failure on one of our follower databases, we proceeded by switching to the backup database until the issue is resolved. We can confirm that no data has been lost. The outage lasted about 5 minutes.
We are currently investigating an issue with assets (CSS/JS) not loading.
The issue has been identified and a fix is being implemented.
This incident has been resolved.
We're noticing certain knowledge bases responding slower than usual. We're actively looking into this.
A fix has been implemented and we are monitoring the results.
We are continuing to monitor for any further issues.
This incident has been resolved.
We're experiencing an elevated level of API errors and are currently looking into the issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We're noticing that certain knowledge bases that use our LetsEncrypt SSL are having their SSL not work properly for their knowledge base (not backend). If you have this issue, it'll render a "Not secure" message, which you can override and still use the knowledge base. We are, however inspecting why this is happening to certain clients and expect a fix ASAP.
We are continuing to investigate this issue.
This incident has been resolved.
Certain customers that were using custom SSL certificates had random timeout issues happen because our SSL certificate server that was hosting and rendering the SSL was overloaded. We have since fix the issue and are permanently reconstructing SSL server to support custom SSL certificates. This was an issue because we were using heroku and a customer to sell on the DNS front, so things would not scale together but separately
This incident has been resolved. We are reconstructing SSL servers.
Some clusters were affected by slow responses and big spikes in latency. The issue was resolved within 20 minutes of the first incident notice.
Some clusters were affected by slow responses and big spikes in latency. The issue was resolved within 45 minutes of the first incident notice.
Our Hosting Infrastructure provider, Heroku.com / AWS: Hardware failure was detected on one of our follower database, we proceeded by switching to backup database until hardware issue is resolved. We can confirm that no data has been lost. Outage lasted ~10 min.
Some clusters were affected by slow responses and big spikes in latency. The issue was resolved within 6 hours of the first incident notice.