27 Codetree incidents · Oktober 2015 — official updates, affected components, duration and resolution details.
Web service became unstable and no longer accessible
The issue has been identified and currently being fixed
AWS Fargate changed the way containers share volumes within a task. This change in behavior caused a deployment failure along with failures on rollback. The solution required redesigning the container infrastructure to not require shared volumes
Search and Indexing services are degraded and we are working to fix this issue. When data is saved / updated, you may receive an error, but in most cases, all data is persisted (but not indexed).
A fix has been promoted and looking to see if performance improves
resolved
We are currently investigating slow performance with Codetree site.
This incident has been resolved.
The issue with burndown activities data has been identified and is being resolved
this incident is resolved and ongoing monitoring is in place
We are currently investigating a degradation of performance and partial outage today.
This incident has been resolved; We'll continue to monitor to ensure all clients are back to normal
Github authentication and API services are degraded due to Github itself currently experiencing issues
continuing to monitor Github for a resolution to their performance issues: https://www.githubstatus.com/incidents/4sh7f76nh5gq
Github resolved their outage as of 5:01 UTC
Some community members are having difficulty opening some issues due to a rendering error
We have identified the problem with a recent release and are working on a fix.. We hope to have something within the next hour or so
A fix has been implemented and we are monitoring results
This incident has been resolved.
We are currently experiencing issues with our caching and client side notifications, overall degrading the experience. We have identified a couple of the issues and working to get fixes in place as soon as possible
We are continuing to work on a fix for this issue.
This incident has been resolved.
We are currently experiencing some issues with search; The search indices are being rebuilt
The issue has been identified and the index is being rebuilt
The index issues have been resolved
GitHub API and other services are currently degraded and unreliable for usage
GitHub seems to be running smoothly again
There seems to be an issue with a recent change to the issues view for a handful of accounts. If you run into this issue, you should still be able to access your project view the board view. https://codetree.com/projects/:project_id/board
We are continuing to work on a fix for this issue.
fix has been pushed and issue has been resolved
A caching server failed leading a small portion of users to not be able to access the site. We are restoring service now
Caching services restored and full connectivity is available again
This incident has been resolved.
We are currently investigating this issue
Search Indexing server failed due to an AWS us-east event today. New Search Indexing instance is online and indexes are currently being rebuilt
Search Indexes have all been refreshed and operations are back to normal
We are currently investigating intermittent 500 gateway issues
This incident has been resolved.
We are currently experiencing intermittent access to website. investigation ongoing
We are continuing to investigate this issue.
resolved connectivity failure
connectivity has been restored; access will continued to be monitored for the next couple of hours
We are experiencing some delays syncing issues from GitHub. We are investigating the cause of the issue.
We're back up and issues are syncing from GitHub normally again. You may see a slight delay as we process the backlog.
GitHub is experiencing an API outage that is causing occasional errors with Codetree. Until GitHub resolves their outage Codetree service won't be 100%.
GitHub has "addressed the source of the exceptions and are continuing to monitor as the site recovers." At Codetree we're doing the same.
GitHub is experiencing API slowness which is affecting Codetree performance.
GitHub API slowdown has been resolved.
We are experiencing an outage right now. We're investigating and will update as soon as we know more.
The issue has been caused by an outage with the company hosting our Redis instance. We've contacted support and will update when we know more.
The outage has been resolved and we're back up.
We've identified the cause and are working towards a resolution.
The incident has been resolved and all systems are back online.
