13 Crowdin incidents · 2022年11月 — official updates, affected components, duration and resolution details.
ソリューション ステータスページのレポーティングの問題を解決し、クラウドイン翻訳のダウンタイムの7時間53分を誤って示したインシデントレコードを削除しました。 本サービスは、時間全体にわたって完全に運用されていました。実際のダウンタイムやユーザーへの影響はありません。 何が起こったのか? 「Major Outage」ステータスは、自動化通知の不具合によってトリガーされました。 2つの競合アップデート(DOWNとUP)が同時に到着しました。 レース条件により、ダウンアラートを処理したシステムが同時 UP通知を無視し、ステータスページが誤った状態に陥った状態を残します.
公式のインシデント更新を自動翻訳しています。
We noticed unexpected service behavior, mainly crowdin.com is impacted. Our developers have already started investigating the case, and you will be posted on further updates.
The issue has been identified, and a solution is being implemented. As a temporary measure, users may encounter a CAPTCHA when navigating between pages.
All services are back to the normal state. We will keep the incident open for the next 10 minutes to provide you with a final confirmation.
After thorough monitoring, we can confirm the service is fully back to normal. The incident has now been marked as "Resolved". We will continue to monitor Crowdin's health 24/7/365. Thank you!
We identified degraded performance among organizations hosted in the EU data center. All other Crowdin components are fully accessible. We will keep you posted on any further updates.
The solution was delivered several minutes after the incident was reported, we were monitoring the performance of EU data center organizations and confirmed that it got back to normal
Between 13:40 and 14:10 UTC, a subset of Crowdin Enterprise users experienced server errors when attempting to access projects via the web interface (UI). The issue affected users with specific project configurations. Access via the API remained operational. The service was fully available for translators, though. Our team identified and deployed a fix for the underlying server issue. The problem is resolved, and project access is fully restored. If you were affected by this issue, please refresh your browser page to ensure full functionality. We apologize for the inconvenience.
We have noticed that builds are failing or taking longer than usual due to an issue with the cloud provider. This issue is not global; we are continuing to monitor the status. We will update the status once we confirm that the builds are working properly.
Almost right after the incident was raised, the project builds started working as expected. After conducting various tests, we can confirm that the builds have returned to normal, and the incident can be considered resolved.
As you may be aware, Amazon AWS has experienced a major service disruption. The initial reports were logged at 8:11 am UTC, and within an hour, we noticed that translations export via UI and API started to fail almost instantly. Other components of Crowdin (dashboard, translation editor) were not impacted. We hope that it did not affect your work significantly. Our ticketing system got impacted too, and we couldn't react to the live chat efficiently. Sorry about that. All services are operational currently. We will keep monitoring the situation and update the incident once we confirm that every Crowdin component works as usual.
After nearly 3 hours of monitoring, we confirm that translations are exported on time, and all connectors are fully accessible. We consider the case as resolved.
Our team has spotted a performance downgrade at Crowdin Enterprise. We are continuing to monitor the situation and work on a resolution.
Everything appears to be working properly. We will continue to monitor the system.
After additional monitoring, we can confirm that the service is fully operational
We noticed unusual behavior on the project dashboard, and the API is working more slowly than usual. It's being investigated further by our development team.
All services are fully operational
We experienced a temporary downtime of our Downloads and redirect service, which impacted the ability of some users to download translations.
We noticed that most of the apps are malfunctioning right now, other services (e.g. translation editor, API) function as usual. We are investigating it already
A fix has been implemented, we will monitor the status for a while
The store is fully functioning again, along with all its apps 🙌
We noticed unusual performance on Crowdin, already started investigating it
The service got back to normal, but we keep monitoring the results.
This accident has been fully resolved, after monitoring we confirmed that all functionality returned to normal
Between 13:19 and 13:36 UTC, we experienced performance degradation in crowdin.com and Crowdin Enterprise. The issue has been resolved, and the services are now operating normally.
When your product or service isn’t functioning as expected, let your customers know by creating an incident. Communicate early, even if you don’t know exactly what’s going on.
As you continue to work through the incident, update your customers frequently.
Let your users know once a fix is in place, and keep communication clear and precise.
Empathize with those affected and let them know everything is operating as normal.