50 Transifex incidents · marzec 2019 — official updates, affected components, duration and resolution details.
Możesz mieć trudności z pobieraniem zasobów i / lub korzystaniem z API. Obecnie badamy tę sprawę. Dziękuję za cierpliwość i zrozumienie!
Kontynuujemy śledztwo w tej sprawie! Dziękuję za cierpliwość i zrozumienie!
Automatyczne tłumaczenie oficjalnej aktualizacji incydentu.
Obecnie badamy tę sprawę
Problem został zidentyfikowany i jest w trakcie wdrażania
Ten incydent został rozwiązany.
Automatyczne tłumaczenie oficjalnej aktualizacji incydentu.
We are currently investigated this issue
The issue has been identified and a fix is being implemented
A fix has been implemented and we are monitoring the results
The system looks stable. We apologize for any issues you might be experiencing.
We are currently investigating the issue
We are still actively investigating the issue and working to identify the root cause. We sincerely apologize for the delays and the API errors you are experiencing, and we understand the impact this may have on your workflows. Our team is fully engaged, and we will share another update as soon as we have more information.
The issue has been identified, and a fix has been applied. Service has been restored, and performance is returning to normal.
The fix has been successfully applied, and system performance has stabilized. We are continuing to monitor all systems closely to ensure sustained stability.
The issue has been fully resolved, and services are now operating normally. We apologize for any inconvenience caused and thank you for your patience and understanding.
**API v3 Partial Downtime** **Date:** 02 April 2026 **Service Impacted:** api-v3 **Impact Scope:** Services depending on api-v3 experienced degraded functionality **Summary** On April 2nd, api-v3 experienced partial downtime due to a database performance issue. A database query caused contention within the database, leading to increased resource utilization and degraded responsiveness of api-v3 pods. As a result, api-v3 instances were intermittently failing health checks and restarting, which further amplified database load and connection pressure. **Impact** * Degraded performance and intermittent unavailability of api-v3 * Increased error rates for dependent services * Elevated database CPU usage and connection counts **Root Cause** The incident was caused by a database query that partially blocked operations within the database. This led to: * Query contention and locking behavior * Increased execution times for other queries * Accumulation of database connections from api-v3 pods This combination resulted in reduced system responsiveness and instability in api-v3. **Mitigation & Resolution** * Problematic long-running queries were identified and terminated * The affected query was optimized to improve performance and reduce locking behavior Following these actions, system performance returned to normal and stability was restored. **Follow-up Actions** * Optimize database query performance related to the incident * Improve monitoring and alerting for long-running or blocking queries * Evaluate database connection handling and limits in api-v3 * Consider implementing specific query timeouts and safeguards to prevent similar issues **Current Status** The issue has been resolved, and the system has remained stable since the fix was applied.
We are currently investigating the issue.
We are still investigating the issue. We apologize for any inconvenience this issue has caused you.
The issue has been identified, and a fix is being implemented.
The app performance degradation has been resolved after the fix we implemented. We are now monitoring the system performance to ensure everything remains stable. Thank you for your patience.
The incident has been resolved, and everything is now working as expected. Thank you so much for your patience, and apologies for the disruption earlier!
**Service-to-Service Communication Degradation** **Summary** During the incident, we observed intermittent, seemingly random delays when refreshing the application at [app.transifex.com](http://app.transifex.com). These delays affected the handling of customer requests and overall platform responsiveness. **Investigation & Findings** Following the initial reports, we investigated our load balancer logs and identified an unusually high volume of requests between our core service and the Transifex-delivery service. By correlating the load balancer logs with our application logs, we determined that a substantial number of requests were being throttled and retried between those services. This led to performance degradation when handling incoming requests on our main platform. **Impact** The issue caused performance degradation across our platform at irregular time intervals. **Mitigation & Resolution** As part of scheduled maintenance, we temporarily paused translation activity to adjust certain configurations. After the maintenance changes were applied, translation activity was re-enabled. However, the system did not immediately return to a fully stable state. To ensure reliable request handling and prevent further performance degradation, we made additional adjustments to the affected language configurations to reduce internal service load and fully stabilize the platform. Once system stability was fully confirmed, all affected languages and their translations were restored, and the platform was gradually returned to full functionality. **Preventive Actions & Improvements** To reduce the risk of similar issues occurring in the future, we have introduced improvements to how requests are handled between our core platform services and the Transifex-delivery service, including configuring throttling and retry mechanisms.
We are currently investigating the issue.
We are investigating the issue.
The issue is still under investigation, and our team is working to resolve it as quickly as possible. Thank you for your patience.
The API and the app performance degradation have been resolved after taking the necessary actions on our side. We are now monitoring the system performance to ensure everything remains stable. Thank you for your patience.
The issue has been resolved, and everything is now working as expected. Thanks so much for your patience — apologies for the disruption earlier!
We are currently experiencing an issue affecting the subscription pages within our app. As a result, some users may encounter errors, slow performance, or temporary unavailability when accessing subscription-related features. Our investigation indicates that the issue stems from an upstream AWS service disruption (https://health.aws.amazon.com/health/status). We are closely monitoring the situation and will restore full functionality as soon as AWS resolves the underlying issue. Updates will be posted here as more information becomes available.
Latest update from AWS: "The underlying DNS issue has been fully mitigated, and most AWS Service operations are succeeding normally now. Some requests may be throttled while we work toward full resolution. Additionally, some services are continuing to work through a backlog of events such as Cloudtrail and Lambda. While most operations are recovered, requests to launch new EC2 instances (or services that launch EC2 instances such as ECS) in the US-EAST-1 Region are still experiencing increased error rates. We continue to work toward full resolution. If you are still experiencing an issue resolving the DynamoDB service endpoints in US-EAST-1, we recommend flushing your DNS caches. We will provide an update by 4:15 AM, or sooner if we have additional information to share." We’re monitoring the situation closely and will provide additional updates as AWS shares more information.
Subscription pages are currently operating normally. We will continue to monitor and provide updates as needed. We appreciate your patience.
AWS is continuing recovery efforts. Our systems remain stable, but we continue to monitor closely for any potential impact.
The underlying AWS issue has been resolved, and everything is now fully operational. We’ll continue to monitor system performance, but no further impact is expected. This incident is now considered resolved.
We are performing scheduled maintenance on our infrastructure from 12:00 (UTC) until 12:30 (UTC). During this time, AI Translations may experience brief delays. Thank you for your patience!
Maintenance is complete.
Maintenance is complete. We will continue to monitor closely.
All services are back to normal. Thank you for your patience during this time.
We are investigating intermittent delays with GitHub, Bitbucket, and Gitlab Integrations. You may experience a delay in your workflows. Our team is actively working to mitigate the delays while simultaneously investigating the root cause of the issue. We will provide an update as soon as we have more information to share. Thank you all for your patience, and apologize for any inconvenience caused.
The issue has been identified, and we are actively working on applying the necessary fix to restore functionality without delays.
A fix has been implemented and we are monitoring the results.
The issue has been resolved, and Git app integrations operate normally. The incident was caused by a high rate of incoming tasks, which created a bottleneck in our queueing system and led to delays in task processing. Our team has implemented the necessary fix and will continue to monitor the system closely to ensure continued stability. We sincerely apologize for the inconvenience this may have caused and appreciate your patience throughout the resolution process. If you continue to experience any issues, please don’t hesitate to contact our support team.
We are currently experiencing delays in fillup processing. We apologize for any inconvenience this may be causing and appreciate your patience as we work to resolve this.
We have identified the cause of the delays. The issue is related to a high queue volume, and our team is actively working to address it. Steps are being taken to restore normal processing speeds.
A solution has been applied, and we're closely monitoring the situation.
We are happy to inform you that the issue has been fully resolved. Our team worked diligently to address the problem, and everything is now back to normal. Thank you for your patience and understanding during this time.
We are currently facing issues with the delivery of translations through our CDN, specifically related to Transifex Native projects. As a result, you may encounter issues with the availability of translations for certain languages. Our team is actively investigating the issue and is committed to restoring full functionality as quickly as possible. We appreciate your patience during this time and will provide updates as soon as more information becomes available.
We are still investigating the issue affecting translation delivery for certain languages. We will share updates as soon as we have more information.
Due to a configuration issue, new content updates were unavailable in certain locales. Our team has applied a fix, and normal functionality has been restored. We will continue to monitor to ensure everything works as expected.
We are pleased to inform you that the incident reported has been successfully resolved. We appreciate your patience and understanding during this incident. If you have any questions or require assistance, please do not hesitate to reach out to our support team.
You may experience intermittently a 504 error from our API. Our team is working on identifying the root cause and closely monitoring the system's behavior. We will provide an update as soon as we have more information to share. Thank you all for your patience and we apologize for any inconvenience caused.
We are still investigating the issue. At the moment, we are applying some changes, and we need to monitor the situation. We will keep you posted on our progress.
API is at operational status again. We will continue to monitor the situation and provide updates.
Our team developed a fix to deal with the intermittent issues on our API.
The fix has been deployed, and our team is monitoring the results.
The Issue has been addressed, and everything is back to normal. Thank you for your patience, and we apologize for any inconvenience.
Our team is actively working to identify the cause and restore full functionality as soon as possible. We apologize for any inconvenience this may cause and appreciate your patience. Thank you for your understanding.
We are pleased to inform you that the issue has been resolved, and all systems are back to normal operation.
Our systems are operating normally, and all services are back to full functionality.
We are actively investigating the issue and will provide updates as we progress. We apologize for any inconvenience this may have caused.
The application is now back online. We're closely monitoring to ensure everything is back to normal. The issue was related to a new internal authentication mechanism that was deployed as part of further security enhancements. We understand the importance of uninterrupted service and work diligently to minimize inconvenience. Thank you for your patience.
We're happy to report that the incident has been successfully resolved. We sincerely apologize for any inconvenience this may have caused and appreciate your patience and understanding during this time. Thank you for your continued trust in our services.
We are currently investigating this issue.
We are continuing to investigate this issue.
This incident has been resolved.
A manual update on a feature switch caused a partial outage to our application. After investigating and identifying the root cause, we resolved the incident by 18:48 UTC. We are taking measures to prevent such an issue from occurring in the future. We sincerely apologize for the inconvenience.
We have noticed that grouped notifications have not been functioning properly for the past few weeks. We are investigating the issue and working towards a solution. Thank you for your patience and apologize for any inconvenience caused.
The issue has been identified, and a solution has been applied. We will continue monitoring the situation to ensure everything is back to normal.
The situation has been resolved, and things are now operating normally.
You may experience difficulties accessing Transifex. We are currently investigating the issue. Thank you for your patience and understanding!
A fix has been implemented and we are monitoring the results.
Everything is back to normal. Thank you for your patience and we apologize for the inconvenience caused.
On Thursday, October 20th, 2022, starting at 17:20:30 \(UTC\+03:00\), our users started facing difficulties using our services. We would like to highlight that during this incident no data was lost. We were running our routine maintenance updates for our systems. During this process, our CI tool misbehaved and our internal components started experiencing a partial outage. Before restoring the functionality, the issue got escalated. During this incident, the following systems were affected: * Transifex Application * Notifications * API/CLI * Transifex Live * Website Detection was almost immediate \(~5mins\). Once the incident was confirmed, we triggered our major incident management process and formed a cross-functional incident management team. There are two phases in the incident. Partial and complete unavailability. * **Partial unavailability** - _From 20/10/2022, 17:20:30 \(UTC\+03:00\) to 20/10/2022, 19:00:08 \(UTC\+03:00\)_ Our website and all of our components were up and running with one main issue. File downloads and uploads were unavailable and as a result, our users couldn’t complete these actions successfully. * **Complete unavailability** - _From 20/10/2022, 19:00:08 \(UTC\+03:00\) to 20/10/2022, 20:00:00 \(UTC\+03:00\)_ In order to address the issue, our DevOps team had to proceed with recovery. In order for this to happen, we had to cause complete unavailability so that we could replace resources affected by the incident. After resolving the incident, we had been monitoring the situation closely for the next 4 hours and confirmed that everything was back to normal. We have taken a number of immediate actions and are committed to making changes to avoid this situation in the future. Here are specific areas where we have made or will make significant changes: * We thoroughly reviewed our maintenance update process and identified parts to improve. We will update our training, and tooling, and continuously improve our standard operating procedure. * We set up safeguards to protect and isolate our core infrastructure. * We already started working on a failover infrastructure implementation. We understand that our product is mission-critical to your business, and we don’t take that responsibility lightly. To our customers and our partners, we thank you for your continued trust and partnership. We hope the details and actions outlined here to show our commitment that Transifex will continue to provide a cloud platform with scalable infrastructure and a steady cadence of enhancements.
We are experiencing a major outage in our infrastructure, our team is investigating the issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
Currently, when a new comment/issue is added to a glossary term or a new announcement is created, no notifications are sent to the relevant users. We are currently investigating the issue. Please accept our apologies for any inconvenience caused.
This incident has been resolved.
Our team is investigating the issue and working towards a solution. Thank you for your patience.
We are working on a fix. We will keep you posted on our progress.
A fix has been implemented and we are monitoring the situation.
The issue has been resolved and everything is back to normal.