51 Lokalise incidents · agosto 2022 — official updates, affected components, duration and resolution details.
Stiamo attualmente indagando su questo problema.
Questo incidente è stato risolto.
Tradotto automaticamente dall'aggiornamento ufficiale dell'incidente.
We are currently investigating this issue.
The issue has been identified and a fix is being implemented. Application and API are now operational with degraded performance.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
On February 4, 2026, Lokalise experienced a brief service interruption where both the application and API were unavailable for approximately 10 minutes. The incident began at 16:18 UTC and was fully resolved within 12 minutes as our engineering team restored service functionality. **What happened?** **The Cause:** A configuration update intended to enable new application metrics unexpectedly triggered a series of intensive, unoptimized database scans. This resulted in an immediate exhaustion of our backend processing capacity, preventing the web server from fulfilling incoming requests. **The Fix:** Our team identified the faulty configuration release and promptly initiated a rollback to the last known stable state. Once service was restored, a permanent fix was deployed to remove the problematic configuration entirely. **Timeline \(UTC\):** * **16:18:** Automated alerts reported service unavailability; investigation initiated. * **16:20:** The root cause was identified as a recent configuration deployment. * **16:27:** Rollback to a stable version was initiated. * **16:30:** Application and API functionality were fully restored. * **17:07:** System verified as stable after a final hotfix deployment. **What we are doing to prevent this in the future** * **Enhancing pre-deployment validation:** We are updating our automated review agents to flag potential performance issues in legacy code or dormant features before they are activated via configuration changes. * **Optimizing metric design:** We are transitioning to a system that pre-computes application metrics in isolated processes to ensure that monitoring activities never compete with core application resources. * **Refactoring database interactions:** Our team is reviewing and refactoring resource-intensive database queries to improve overall platform stability and response times. * **Consolidating internal alerting:** We are refining our monitoring workflows to consolidate overlapping alerts, allowing our engineers to focus even more quickly on resolution during critical events. We sincerely apologize for the disruption this incident caused to your work and your automated workflows. We recognize the trust you place in Lokalise for your daily operations, and we are committed to enhancing our processes to prevent similar occurrences. If you have any questions or require further assistance regarding this incident, please reach out to us at [**[email protected]**](mailto:[email protected]).
We are currently investigating this issue.
We are continuing to investigate this issue.
This incident has been resolved.
On January 26, 2026, Lokalise experienced a service outage and performance degradation between 15:52 UTC and 16:32 UTC. During this time, the Lokalise application was unavailable, and users of the API experienced intermittent connectivity and high latency. **What happened?** **The Cause:** The incident was triggered during a migration of our monitoring systems. A configuration mismatch caused a high volume of internal network requests to fail, which subsequently overwhelmed our internal DNS services. This prevented various parts of our infrastructure from communicating with one another, including our primary databases and application services. **The Fix:** Our engineering team identified the source of the traffic and disabled the legacy monitoring configuration. We also rotated affected infrastructure nodes and adjusted our service scaling to alleviate pressure on our databases. These actions restored normal communication between our services, bringing the platform back to full operational status. **Timeline \(UTC\):** * **15:52:** Service degradation and unavailability detected and investigation initiated. * **16:24:** Root cause identified as internal network congestion impacting service connectivity. * **16:27:** Corrective actions implemented; service begins to stabilize. * **16:32:** Full service restored and performance monitored for stability. **What we are doing to prevent this in the future** * **Enhancing scaling capabilities:** We are upgrading our internal DNS services to scale horizontally, ensuring they can handle unexpected spikes in traffic without impacting the wider platform. * **Improving resource monitoring:** We are implementing additional alerting for resource exhaustion to identify and mitigate infrastructure bottlenecks before they impact service availability. * **Refining deployment procedures:** We are updating our internal documentation and validation steps to ensure infrastructure dependencies are strictly coordinated during system migrations. We sincerely apologize for the disruption this incident caused to your workflow. We understand how critical Lokalise is to your operations and are committed to improving our system's resilience to prevent a recurrence. If you have any questions or require further assistance, please contact us at [[email protected]](mailto:[email protected]).
We are currently investigating this issue.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
We are continuing to investigate this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
**What happened** On Sep 8, 2025 at 09:59 UTC, a change introduced in our system unexpectedly missed a Feature Toggle condition, which caused a spike in the number of requests to one of our internal services. This led to degraded performance and temporary unavailability of both the Lokalise App and API. Our monitoring and alerting systems immediately triggered the alerts, allowing us to quickly identify and restore the service. **Impact** The Lokalise App was unavailable for approximately **9 minutes**, and the API experienced downtime for about **6 minutes** within that window. No customer data was lost or corrupted. **What we are doing to prevent this in the future** We are implementing stricter safeguards around feature rollout validation, improving how our services handle spikes in traffic, and adding additional error handling to reduce the risk of cascading failures. We sincerely apologize for the disruption this caused. Thank you for your patience and continued trust in Lokalise. If you have any questions, feel free to contact us at [[email protected]](mailto:[email protected]).
We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
**Summary** On July 30, 2025, Lokalise became unavailable and unstable at 11:55 UTC due to a new feature release. The new feature caused an unpredictable usage pattern with excessive memory usage and resulted in an ingress crash-looping. This was rectified and by 12:10 UTC, and Lokalise became fully operational again. **What happened?** During the update to one of our services, a configuration mismatch was introduced. The change, intended to increase the maximum allowed size, was not fully propagated across our infrastructure. This discrepancy between the service and the infrastructure layer led to excessive memory consumption, ultimately causing the service to become unstable and fully unavailable. Our team identified the problem quickly, the expected configuration was correctly propagated and by 12:10 UTC the problem was resolved. **Impact** 11:55 UTC: Lokalise's performance starts to degrade. 11:58 UTC: Lokalise is not available and an investigation is started. 12:05 UTC: Problem is identified and solution is applied. 12:10 UTC: Lokalise's performance is restored. 12:10 - 12:30 UTC: Performance is monitored and eventually the problem incident is resolved. We sincerely apologize for the disruption this caused. Thank you for your patience and continued trust in Lokalise. If you have any questions, feel free to contact us at [[email protected]](mailto:[email protected]).
We are currently investigating this issue.
We have identified the root cause. We are actively working to resolve it.
Problem resolved. We are monitoring the system.
Issue is resolved.
On July 18, 2025, Lokalise became unavailable at 12:31 UTC due to a maintenance error that disabled an encryption key needed for data access. We fully restored service by 13:32 UTC with no loss of customer data. **What happened?** During a periodic scheduled clean up procedure, an encryption key presumed to be unused was disabled. This key, in fact, was still in use by one of the auxiliary databases. With the key disabled, the application was unable to access data and this led to a disruption of service. Our team identified the problem quickly and worked closely with our cloud provider’s support team to resolve the issue. The encryption key was re-enabled and a point-in-time recovery was performed to restore the service. The main application was back online by 13:25 UTC, and all services were fully operational by 13:32 UTC. The root cause of the incident was that the encryption key was mistakingly referenced in two terraform state files. One of the state files was no longer associated with any infrastructure, the encryption key therefore was deemed unused, and scheduled for removal. **Impact** * 12:31 – 13:25 UTC: Lokalise application was unavailable. * 13:25 – 13:32 UTC: Some background services were still recovering. **What we are doing to prevent this in the future** We take reliability very seriously and have performed a thorough analysis of this incident. We are going to make a number of improvements to prevent this type of incident from happening in the future. These include: * improve procedure and automation around disabling and decommissioning of encryption keys to cross-check all data storages to ensure the key is not used anymore * improve monitoring to detect the databases going into grace period when an encryption key is disabled, but the database is still functional; this will allow us to re-enable the key without the need in point-in-time recovery and service interruption * review terraform state files for any keys and other resources referenced in multiple state files and address findings if any We sincerely apologize for the disruption this caused. Thank you for your patience and continued trust in Lokalise. If you have any questions, feel free to contact us at [[email protected]](mailto:[email protected]).
We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
We have identified the issue, and new async exports are now processing fine. We are implementing a fix to allow stuck processes to finish.
This incident has been resolved.
Async export processes that have been triggered between 07:15 and 08:15 UTC time today got stuck and didn’t complete. We will share more details in the postmortem message.
Lokalise application is experiencing issues with processing search, filtering and statistics in the UI. We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
Lokalise is experiencing issues with AI translation tasks and MT orders processing. We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently experiencing issues with UI import and bulk actions. We are investigating the issue.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
Between February 27 and March 1, 2025, our service experienced performance degradation in background tasks that impacted essential processes including file imports, synchronization with content integrations, and some bulk actions. **What happened?** A faulty release unintentionally caused our background tasks to stop working as expected. Although the issue did not appear immediately, it became noticeable the next evening. Our team promptly identified the problem, initiated an investigation, and implemented a corrective patch, ensuring full service restoration. **Impact** * **February 27th, 2025, 20:00 UTC to March 1, 2025, 12:02 UTC**: Customers experienced slower performance. * **March 1, 2025, 14:30 UTC**: Corrective patch deployed. **What we are doing to prevent this in the future** We recognize the importance of this event, and are taking further steps to ensure it does not happen again. Our key actions include: * Additional monitoring of background tasks to quickly spot issues. * Exploring options for our system to recover automatically from minor errors. * Enhancing our test coverage to help identify potential issues earlier in the release process. We sincerely apologize for the inconvenience this incident caused. Thank you for your patience and understanding as we continue to improve Lokalise’s resilience and reliability. If you have any questions or require further assistance, please contact [[email protected]](mailto:[email protected]).
Some actions like file import, bulk actions are not being processed. We are investigating this issue.
All functionality that is relying on background processes is affected. We are continuing to investigate this issue.
We have restored the functionality to unblock customers. We are continuing to investigate the root cause.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
On February 19, 2025, our service experienced performance degradation in background tasks that impacted essential processes including file imports, synchronization with content integrations, and some bulk actions. **What happened?** A faulty release unintentionally caused our background tasks to stop working as expected. Although the issue did not appear immediately, it became noticeable the next morning. Our team promptly investigated and mitigated the problem, quickly restoring full service. **Impact** * 08:54 UTC to 09:23 UTC: Customers experienced slower performance. **What we are doing to prevent this in the future** We recognize the importance of this event, and we are taking steps to ensure it does not happen again. Our key actions include: * Additional monitoring of background tasks to quickly spot issues * Exploring options for our system to recover automatically from minor errors. We sincerely apologize for the inconvenience this incident caused. Thank you for your patience and understanding as we continue to improve Lokalise’s resilience and reliability. If you have any questions or require further assistance, please contact [[email protected]](mailto:[email protected]).
We are currently investigating this issue.
We are continuing to investigate this issue.
A fix has been implemented and we are monitoring the results.
We are continuing to monitor for any further issues.
We are investigating the slowness of application and Lokalise OTA service unavailable.
We have applied the fix and monitoring for issues.
This incident has been resolved.
On February 10, 2025, our service experienced a 16-minute outage followed by 39 minutes of degraded functionality due to an issue with a system configuration. **What happened?** While improving our disaster recovery process, a misconfiguration in the system was introduced unintentionally. Initially, this did not cause issues, but when we attempted to roll back the change, it led to unexpected complications. As a result, our service platform became temporarily unavailable, requiring the reconstruction of certain system components to restore full functionality. **Impact** * **12:30 – 12:46 UTC**: Service outage. * **12:46 – 13:25 UTC**: Service degradation—APP and API were operational. Some services remained unavailable \(OTA, Workflows, Connectors, Review Center\). **What we are doing to prevent this in the future** We recognize the importance of this event, and we have taken steps to ensure it does not happen again. Our key actions include: * Improving validation and monitoring processes to identify configuration issues before deployment. * Enhancing our tools and automations for faster services restoration. * Implementing blue-green deployment techniques, or equivalent, for seamless system upgrades. We sincerely apologize for the disruption this caused and appreciate your patience as we work to make our systems more resilient. If you have any questions, please reach out to [[email protected]](mailto:[email protected]).
We are seeing issues with import and export functionality of the Lokalise app. We are investigating the issue.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
On February 5, 2025, we experienced a major service degradation affecting our Import functionality. The incident began at 11:00 UTC and during this time, customers were unable to perform imports in the UI or utilize Git integrations, which may have disrupted their workflows. Services have been fully restored by 13:26 UTC. The outage was caused by a misconfiguration during our migration to a new infrastructure system. To resolve the issue, our team rolled back to the previous system. In response to this incident, we are implementing several measures to prevent similar occurrences in the future: * We will enhance our deployment health checks to cover a previously missing set of dependencies, so that we can catch misconfiguration before newly deployed instances start to be used. * Additional monitoring and alerting systems will be established to quickly identify and address issues as they arise. * We are increasing coverage of our automated tests to ensure all the critical functionality is covered. We sincerely apologize for the inconvenience this incident caused to our customers. We understand how crucial these services are for your operations, and we appreciate your patience as we work to improve our systems and ensure reliable service in the future. Your trust is important to us, and we are committed to enhancing our processes to prevent such occurrences. Thank you for your understanding.
Several content integrations were affected including Contentful, Storyblok, Zendesk Guide, HubSpot, and ContentStack.
On December 19, 2024, we experienced a significant service degradation affecting several content integrations, including Contentful, Storyblok, Zendesk Guide, HubSpot, and ContentStack. The incident began at 22:32 UTC and lasted approximately 9 hours, primarily impacting our customers in the US and APAC regions during their business hours. The issue was first reported by one of our customers. The incident was triggered by a late release that introduced a misconfiguration on our production environment. Upon receiving the customer complaint at 01:00 UTC on December 20, our team promptly initiated a rollback of the faulty release. By 08:15 UTC, we successfully restored services to a stable state. If you had scheduled imports or exports during the affected hours, it is likely that they failed due to this incident. We understand how important these operations are for your business, and we apologize for any disruption this may have caused. To prevent similar incidents in the future, we are taking the following steps: * We will ensure consistent configurations across all environments to catch potential issues early. * We are enhancing our testing protocols to include scenarios that replicate customer-reported issues. * We will reassess our release schedule, particularly regarding significant changes, to avoid late releases before critical periods. We sincerely apologize for the inconvenience this incident caused to our customers. We appreciate your understanding and patience as we work to improve our systems and ensure reliable service in the future. Your trust is important to us, and we are committed to enhancing our processes to prevent such occurrences.