Opsgenie driftsforstyrrelser

1. 14. september 2026 kl. 16.46 UTCinvestigating

   Vi undersøger aktivt rapporter om en tjenesteforstyrrelse, der påvirker Opsgenie og Jira Service Management. Vi vil dele opdateringer her som mere information er tilgængelig.

2. 14. september 2026 kl. 17.16 UTCidentified

   Vi har identificeret den sandsynlige årsag til problemet, og vores hold arbejder ihærdigt på en afbødning. Vi vil fortsætte med at dele yderligere opdateringer her, da mere information er tilgængelig.

3. 14. september 2026 kl. 17.27 UTCmonitoring

   Spørgsmålet er nu løst, og tjenesterne fungerer normalt for alle berørte kunder. Vi vil fortsat nøje overvåge stabiliteten.

4. 14. september 2026 kl. 17.43 UTCresolved

   Den 14. september 2026, Opsgenie og Jira Service Management oplevede en afbrydelse, og ramte brugere oplevede forsinkede indberetninger anmeldelse og manglende evne til at se indberetninger i brugergrænsefladen. Spørgsmålet er nu løst, og tjenesten fungerer normalt for alle berørte kunder.

5. 18. september 2026 kl. 17.02 UTCpostmortem

   # # # Oversigt Den 14. september 2026, mellem 15: 30 og 17: 10 UTC, Atlantiske kunder, der bruger Opsgenie, Jira Service Management, og Compass oplevede forsinkelser i modtagelse af alarmmeddelelser og intermitterende fejl, når du bruger Ops funktioner. Spørgsmålet blev udløst af langvarige transaktioner i en kernetjeneste, hvilket førte til udtømning af trådpuljen og forårsagede efterfølgende anmodninger til at blive uresponderende. De langløbende transaktioner var forårsaget af en konfiguration ændring udgivet mellem september 9, 2026 og september 11, 2026 UTC. Holdet aktivt overvåget virkningen af ændringen på systemet indtil begyndelsen af september 14, 2026 UTC, og observerede ingen anomalier. Den stigende trafik i USA 's arbejdstid den 14. september 2026 forårsagede imidlertid disse uventet langvarige transaktioner. Hændelsen blev opdaget inden for et minut af vores automatiserede overvågningssystemer, og fuld service stabilisering opstod efter isolering og rulning tilbage ændringen på September 14, 2026 på 17: 10 UTC. # # IMPACT Hændelsen primært berørte Ops funktioner på tværs af Opsgenie, Jira Service Management, og Compass for kunder i den amerikanske region. Under hændelsen oplevede end-to-end flow for alarm oprettelse og meddelelse levering en gennemsnitlig forsinkelse på 22 minutter. Ingen indberetninger eller anmeldelse nyttelast blev droppet under hændelsen; alle i kø begivenheder blev behandlet og leveret med succes som tjenester inddrives. Derudover oplevede relaterede Ops API endepunkter og Web UI-strømme forhøjet latency og intermitterende fejl. Afbrydelsen begyndte kl 15: 30 UTC og blev løst ved 17: 10 UTC (samlet varighed: 1 time og 40 minutter). Engineering teams blev alarmeret straks via backup disaster- alarmering rørledninger og begyndte afbødning straks. Men afbrydelsen påvirkede interne anmeldelse og koordination strømme mellem krydsfunktionelle hold (herunder kundesupport og hændelsesmeddelelser), hvilket resulterede i forsinkelser i offentliggørelsen eksterne Statusside opdateringer. # # ROOT CAUSE Hændelsen blev udløst, når en nyligt frigivet konfiguration ændring gjort overflødige downstream service opkald på hver side belastning, forårsager en trafik spike. Den konfiguration ændring, udgivet mellem September 9, 2026, og September 11, 2026, blev aktivt overvåget for effekt indtil begyndelsen af September 14, 2026. En trafikstigning i den amerikanske arbejdstid forårsagede imidlertid uventede problemer. Den downstream service begyndte rate- begrænsende anmodninger, og en aggressiv retry strategi uden tilstrækkelig backoff holdt arbejderen tråde åbne, hvilket fører til pool udmattelse og timeouts på indgående kritisk trafik. # # # REMEDIALE AKTIONER PLAN & NEXT STEPS Hændelsen blev afhjulpet ved at deaktivere bonusflaget, der udløste den interservice rate begrænsning. Vi forstår, at pålidelig adgang til Atlantiske produkter er afgørende for dine hold. Vi prioriterer følgende tiltag for at forhindre gentagelse: - * * Forbedre service- to- Service Resiliens * * - Gennemgang og tilpasning af takstbegrænsende konfigurationer mellem interne tjenester. - Optimere retry strategier for at forhindre langtrukne transaktioner og holde virkningen isoleret til problematisk punkt. - * * Forbedre den arkitektoniske isolering * * - Forbedre arkitekturen af denne kernetjeneste til at have mere isolation mellem forskellige transaktioner, således at nedbrydninger i en transaktionstype ikke påvirker uafhængige kritiske strømme. - * * Forbedre proaktiv Alerting * * - Tilføj tidlig varsling for trådpulje mætning og forhøjede interservice-grænse svar, før de påvirker slutbrugerne. - Implementere sikkerhedskopieringsprocesser for kundesupport og hændelseskommunikationsveje Vi undskylder til kunder, hvis tjenester blev påvirket under denne hændelse; vi tager øjeblikkelige skridt til at forbedre platformens ydeevne og tilgængelighed. Tak. Atlantisk kundesupport

Automatisk oversat fra den officielle hændelsesopdatering.

1. 27. august 2026 kl. 10.26 UTCidentified

   Den 19. august 2026 identificerede vi et problem, der påvirker leveringen af Microsoft Teams meddelelser til Opsgenie standalone kunder. Baggrunden er blevet identificeret, og vores hold arbejder sammen med vores partnere for at løse dette problem. Dette spørgsmål påvirker ikke Jira Service Management Operations kunder. Vi giver næste opdatering om 24 timer.

2. 28. august 2026 kl. 10.38 UTCidentified

   Vores team fortsætter med at arbejde med vores partnere i retning af at løse Microsoft Teams anmeldelse levering problem for Opsgenie standalone kunder. Vi vil levere den næste opdatering om 72 timer eller før, hvis vi har en materialeopdatering.

3. 31. august 2026 kl. 10.29 UTCidentified

   Vores hold fortsætter med at arbejde med vores partnere, mod at løse Microsoft Teams anmeldelse levering problem for Opsgenie standalone kunder. Vi vil give den næste opdatering om 24 timer, eller før, hvis der er en betydelig opdatering.

4. 2. september 2026 kl. 10.17 UTCidentified

   Vores team fortsætter med at arbejde med vores partnere i retning af at løse Microsoft Teams anmeldelse levering problem for Opsgenie standalone kunder. Vi vil give den næste opdatering om 24 timer, eller før, hvis der er en betydelig opdatering.

5. 3. september 2026 kl. 06.06 UTCresolved

   Den 19. august identificerede vi et problem, der påvirker leveringen af Microsoft Teams meddelelser til Opsgenie standalone kunder. Den grundlæggende årsag blev tilskrevet en operationel fejl forårsaget af vores eksterne partner, og de har truffet korrigerende foranstaltninger til at genoprette tjenester. Hvis du stadig oplever problemer, skal du følge disse trin for at genoprette din integration: https: / / support.atlassian.com / opsgenie / docs / restore-the- opsgenie- microsoft-teams- v2- integration /. Hvis du har brug for yderligere hjælp, bedes du kontakte Atlantis Support via support.atlassian.com / contact.

Automatisk oversat fra den officielle hændelsesopdatering.

1. 21. juli 2026 kl. 09.48 UTCidentified

   Vi arbejder med vores underretningsudbyder for at løse problemet. Vi vil give en anden opdatering om 24 timer eller så snart mere information bliver tilgængelig.

2. 22. juli 2026 kl. 06.59 UTCidentified

   Et lille antal voice notifikationer for Jira Service Management og Opsgenie kunder ved hjælp af Indien baserede mobilnumre kan mislykkes. Kunder kan se fejlbehæftede voice call logs og ikke modtage den forventede personsøgning opkald. Vores tredje part anmeldelse udbyder arbejder aktivt hen imod at genoprette stemme anmeldelse levering og vi vil levere opdateringer, når problemet er løst. Hvis du har brug for yderligere støtte eller information, bedes du kontakte vores support team.

3. 24. juli 2026 kl. 13.07 UTCresolved

   Vores underretningsudbyder har identificeret roden til det problem, der forårsager voice call svigt til nogle Indiabaserede mobilnumre. Vi forstår den ulejlighed, dette kan forårsage, og værdsætter din tålmodighed, mens vores udbyder arbejder på at genoprette tjenesten. Hvis du har oplevet problemer med taledokumenter, bedes du kontakte vores support for hjælp.

Automatisk oversat fra den officielle hændelsesopdatering.

1. 20. oktober 2025 kl. 07.56 UTCinvestigating

   We have noticed that Atlassian Cloud services are impacted and our teams are actively investigating the same. We shall keep you informed of the progress every hour.

2. 20. oktober 2025 kl. 08.24 UTCinvestigating

   Atlassian Cloud services are impacted and we are aware that our customers might not be able to create support tickets. Our teams are actively investigating the same. We shall keep you informed of the progress every hour.

3. 20. oktober 2025 kl. 09.24 UTCinvestigating

   We are experiencing an outage due to some issue at the end of our public cloud provider. We are working closely with them to get this resolved or mitigated as quickly as possible. ETA of the same is not know at the moment. We shall continue to share updates every hour, if not sooner.

4. 20. oktober 2025 kl. 09.58 UTCidentified

   We understand that our public cloud provider has identified the cause of the issue. We are starting to see some recovery and is working towards mitigation. We appreciate your patience. We shall continue to share updates every hour, if not sooner.

5. 20. oktober 2025 kl. 11.07 UTCidentified

   We continue to work with our public cloud provider towards mitigating the issue at the earliest. We appreciate your patience. We shall continue to share updates every hour, if not sooner.

6. 20. oktober 2025 kl. 12.06 UTCidentified

   Atlassian team is actively engaged and continues to work with our public cloud provider to mitigate this issue at the earliest. We are starting to see partial operations succeed. We appreciate your patience. We shall continue to share updates every hour, if not sooner.

7. 20. oktober 2025 kl. 13.58 UTCidentified

   Our public cloud provider is working to mitigate this issue quickly. We are seeing some early positive indicators and are continuing to monitor. We appreciate your patience and will continue to provide updates every hour or sooner.

8. 20. oktober 2025 kl. 14.02 UTCidentified

   We understand your pain and mitigating or fixing this issue is of utmost importance. Our public cloud provider is actively working to mitigate this issue on priority. We have been seeing partial operational success. We appreciate your patience and will continue to provide updates every hour or sooner.

9. 20. oktober 2025 kl. 16.36 UTCidentified

   Update - Thank you for your continued patience. We understand the impact this issue is having on your operations and want to assure you that resolving this matter is our highest priority. Our public cloud provider is still actively working to mitigate this issue with urgency and while we do not have a definitive ETA at this time, we remain committed to full resolution and deeply appreciate your patience as we work through this situation. We will be providing hourly updates on this issue.

10. 20. oktober 2025 kl. 17.27 UTCidentified

    Update - We understand the impact this issue is having on your operations and want to assure you that resolving this matter is our highest priority. Our public cloud provider is actively working to mitigate this issue with urgency. While we do not have a definitive ETA at this time, we remain committed to full resolution and deeply appreciate your patience as we work through this situation. We will continue to provide updates every hour or sooner as new information becomes available.

11. 20. oktober 2025 kl. 17.43 UTCidentified

    We are currently aware of an ongoing incident impacting Atlassian Cloud services due to an outage with our public cloud provider, AWS. We understand the impact this issue is having on your operations and want to assure you that resolving this matter is our highest priority and we are closely monitoring the health of AWS services. While we do not have a definitive ETA at this time, we remain committed to full resolution and deeply appreciate your patience as we work through this situation. We will continue to provide updates every hour or sooner as new information becomes available.

12. 20. oktober 2025 kl. 18.38 UTCmonitoring

    There have been no changes since our last update. We will provide our next updated by 9:00PM UTC or sooner as new information becomes available. We are currently aware of an ongoing incident impacting Atlassian Cloud services due to an outage with our public cloud provider, AWS. We understand the impact this issue is having on your operations and want to assure you that resolving this matter is our highest priority and we are closely monitoring the health of AWS services. While we do not have a definitive ETA at this time, we remain committed to full resolution and deeply appreciate your patience as we work through this situation.

13. 20. oktober 2025 kl. 21.26 UTCmonitoring

    Monitoring - We've started seeing continued product experience improvement. While we still have a backlog of event processing, we are seeing improvements in systems operational capabilities across all products. We estimate a significant improvement with the next few hours and will continue to monitor the health of AWS services and the effects on Atlassian customers. We appreciate your continued patience and remain committed to full resolution as we work through this situation. We will post our next update in two hours.

14. 20. oktober 2025 kl. 22.58 UTCmonitoring

    Our teams are continuing to monitor the recovery of systems across Atlassian products. This update is to inform that the Atlassian Support portal is fully operational at this time for customers that wish to contact support.

15. 21. oktober 2025 kl. 00.03 UTCmonitoring

    Our team is now seeing recovery across all impacted Atlassian products. We are continuing to monitor for individual products that may still be processing backlogged items now that services are restored. The Atlassian Support portal is currently still displaying a message directing customers to our temporary support channel. Please note that our support portal is currently fully functional for those attempting to raise requests. We are continuing to look into this alert to remove this message. We will provide further update on our recovery status in one hour.

16. 21. oktober 2025 kl. 01.07 UTCmonitoring

    We continue to see recovery progressing across all impacted products as backlogged items continue to be processed. The Atlassian Support portal is currently displaying a message directing customers to our temporary support channel. Please note that our support portal is currently fully functional for those attempting to raise requests. We are continuing to look into this alert to remove this message. We will provide further update on our recovery status in two hours.

17. 21. oktober 2025 kl. 03.24 UTCmonitoring

    The issue relating to the Atlassian Support portal displaying a message to customers to use our temporary support channel has now been resolved. The Atlassian Support portal is fully functional for any ongoing support issues. With regards to other Atlassian products, we continue to see recovery continuing across all impacted products and our teams are continuing to monitor as the recovery continues. We will provide further update on our recovery status within two hours.

18. 21. oktober 2025 kl. 05.24 UTCresolved

    Our team is now able to see full recovery across the vast majority of Atlassian products. We are aware of some ongoing issues with specific components such as migrations and JSM virtual service agents, and our team is continuing to investigate with urgency. We apologise for the inconvenience that this incident has caused and we will provide further information when the Post Incident Investigation has been completed.

19. 24. oktober 2025 kl. 17.54 UTCpostmortem

    ### Postmortem publish date: Nov 19th, 2025 ### Summary All dates and times below are in UTC unless stated otherwise. Customers utilizing Atlassian products experienced elevated error rates and degraded performance between Oct 20, 2025 06:48 and Oct 21, 2025 04:05. The service disruptions were triggered due to an [AWS DynamoDB outage](https://aws.amazon.com/message/101925/#:~:text=1%3A50%20PM.-,DynamoDB,-Between%2011%3A48) and further affected by subsequent failures in [AWS EC2](https://aws.amazon.com/message/101925/#:~:text=service%20disruption%20event.-,Amazon%20EC2,-Between%2011%3A48) and [AWS Network Load Balancer](https://aws.amazon.com/message/101925/#:~:text=service%20disruption%20event.-,Amazon%20EC2,-Between%2011%3A48) within the us-east-1 region. The incident started at Oct 20, 2025 06:48 and was detected within six minutes by our automated monitoring systems. Our teams worked to restore all core services by Oct 21, 2025 04:05. Final cleanup of backlogged processes and minor issues was completed on Oct 22, 2025. We recognize the critical role our products play in your daily operations, and we offer our sincere apologies for any impact this incident had on your teams. We are taking immediate steps to enhance the reliability and performance of our services, so that you continue to receive the standard of service you have come to trust. ### IMPACT Before examining product-level impacts, it's helpful to understand Atlassian's service topology and internal dependencies. Products such as Jira and Confluence are deployed across multiple AWS regions. The data for each tenant is stored and processed exclusively within its designated host region. This design is intentional and represents the desired operational state, as it limits the impact of any regional outage strictly to tenants in-region, in this case us-east-1. While in-scope application data is pinned to the region selected by the customer, there are times when systems need to call other internal services that may be based in a different region. If a problem occurs in the main region where these services operate, systems are designed to automatically fail over to a backup region, usually within three minutes. However, if unexpected issues arise during this failover, it can take longer to restore services. In rare cases, this could affect customers in more than one region. It’s important to note that all in-scope application data for supported products is pinned according to a customer’s chosen region. **Jira** Between Oct 20, 2025 06:48 and Oct 20, 2025 20:00, customers with tenants hosted in the us-east-1 region experienced increased error rates when accessing core entities such as Issues, Boards, and Backlogs. This disruption was caused by AWS's inability to allocate AWS EC2 instances and elevated errors in AWS Network Load Balancer \(NLB\). During this window, users may also have observed intermittent timeouts, slow page loads, and failures when performing operations like creating or updating issues, loading board views, and executing workflow transitions. Between Oct 20, 2025 08:36 and Oct 20, 2025 09:23, customers across all regions experienced elevated failure rates when attempting to load Jira pages. This disruption was caused by the regional frontend service entering an unhealthy state during this specific time interval. Normally, the frontend service connects to the primary AWS DynamoDB instance located in the us-east-1 to retrieve the most recent configuration data necessary for proper operation. Additionally, the service is designed with a fallback mechanism that references static configuration data in the event that the primary database becomes inaccessible. Unfortunately, a latent bug existed in the local fallback path. When the frontend service nodes restarted, they were unable to load critical operational configuration data from primary or fallback sources, leading to the observed failures experienced by customers. Between Oct 20, 2025 06:48 and Oct 21, 2025 06:30, customers experienced significant delays and missing Jira in-app notifications across all regions. The notification ingestion service, which is hosted exclusively in us-east-1, exhibited an increased failure rate when processing notification messages due to AWS EC2 and NLB issues. This issue resulted in notifications being delayed - and in some cases, not delivered at all - to users worldwide. **Jira Service Management \(JSM\)** JSM was impacted similarly to Jira above, with the same timeframes and for the same reasons. Between Oct 20, 2025 08:36 and Oct 20, 2025 09:23, customers across all regions experienced significantly elevated failure rates when attempting to load JSM pages. This affected all JSM experiences including the Help Centre, Portal, Queues, Work Items, Operations, and Alerts. **Confluence** Between Oct 20, 2025 06:48 and Oct 21, 2025 02:45, customers using Confluence in the us-east-1 region experienced elevated failure rates when performing common operations such as editing pages or adding comments. The primary cause of this service degradation was the system's inability to auto-scale due to AWS EC2 issues to manage peak traffic load effectively. Though the AWS outage ended at Oct 20, 21:09, a subset of customers continued to experience failures as some Confluence web server nodes across multiple clusters remained in an unhealthy state. This was ultimately mitigated by recycling the affected nodes. To protect our systems while AWS recovered, we made a deliberate decision to enable node termination protection. This action successfully preserved our server capacity but, as a trade-off, it extended the time required for a full recovery once AWS services were restored. **Automation** Between Oct 20, 2025 06:55 and Oct 20, 2025 23:59, automation customers whose rules are processed in us-east-1 experienced delays of up to 23 hours in rule execution. During this window, some events triggering rule executions were processed out of order because they arrived later during backlog processing. This caused potential inconsistencies in workflow executions, as rules were run in the order events were received, not when the action causing the event occurred. Additionally, some rule actions failed because they depend on first-party and third-party systems, which were also affected by the AWS outage. Customers can see most of these failures in their audit logs; however, a few updates were not logged due to the nature of the outage. By Oct 21, 2025 5:30, the backlog of rule runs in us-east-1 was cleared. Although most of these delayed rules were successfully handled, there were some additional replays of events to ensure completeness. Our investigation confirmed that a few events may never have triggered their associated rules due to the outage. Between Oct 20, 2025 06:55 and Oct 20, 2025 11:20, all non-us-east-1 regional automation services experienced delays of up to 4 hours in rule execution. This was caused by an upstream service that was unable to deliver events as expected. The delivery service encountered a failure due to a cross-region dependency call to a service hosted in the us-east-1 region. Because of this dependency issue, the delivery service was unable to successfully deliver events throughout this time frame, resulting in customer-defined rules not being executed in a timely manner. **Bitbucket and Pipelines** Between Oct 20, 2025 06:48 and Oct 20, 2025 09:33, Bitbucket experienced intermittent unavailability across core services. During this period, users faced increased error rates and latency when signing in, navigating repositories, and performing essential actions such as creating, updating, or approving pull requests. The primary cause was an AWS DynamoDB outage that impacted downstream services. Between Oct 20, 2025 06:48 and Oct 20, 2025 22:46, numerous Bitbucket Pipeline steps failed to start, stalled mid-execution, or experienced significant queueing delays. Impact varied, with partial recoveries followed by degradation as downstream components re-synchronized. The primary cause was an AWS DynamoDB outage, compounded by instability in AWS EC2 instance availability and AWS Network Load Balancers. Furthermore, Bitbucket Pipelines continued to experience a low but persistent rate of step timeouts and scheduling errors due to AWS bare-metal capacity shortages in select availability zones. Atlassian coordinated with AWS to provision additional bare-metal hosts and addressed a significant backlog of pending pods, successfully restoring services by 01:30 on Oct 21, 2025. **Trello** Between Oct 20, 2025 06:48 and Oct 20, 2025 15:25, users of Trello experienced widespread service degradation and intermittent failures due to upstream AWS issues affecting multiple components, including AWS DynamoDB and subsequent AWS EC2 capacity constraints. During this period, customers reported elevated error rates when loading boards, opening cards, adding comments or attachments. **Login** Between Oct 20, 2025 06:48 and Oct 20, 2025 09:30, a small subset of users experienced failures when attempting to initiate new login sessions using SAML tokens. This resulted in an inability for those users to access Atlassian products during that time period. However, users who already had valid active sessions were not affected by this issue and continued to have uninterrupted access. The issue impacted all regions globally because regional identity services relied on a write replica located in the us-east-1 region to synchronize profile data. When the primary region became unavailable, the failover to a secondary database in another region failed, which delayed recovery. This failover defect has since been addressed. **Statuspage** Between Oct 20, 2025 06:48 and Oct 20, 2025 09:30, Statuspage customers who were not already logged in to the management portal were unable to log in to create or update incident statuses. This impact was restricted only to users who were not already logged in at the time. The root cause was the same as described in the Login section above, and it was resolved by the same remediation steps. ### REMEDIAL ACTION PLAN & NEXT STEPS We have completed the following critical actions designed to help prevent cross-region impact from similar issues: * Resolved the code defect in the fallback option to ensure that Jira Frontend Services in other regions remain unaffected during a region-wide outage. * Fixed the issue that prevented timely failover of the identity service which impacted new login sessions. * Resolved the code defect so that delivery services in unaffected regions remain operational during region-wide outages. Additionally, we are prioritizing the following improvement actions: * Implement mitigation strategies to strengthen resilience against region-wide outages in the notification ingestion service. Although disruptions to our cloud services are sometimes unavoidable during outages of the underlying cloud provider, we continuously evaluate and improve test coverage to strengthen resilience of our cloud services against these issues. We recognize the critical importance of our products to your daily operations and overall productivity, and we extend our sincere apologies for any disruptions this incident may have caused your teams. If you were impacted and require additional details for internal post-incident reviews, please reach out to your Atlassian support representative with affected timeframes and tenant identifiers so we can correlate logs and provide guidance. Thanks, Atlassian Customer Support

1. 30. september 2025 kl. 14.07 UTCinvestigating

   We are observing delays in Opsgenie, JSM ops and Compass ops notification flows. No alert has been lost and our team is actively working on it to mitigate the delays. We'll keep you posted with further updates.

2. 30. september 2025 kl. 14.09 UTCidentified

   We continue to work on resolving the notification delays for Jira Service Management, Opsgenie, and Compass. We have identified the root cause and expect recovery shortly.

3. 30. september 2025 kl. 14.22 UTCmonitoring

   We have identified the root cause of the notification delay for Opsgenie, JSM Ops and Compass Ops and have mitigated the problem. We are now monitoring closely.

4. 30. september 2025 kl. 14.32 UTCresolved

   Between 13:10 UTC to 14:00 UTC, we experienced notification delay for Jira Service Management, Opsgenie, and Compass in only EU-region. The issue has been resolved and the service is operating normally.

1. 10. september 2025 kl. 15.23 UTCinvestigating

   We identified degraded performance in web experiences and Alert Rest API for some Jira Service Management and Opsgenie Cloud customers in the US Region. The team has taken actions to mitigate the issue and minimize the impact on search functionality.

2. 10. september 2025 kl. 15.26 UTCidentified

   We continue to work on resolving the incident for Jira Service Management and Opsgenie. We have identified the root cause and taken actions to mitigate the issue and minimize the impact on search functionality.

3. 10. september 2025 kl. 17.56 UTCidentified

   We are continuing to work on a fix for this issue.

4. 10. september 2025 kl. 20.10 UTCidentified

   Our team is still actively working to resolve the issue and making progress toward a resolution. Thank you for your patience.

5. 11. september 2025 kl. 00.11 UTCidentified

   Our team has been able to identify the root cause of these performance issues and have put a mitigation into place. We are now continuing to see recovery of Jira Service Management operations and Opsgenie services. Existing alerts are continuing to be processed at this time. We will provide further update as soon as possible.

6. 11. september 2025 kl. 01.10 UTCidentified

   We are continuing to see recovery of Jira Service Management operations and Opsgenie services. There are a large number of missed alerts to be processed due to the prior service issue and our team is actively investigating methods to help expedite this processing. Alerts should trigger in the meantime but the content of these alerts may not be visible at this time. Our teams are continuing to investigate with urgency. We will provide further updates within the hour.

7. 11. september 2025 kl. 02.10 UTCidentified

   While new alerts should now correctly notify users, the content of those alerts for impacted customers will remain empty until missed alerts are fully processed. Our team is continuing to investigate with urgency potential infrastructure options to expedite this process. We will provide further updates within the hour.

8. 11. september 2025 kl. 03.10 UTCidentified

   New alerts should still continue to notify users as they are generated, however the alert content will still be missing. Our teams are still actively engaging with infrastructure teams to try and expedite historical processing of messages to fully restore services. We will provide further updates within the hour.

9. 11. september 2025 kl. 03.49 UTCidentified

   We have received reports that some customers are experiencing further issues relating to loading pages within Opsgenie and Jira Service Management which may be related to the infrastructure efforts underway to restore services to these products. Please be aware we also have teams actively investigating these issues to ensure as fast a resolution as possible. We want to reiterate that your alerts are still sending notifications correctly, and there is no data loss occurring. However, the underlying issue is affecting the propagation of alert details to notifications, and to the Web and Mobile UI. Viewing schedules may also be impacted at this time while we continue to process the tasks required for recovery. We will provide further updates as soon as possible.

10. 11. september 2025 kl. 04.50 UTCidentified

    Our team is continuing to investigate with the highest level of urgency in order to restore Jira Service Management and Opsgenie services. At this time we are prioritising infrastructure to try and restore new and active alerts to be populated with their alert content as soon as possible, while continuing to deliver existing messages as they are ready. We will continue to provide updates as we progress, and will ensure we have an update posted within the hour.

11. 11. september 2025 kl. 05.51 UTCidentified

    Our infrastructure teams are working diligently to try and ensure all services are restored to Jira Service Management and Opsgenie as soon as possible. New alerts are continuing to notify users, however we are still prioritising infrastructure to restore the content inside these alerts for Web and Mobile UIs. We will continue providing updates when available and will ensure we have further update within the hour.

12. 11. september 2025 kl. 06.51 UTCidentified

    Our teams priority remains to restore full functionality to new alerts to users from Jira Service Management and Opsgenie. Escalation teams are fully engaged on this incident to try and ensure we can get to a resolution as soon as possible. At this time these alerts are still successfully notifying users, but without the intended content within the alerts. We will continue providing specific updates when available and will ensure we have further update within the hour.

13. 11. september 2025 kl. 07.50 UTCidentified

    All concerned teams are engaged and are working to restore full functionality to new alerts to users from Jira Service Management and Opsgenie. Escalation teams are also engaged on this incident to try and ensure we can get to a resolution as soon as possible. Currently, the alerts continue to successfully notify users, but without the intended content within the alerts. We will continue providing specific updates when available and will ensure we have further update within the hour.

14. 11. september 2025 kl. 08.53 UTCidentified

    Atlassian teams remain engaged with priority to restore full functionality to new alerts to users from Jira Service Management, Opsgenie and Compass. Escalation teams are also engaged on this incident to ensure we can get to a resolution as soon as possible. At this point, the alerts continue to successfully notify users, except that the responder name may be missing. We will continue providing specific updates when available and will ensure we have further update within the hour.

15. 11. september 2025 kl. 09.51 UTCidentified

    Atlassian teams continue to remain engaged with priority to restore full functionality to new alerts to users from Jira Service Management, Opsgenie and Compass. Escalation teams also continue to remain engaged on this incident to ensure our focus is not diluted from a resolution as soon as possible. Currently, the alerts continue to successfully notify users, except that the responder name may be missing. We will continue providing specific updates when available and will ensure we have further update within the hour.

16. 11. september 2025 kl. 10.59 UTCidentified

    Our teams have are identified some potential resolution steps for this issue and continue to test the same on priority to be able to restore full functionality to users from Jira Service Management, Opsgenie and Compass. Currently, the alerts continue to successfully notify users, except that the responder name may be missing. We will continue providing specific updates when available and will ensure we have further update within the hour.

17. 11. september 2025 kl. 11.53 UTCidentified

    Atlassian teams continue to test the identified potential resolution steps for this issue. Our goal is to restore full functionality to users from Jira Service Management, Opsgenie and Compass. Currently, the alerts continue to successfully notify users, except that the responder name may be missing. We will continue providing specific updates when available and will ensure we have further update within the hour.

18. 11. september 2025 kl. 13.02 UTCidentified

    We are progressing well with the testing of identified potential mitigation steps for this issue; and we are seeing positive early results. We will update with an ETA and further information in the next update. Alerts continue to successfully notify users, in some cases the responder name may be missing. Our goal continues to restore full functionality to users from Jira Service Management, Opsgenie and Compass. We will continue providing specific updates when available and will ensure we have further update within the hour.

19. 11. september 2025 kl. 14.00 UTCidentified

    We have progressed further in preparing our system with a probable fix. Initial testing has been successful. We are in the process of completing further testing and finalising the fix. We expect the fix to be completed in next few hours. Alerts continue to successfully notify users, in some cases the responder name may be missing. Our goal continues to restore full functionality to users from Jira Service Management, Opsgenie and Compass. We will continue providing specific updates when available and will ensure we have further update within the hour.

20. 11. september 2025 kl. 15.05 UTCmonitoring

    We have deployed the fix and all operations are back to normal. We will continue to monitor the operations. We will continue providing specific updates when available.

21. 11. september 2025 kl. 18.45 UTCresolved

    Between September 10, 2025, 3:19 PM UTC and September 11, 2025, 2:44 PM UTC, there was degraded performance in web experiences and the REST APIs for some Jira Service Management, Opsgenie, and Compass Cloud customers in the US region. We have deployed a fix to mitigate the issue and have verified that the services have recovered. The issue has been resolved and the service is operating normally.

22. 24. september 2025 kl. 06.24 UTCpostmortem

    ### **SUMMARY** Between September 10, 2025, at 13:56 UTC and September 11, 2025, at 14:40 UTC, Jira Service Management, Compass Cloud Operations, and Opsgenie customers in the U.S. region experienced degraded performance across web and mobile applications, as well as Alert REST APIs. Certain customers experienced difficulties with alert searches and page loading, particularly on alert-related pages. This incident was initiated by an EBS volume upgrade conducted within the cloud-based managed ElasticSearch clusters. Our automated monitoring systems identified the incident within minutes, and it was resolved after 24 hours and 43 minutes by manually implementing vertical and horizontal scaling measures. ### **IMPACT** Between September 10, 2025, 13:56 UTC and September 11, 2025, 14:40 UTC, some customers in the U.S. region experienced degraded performance in Jira Service Management, Opsgenie, and Compass Cloud. During this time, web and mobile applications, as well as the Alert REST APIs, were impacted. Some customers may have seen issues with alert searches and slow loading of alert pages. During the incident, 22% of customers in the US region experienced failures in web API functionality, 8.6% encountered failures with the REST Alert API, and 1.3% experienced delays in notification delivery. Importantly, there was no loss of data during this event. ### **ROOT CAUSE** The degradation was caused by an EBS volume upgrade in the cloud-based managed ElasticSearch clusters, which necessitated a Blue/Green deployment strategy. One of the ElasticSearch nodes approached its shard size threshold, prompting the upgrade and subsequent deployment. This deployment resulted in elevated latency, increased 4xx HTTP response codes, and timeouts affecting both search and indexing operations. Recovery time exceeded expectations due to the prolonged blue/green deployment. After completion, the ElasticSearch cluster remained unhealthy and did not return to its normal state. Additionally, the switchover to the backup region failed because it was configured similarly in size and setup to the primary cluster. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** We understand that outages can affect your productivity. We are prioritizing the following improvement actions to help prevent similar incidents in the future: * Increase ElasticSearch cluster capacity and optimize settings to handle peak search loads efficiently. * Strengthen our high availability \(HA\) and failover architecture to ensure rapid and reliable recovery in the event of primary region failures. * Optimize cluster upgrade and shard rebalancing strategies to prevent similar issues. We apologize to customers whose services were impacted during this incident. We are taking steps to improve the platform’s performance and availability.   Thanks, Atlassian Customer Support

1. 19. juni 2025 kl. 15.21 UTCinvestigating

   We are investigating degraded performance issues in Jira Service Management, Opsgenie, and Compass Cloud customers. We will provide more details within the next hour.

2. 19. juni 2025 kl. 15.42 UTCmonitoring

   A fix has been implemented and we are monitoring the results.

3. 19. juni 2025 kl. 16.37 UTCresolved

   This incident has been resolved.

1. 17. juni 2025 kl. 12.37 UTCinvestigating

   We are investigating cases of degraded performance for some Jira Service Management and Opsgenie Cloud customers. We will provide more details within the next hour.

2. 17. juni 2025 kl. 12.39 UTCmonitoring

   We have identified the root cause of the performance degraded service and have mitigated the problem. We are now monitoring closely.

3. 17. juni 2025 kl. 12.47 UTCresolved

   This incident has been resolved, and Jira Service Management and Opsgenie are back in operational mode.

1. 10. maj 2025 kl. 07.08 UTCidentified

   We identified degraded performance at alert search functionality for some Jira Service Management and Opsgenie Cloud customers due to the infrastructure issue from cloud provider. No impact has been observed at alert critical flows like notification. The team has taken actions to mitigate the issue and minimize the impact to search functionality

2. 10. maj 2025 kl. 09.49 UTCmonitoring

   The problem is mitigated, and we are now monitoring closely.

3. 10. maj 2025 kl. 10.51 UTCresolved

   All search functionality is operational without any latency. Thank you for your patience.

1. 4. april 2025 kl. 10.05 UTCinvestigating

   We are investigating cases of degraded performance for Alert Schedules experiencing timeouts and slowness for Opsgenie Cloud customers. Requests have been taking more than 30s and some have been timing out. We will provide more details within the next hour.

2. 4. april 2025 kl. 10.52 UTCresolved

   This incident has been resolved.

1. 5. marts 2025 kl. 06.34 UTCinvestigating

   We are aware of an issue where some API calls configured to use api.opsgenie.com/eu are intermittently returning a 404 error with the message 'no Route matched with those values'. API calls to api.eu.opsgenie.com and api.opsgenie.com (without /eu) are not affected. Our team is investigating this issue.

2. 5. marts 2025 kl. 07.24 UTCresolved

   Issues where some API calls configured to use api.opsgenie.com/eu were intermittently returning a 404 error with the message 'no Route matched with those values' should now be resolved. API calls to api.eu.opsgenie.com and api.opsgenie.com (without /eu) were not affected at this time.

1. 14. januar 2025 kl. 15.48 UTCinvestigating

   Our engineering team is actively investigating this incident and working to bring the Opsgenie service back up as quickly as possible. Users affected by this incident may notice that Services functionality is slow or completely unavailable for the web page We will update this page as we have additional information.

2. 14. januar 2025 kl. 15.55 UTCidentified

   Our team has identified the issue with the Services page in Opsgenie and is working to fix it.

3. 14. januar 2025 kl. 16.47 UTCresolved

   Our engineers have been closely monitoring the platform and are declaring this incident resolved. Thank you for your patience.

1. 16. december 2024 kl. 17.18 UTCresolved

   Our team has identified the issue in Schedule API between 15:30 UTC and 17:00 UTC. We saw performance degradation and 5XX errors in response. Faulty deployment has been reverted quickly in the USA region and rapid recovery is seen. We are monitoring the system for a full recovery right now. The Schedule API is up and running again without any data loss.

1. 6. november 2024 kl. 19.47 UTCinvestigating

   We've noticed that Opsgenie Web UI is responding slowly or unavailable in US region. Our engineering team is actively investigating this incident and working to bring Opsgenie back up to speed as quickly as possible. We'll keep you posted with further updates on this page.

2. 6. november 2024 kl. 19.49 UTCmonitoring

   Our engineering team has implemented fixes. We will continue to monitor all systems. Thank you for your patience.

3. 6. november 2024 kl. 19.59 UTCresolved

   Our engineers have been closely monitoring the platform and are declaring this incident resolved. Thank you for your patience.

1. 24. september 2024 kl. 19.55 UTCinvestigating

   We've been notified that a number of OEC clients have been failing to create Jira tickets. We're currently investigating the issue.

2. 24. september 2024 kl. 19.56 UTCinvestigating

   We are continuing to investigate this issue.

3. 24. september 2024 kl. 20.22 UTCidentified

   We've identified a recent change that has broken some endpoint routing configurations and caused OEC endpoint requests to be directed to wrong service. We're reverting that change on production at the moment.

4. 24. september 2024 kl. 20.49 UTCmonitoring

   We reverted the faulty routing configuration change and started getting traffic on OEC endpoints

5. 24. september 2024 kl. 20.54 UTCresolved

   We've verified that OEC endpoints are back online.

1. 12. september 2024 kl. 02.08 UTCinvestigating

   Users attempting to sign up are encountering reCaptcha errors that are preventing a successful signup.

2. 12. september 2024 kl. 02.40 UTCmonitoring

   We have identified the root cause and the issue appears to be resolved.

3. 12. september 2024 kl. 03.28 UTCresolved

   This issue has been resolved.

1. 28. august 2024 kl. 15.01 UTCidentified

   Our team has identified the issue with Opsgenie Web Application / Edit Rotation feature in US, EU and Sydney regions and is working to fix it. Check back soon for another update! Our team is working hard to get the feature up and running again.

2. 28. august 2024 kl. 15.02 UTCidentified

   We are continuing to work on a fix for this issue.

3. 28. august 2024 kl. 15.17 UTCresolved

   Our engineers have been closely monitoring the platform and are declaring this incident resolved. Thank you for your patience.

1. 3. juli 2024 kl. 20.51 UTCinvestigating

   We are investigating an issue with that is impacting Atlassian, Atlassian Partners, Atlassian Support, Confluence, Jira Work Management, Jira Service Management, Jira, Opsgenie, Atlassian Developer, Atlassian (deprecated), Trello, Atlassian Bitbucket, Guard, Jira Align, Jira Product Discovery, Atlas, Atlassian Analytics, and Rovo Cloud customers. We will provide more details within the next hour.

2. 3. juli 2024 kl. 21.45 UTCmonitoring

   We have mitigated the problem and continue looking into the root cause. The outage was between 8:08pm 03/07 UTC - 08:31pm 03/07 UTC We are now monitoring closely.

3. 4. juli 2024 kl. 00.36 UTCresolved

   Between 03-07-2024 20:08 UTC to 03-07-2024 20:31 UTC, we experienced downtime for Opsgenie. The issue has been resolved and the service is operating normally.

1. 21. juni 2024 kl. 00.17 UTCinvestigating

   We are investigating an intermittent issue with accessing Atlassian Cloud services that is impacting some Atlassian Cloud customers. We will provide more details once we identify the root cause.

2. 21. juni 2024 kl. 01.31 UTCmonitoring

   We have identified the root cause of the intermittent errors and have mitigated the problem. We are now monitoring closely.

3. 21. juni 2024 kl. 02.47 UTCresolved

   Between 2024-06-20 22:04 UTC to 2024-06-20 22:28 UTC, we experienced intermittent issue for users to access the services for some Atlassian Cloud customers. The issue has been resolved and the service is operating normally.

1. 3. juni 2024 kl. 23.11 UTCidentified

   We are investigating an issue with error responses for some Cloud customers across multiple products. We have identified the root cause and expect recovery shortly.

2. 4. juni 2024 kl. 00.32 UTCresolved

   Between 22:18 UTC to 22:56 UTC, we experienced errors for multiple Cloud products. The issue has been resolved and the service is operating normally.

3. 11. juni 2024 kl. 00.56 UTCpostmortem

   ### Summary On June 3rd, between 09:43pm and 10:58 pm UTC, Atlassian customers using multiple product\(s\) were unable to access their services. The event was triggered by a change to the infrastructure API Gateway, which is responsible for routing the traffic to the correct application backends. The incident was detected by the automated monitoring system within five minutes and mitigated by correcting a faulty release feature flag, which put Atlassian systems into a known good state. The first communications were published on the Statuspage at 11:11pm UTC. The total time to resolution was about 75 minutes. ### **IMPACT** The overall impact was between 09:43pm and 10:17pm UTC, with the system initially in a degraded state, followed by a total outage between 10:17pm and 10:58pm UTC. _The Incident caused service disruption to customers in all regions and affected the following products:_ * Jira Software * Jira Service Management * Jira Work Management * Jira Product Discovery * Jira Align * Confluence * Trello * Bitbucket * Opsgenie * Compass ### **ROOT CAUSE** A policy used in the infrastructure API gateway was being updated in production via a feature flag. The combination of an erroneous value entered in a feature flag, and a bug in the code resulted in the API Gateway not processing any traffic. This created a total outage, where all users started receiving 5XX errors for most Atlassian products. Once the problem was identified and the feature flag updated to the correct values, all services started seeing recovery immediately. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** We know that outages impact your productivity. While we have several testing and preventative processes in place, this specific issue wasn’t identified because the change did not go through our regular release process and instead was incorrectly applied through a feature flag. We are prioritizing the following improvement actions to avoid repeating this type of incident: * Prevent high-risk feature flags from being used in production * Improve the policy changes testing * Enforcing longer soak time for policy changes * Any feature flags should go through progressive rollouts to minimize broad impact * Review the infrastructure feature flags to ensure they all have appropriate defaults * Improve our processes and internal tooling to provide faster communications to our customers We apologize to customers whose services were affected by this incident and are taking immediate steps to address the above gaps. Thanks, Atlassian Customer Support