Störungsverlauf von Rainforest Qa

1. 24. Juni 2026 um 06:22 UTCidentified

   Wir haben festgestellt und Probleme mit unseren Windows 11 VMs, die dazu führen, dass sie nicht booten und funktionieren. Der Fix wurde identifiziert und wir arbeiten daran, ihn einzuführen, aber dies kann einige Stunden dauern. In der Zwischenzeit werden Runs, die diese VM-Typen verwenden, verzögert oder können Fehler machen. Wir arbeiten daran, die Situation zu lösen.

2. 24. Juni 2026 um 10:29 UTCmonitoring

   Wir haben das Problem behoben, das unsere VMs betrifft, und die Läufe scheinen jetzt normal voranzukommen. Es kann immer noch einige Verzögerungen geben, da wir einen kleinen Rückstand an ausstehenden Durchläufen durcharbeiten, und wir werden die Situation weiterhin beobachten.

3. 24. Juni 2026 um 13:55 UTCresolved

   Dieser Vorfall wurde behoben.

Automatisch aus der offiziellen Störungsmeldung übersetzt.

1. 18. November 2025 um 11:51 UTCidentified

   We are currently experiencing an issue with one of our providers that is causing the application to be intermittently available.

2. 18. November 2025 um 12:25 UTCidentified

   Our upstream provider is aware of the issue and is continuing to investigate.

3. 18. November 2025 um 14:05 UTCidentified

   Our upstream provider has identified the issue and is currently working on restoring service

4. 18. November 2025 um 14:59 UTCmonitoring

   Our upstream provider has implemented a fix and services are returning to normal operation. We're continuing to monitor the situation to ensure everything has recovered.

5. 18. November 2025 um 16:40 UTCmonitoring

   Our provider is still reporting intermittent errors which may have an impact on some clients being able to access Rainforest services. We are continuing to monitor the situation.

6. 18. November 2025 um 18:19 UTCresolved

   This incident has been resolved.

1. 8. September 2025 um 17:02 UTCinvestigating

   We are investigating an issue with our background job processing which is causing tasks to be delayed. This will impact starting new tests, execution of current tests, and many other pieces of functionality within the application. We are currently working to resolve this.

2. 8. September 2025 um 17:16 UTCmonitoring

   We have implemented a fix and background job processing appears to be returning to normal. We will continue to investigate to ensure everything is working as it should be. There is a backlog of tests awaiting execution, so test results may continue to be delayed as we work our way through executing them all.

3. 8. September 2025 um 17:39 UTCresolved

   All systems are back to normal.

1. 28. Juli 2025 um 13:01 UTCinvestigating

   We're currently investigating an issue affecting user logins. Some users may be having trouble logging in.

2. 28. Juli 2025 um 13:16 UTCresolved

   We've addressed the issue and logins should be functional for all users.

1. 12. Juni 2025 um 18:50 UTCinvestigating

   One of our upstream providers is currently experiencing an outage. We are monitoring the impact of this on our application.

2. 12. Juni 2025 um 20:47 UTCmonitoring

   Our upstream provider appears to have addressed the issue and we are no longer seeing any issues, but will continue to monitor the situation.

3. 12. Juni 2025 um 21:13 UTCresolved

   This incident has been resolved.

1. 22. April 2025 um 15:06 UTCidentified

   We have identified an issue that is affecting step generation, self-healing snippets, and AI test assertions. Those service will be unavailable until such time as the incident is fixed.

2. 22. April 2025 um 15:39 UTCmonitoring

   A fix has been applied and will be rolling out over the next hour. AI services should now be broadly available for most platforms but some older runs / previews may still be affected. We'll continue to monitor the situation until the rollout is complete.

3. 22. April 2025 um 16:03 UTCresolved

   The fix has been fully applied and all AI services should be back to normal.

1. 7. April 2025 um 17:30 UTCinvestigating

   We are investigating an issue related to background job processing. Some results or runs may be delayed during this time.

2. 7. April 2025 um 17:37 UTCmonitoring

   We've implemented a fix and all delayed background jobs have been processed. We are monitoring to make sure everything continues working properly.

3. 7. April 2025 um 17:52 UTCresolved

   This incident has been resolved.

1. 28. März 2025 um 16:29 UTCidentified

   The Rainforest application is currently offline. This affects access to the application, as well as any tests or runs that were in progress. We're looking into it and will provide an update shortly.

2. 28. März 2025 um 16:35 UTCmonitoring

   The application is back online, and we're monitoring to make sure everything is working properly again.

3. 28. März 2025 um 16:48 UTCresolved

   This incident has been resolved.

1. 7. Januar 2025 um 12:33 UTCidentified

   We've identified an issue which is causing problems when trying to execute tests. Test runs are not being processed. Virtual machines in the Visual Editor are not available. We're working on a fix.

2. 7. Januar 2025 um 13:40 UTCmonitoring

   We've applied a fix and everything should be operational again. We're processing the backlog of runs created in the last 3 hours which will take some time. We'll update the status to "Resolved" once the backlog is processed.

3. 7. Januar 2025 um 14:01 UTCresolved

   The backlog of runs has been processed and everything should be back to normal.

1. 10. Oktober 2024 um 16:30 UTCinvestigating

   We are investigating an issue that is delaying the availability of test result videos

2. 10. Oktober 2024 um 17:52 UTCresolved

   We have resolved the issue affecting test video recording. Test runs from approximately 06:30 UTC to 17:35 UTC may be missing their video recordings.

1. 2. Oktober 2024 um 12:48 UTCidentified

   We are working on an issue that is delaying the allocation of virtual machines when starting a run. We have identified the issue and are working to resolve it.

2. 2. Oktober 2024 um 13:42 UTCidentified

   We have applied a temporary fix to mitigate the issue and are continuing to work on a more permanent fix.

3. 2. Oktober 2024 um 19:28 UTCmonitoring

   A permanent fix has been implemented and performance appears to have returned to normal. No further delays are expected.

4. 2. Oktober 2024 um 20:15 UTCresolved

   This incident has been resolved.

1. 1. Oktober 2024 um 19:41 UTCidentified

   We've identified an issue that may have caused some runs to be delayed.

2. 1. Oktober 2024 um 19:54 UTCresolved

   We've fixed the issue and runs are now starting and progressing as usual.

1. 18. September 2024 um 16:43 UTCinvestigating

   We are currently investigating the issue.

2. 18. September 2024 um 17:52 UTCmonitoring

   A fix has been implemented, we are now monitoring.

3. 18. September 2024 um 18:19 UTCresolved

   The incident has been resolved.

1. 12. Juni 2024 um 23:24 UTCinvestigating

   We've noticed that some test executions are not being processed correctly, which is holding up the completion of runs. We're currently investigating it.

2. 13. Juni 2024 um 00:02 UTCinvestigating

   We've identified the component we believe is causing the problem and are investigating why it's behaving badly.

3. 13. Juni 2024 um 02:15 UTCmonitoring

   We've shipped a fix and are monitoring the situation.

4. 13. Juni 2024 um 06:05 UTCresolved

   This incident has been resolved.

1. 7. Juni 2024 um 12:11 UTCinvestigating

   Operations involving AI are taking some time to complete. We're investigating the issue.

2. 7. Juni 2024 um 12:57 UTCmonitoring

   A fix has been implemented and we are monitoring the results.

3. 7. Juni 2024 um 14:46 UTCresolved

   This incident has been resolved.

4. 11. Juni 2024 um 13:16 UTCpostmortem

   We realized that our provider for AI infrastructure was having a soft outage which was causing some requests to take a very long time \(10 minutes\) to error. We worked around this by decreasing the timeout for the requests so that they would error faster and we could then retry them.

1. 6. Juni 2024 um 15:39 UTCidentified

   We’ve identified another problem with action generation and are working on a fix

2. 6. Juni 2024 um 16:23 UTCmonitoring

   We implemented a fix and are monitoring the results.

3. 6. Juni 2024 um 16:36 UTCresolved

   This incident has been resolved.

1. 6. Juni 2024 um 14:53 UTCidentified

   We’ve identified a problem with action generation and are working on a fix”

2. 6. Juni 2024 um 14:58 UTCmonitoring

   We've shipped a fix and will continue to monitor the situation.

3. 6. Juni 2024 um 15:07 UTCresolved

   The issue has been resolved

1. 8. Mai 2024 um 10:04 UTCidentified

   We're experiencing issues with failure summarization. The fix is being deployed.

2. 8. Mai 2024 um 10:13 UTCmonitoring

   A fix has been implemented and we are monitoring the results.

3. 8. Mai 2024 um 10:20 UTCresolved

   This incident has been resolved.

1. 15. April 2024 um 10:38 UTCinvestigating

   We are currently experiencing an issue with runs being delayed and currently working on a fix.

2. 15. April 2024 um 10:40 UTCinvestigating

   We are continuing to investigate this issue.

3. 15. April 2024 um 10:56 UTCidentified

   We've identified the cause of the delays and are working on a fix.

4. 15. April 2024 um 12:49 UTCmonitoring

   We've shipped a fix and will continue to monitor the situation.

5. 15. April 2024 um 18:24 UTCresolved

   This incident has been resolved.

1. 12. April 2024 um 14:59 UTCinvestigating

   We've noticed a problem with our fleet of VMs, which are used for test execution and test authoring. This may mean that you can't edit tests, and that test execution will be delayed.

2. 12. April 2024 um 16:59 UTCinvestigating

   The problem is somewhat improved, but we're continuing to investigate.

3. 12. April 2024 um 18:23 UTCresolved

   This incident has been resolved.