Testingbot outage history

We experienced 90 minutes of stability issues due to a hardware problem. This has been resolved and we are taking steps to prevent this from happening in the future.

Our upstream datacenter is experiencing issues causing services on TestingBot to fail. We are following up.

This incident has been resolved.

Our datacenter provider is experiencing issues, causing tests on TestingBot to fail. You are currently not able to log in or run tests. We will update this section as soon as we get more information.

The datacenter has recovered. Cause was a broken router.

This incident has been resolved.

Maestro tests against physical Android devices were returning errors.

A previous deploy introduced a bug that failed the start of Maestro tests when some conditions were met. We have added a fix, test and alert.

TestingBot Storage has been experiencing issues today on February 3, 2026 from 16:20 UTC up to 19:00 UTC. This has been resolved.

Today between 10:19 and 10:29 we had an incident with our database which caused downtime. This was resolved and we will be making changes to prevent this from happening.

Since 1:00 UTC time we have been experiencing elevated wait times, causing some tests to wait for up to 5 minutes before they could start. We deployed a fix and are monitoring the waiting times. They are back to < 10 seconds since 04:30 UTC.

Today we experienced long waiting times for Windows \+ macOS machines. The issue was caused by a server issue, where memory issues were causing restarts of our hub \(h[ub.testingbot.com\)](http://hub.testingbot.com) This in turn caused the machines running the tests to fail to communicate with the hub, causing tests from clients to wait up to 5 minutes before a test could start. If your test runner is configured with a http connect timeout < 5 minutes, this may have caused issues where tests would fail to start. The memory issue was resolved and we are looking into a solution to prevent this from happening again.

One of our server racks has lost power, resulting in a loss of service. Website and grid are affected. We're working on restoring this asap.

Our website and service is back online.

On Sunday 26th of November 2017 our datacenter experienced an issue with one of their switches, which caused network downtime from 11:23 until 11:55 UTC.

Between 14:30 and 14:55 there was a spike in tests failing to start. We've identified this issue and fixed this.

Our ISP had an issue that caused downtime between 9:44 and 9:50

Since 16:30 UTC we started seeing increased errors when starting tests. A fix has since been deployed to resolve this issue.

Due to maintenance to our database servers our service might be temporarily down or unresponsive. We're working on getting everything back into shape as fast as possible.

Maintenance has completed.

We're seeing high wait times for all VMs - working on pushing a fix

This incident has been resolved.

We're still seeing cascading effects from the outage of an hour ago, we're working on getting everything back to normal

This incident has been resolved.

Between 18:40 and 18:54 our Selenium Grid may have been unavailable. We're taking actions to prevent this from occurring again in the future.

Website was unavailable between 10:10 and 10:15 UTC today

Between 02:30 and 03:40 UTC today some tests were queued for a longer time (>5mins). Some tests failed to start because of this. The issue has been resolved - we're making changes to make sure this does not happen anymore.

From 18:15 UTC until 18:22 UTC we experienced an issue that caused tests running during that time to slow down significantly or even timeout. The problem has been fixed, all tests should run at normal speed again. Apologies for the inconvenience

This incident has been resolved.

Between 0:00 and 0:30 UTC today we had some issues with the TestingBot tunnel not starting correctly. This has been fixed and measures have been taken to prevent this from happening again in the future.