Historique des pannes de Percy

Percy builds were degraded for ~50 mins [7.30 AM PDT to 8.20 AM PDT on 13th July] due to an existing Google Cloud Provider incident We have moved our infrastructure from the impacted GCP node pool and are back to normal processing GCP incident details - https://console.cloud.google.com/servicehealth/incidentDetails/projects%2Fpercy-prod%2Flocations%2Fglobal%2Fevents%2FZBYNZVS?project=percy-prod&tab=full-history

Percy customers were facing intermittent login issues from 7:45 pm PDT to 10:14 PM PDT on 6th July We have identified the issue and login functionality has been fully restored

A subset of Percy builds experienced slowness from 4:15 PM PDT to 5 PM PDT [29th June] The issue is identified and resolved, services are operating normally

In the 20 mins duration some customers might have seen some missing snapshots in the build or failures in creating builds themselves with the 401 status code on the Percy CLI logs. We have fixed the issue and monitoring the stack.

Percy experienced login failures for a subset of users between 11:54 PM PDT (18 May) – 12:54 AM PDT (19 May) We have identified the issue and login functionality has been fully restored

Between 4:30 AM and 9:30 AM PDT on May 11, 2026, some Percy builds took longer than usual to complete due to increased database contention. Builds were not lost - only delayed. The issue has been mitigated and processing times are back to normal. We're investigating the root cause to prevent recurrence in future.

Customers using Percy on Automate experienced build failures between May 5th, 8:00 AM PDT and May 6th, 1:30 AM PDT. The issue has now been resolved, and Percy on Automate builds should be functioning as expected

We are currently investigating this issue. The impacted features only include Percy's AI features - all non AI features are fully active and has no disruption or performance degradation but AI features like NL descriptions, AI ignore, Build Summary, Bug classification will not be available for the builds created/processed currently.

We are continuing to investigate this issue. The impacted features only include Percy's AI features - all non AI features are fully active and has no disruption or performance degradation but AI features like NL descriptions, AI ignore, Build Summary, Bug classification will not be available for the builds created/processed currently. All dashboard functions are fully functional and should not affect the standard percy workflow for the clients.

A fix is implemented and deployed. We are monitoring the situation. All builds created after 11:15 PDT should have all AI features available.

This incident has been resolved.

Some users might have faced issue while logging in during ~35 min window. No existing sessions were impacted.

- Between February 8, 2026 (10:30 AM – 2:30 PM UTC) and February 9, 2026 (12:00 PM – 1:30 PM UTC) some customers may have noticed incorrect visual diffs on Percy (Safari on iPhone) - The issue occurred due to an incorrect screenshot capture where a “connection not private” error page was rendered instead of the expected page - The impact was limited, affecting a very small percentage of builds across a small number of organisations - Our engineering team has identified and addressed the underlying issue - All Percy services are now operating normally We sincerely apologise for any confusion or inconvenience this may have caused

We are currently investigating this issue.

The issue has subsided, we are actively investigating the root cause

The root cause were identified as related to https://www.cloudflarestatus.com/incidents/3xb30dhqfkw7 - there is no live impact but we are monitoring the situation. We are keeping this open until cloudflare marks the incidence has resolved.

Marking this as resolved as original incidence from Cloudflare is marked as resolved.

A subset of users were impacted by infinite reloads and unable to access percy dashboard while attempting to access a build URL on percy dashboard - which internally attempted sign in with browserstack.com. The issue is fixed as of 5:30 AM PST and all dashboard access should be working as expected now.

We are currently investigating this issue.

The downtime is related to ongoing Cloudflare outage - we are monitoring the status

We are continuing to work on a fix for this issue.

Cloudflare seem to have recovered, we are monitoring the situation.

We are continuing to monitor for any further issues.

percy.io is currently accessible; however, build processing performance is degraded due to an ongoing Cloudflare incident. As a result, active builds are being queued on our infrastructure. We will resume normal build processing once Cloudflare fully resolves the issue.

We have resumed normal operations. Queued Percy builds are now being processed. Build speeds may be slower than usual while we work through the backlog. We have scaled up our infrastructure to clear the backlog as quickly as possible

We have resumed normal operations and cleared up Percy backlog of builds. Build processing speed is back to normalcy

We have resumed normal operations and cleared up Percy backlog of builds. Build processing speed is back to normalcy

We are currently experiencing degradation in visual difference impacting Percy builds due to an underlying Cloudflare issue. Affected builds may show snapshots with blank pages or missing assets, which can result in incorrect Visual differences Window : 10:30 PM PDT 22nd Oct - 1.47 AM PDT 23rd Oct

Percy Review Agent (AI features) are currently experiencing issues due to ongoing AWS outage affecting multiple of our vendors. You would likely see the message `AI diff is unavailable on this comparison` as a message on the dashboard if you attempt to use the AI features. We are actively monitoring the situation and would keep this updated.

The issue has been identified.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We are currently observing degraded performance from 6.30 AM - 7.45 AM PDT in Percy, primarily affecting build processing times and dashboard responsiveness. Our investigation indicates that this is due to an ongoing AWS US-East-1 regional outage impacting one of our critical third-party services - LaunchDarkly - which we use for configuration management and feature flag delivery. We are actively monitoring the situation and working with teams to mitigate impact. Builds may take longer to start or complete until the upstream issue is resolved.

We are currently observing degraded performance from 12:20 PM PDT to 1 PM PDT [20th October] in Percy, primarily affecting build processing times and dashboard responsiveness. Our investigation indicates that this is due to an ongoing AWS US-East-1 regional outage impacting one of our critical third-party services . We are actively monitoring the situation and working with teams to mitigate impact. Builds may take longer to start or complete until the upstream issue is resolved.

On October 14th from 1.30 PM PDT to 8.30 PM PDT (14th October), Percy web experienced a slowness in processing of builds in majority of organisations. This was due to an underlying infrastructure degradation. Our team deployed a fix to address slow build times. All systems are now operating normally. We sincerely apologise for any disruption this may have caused.

[Minor impact on build performance] On September 9th , our team deployed a fix to address the slow build times experienced by some organisations. This was due to an underlying infrastructure degradation. All systems are now operating normally. We sincerely apologise for any disruption this may have caused.

We are currently investigating elevated error rates affecting parts of our backend infrastructure. This may result in intermittent failures or delays in processing builds or visual comparisons. Our engineering team is actively working to identify and resolve the issue. We will provide an update as soon as more information is available.

We are continuing to investigate this issue.

We are experiencing a partial service disruption due to widespread outages with our cloud providers, including Cloudflare and GCP. Our team is actively monitoring the situation and will post further updates as they become available.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.