Historia awarii Perimeterx

We are experiencing an issue with an upstream cloud provider impacting our detection engine. We are in the process of investigating and deploying mitigations

Due to the issues encountered and as a precautionary measure, we have switched the system to bypass mode. We are continuing to monitor the situation and are working on further mitigations.

In line with the recovery of the cloud provider, our systems are also beginning to recover. We are monitoring the results closely and, as a precautionary measure, will remain in bypass mode.

The issue has been resolved, and all services have returned to full operational status.

Possible partial degradation of the cloud detector potentially impacting console visibility and detection

Due to an infrastructure issue the system experienced slowness which impacts console visibility. A fix has been implemented and we are monitoring results. No impact on detection.

Resolved - The infrastructure issue has been resolved, and all systems are operating normally.

We are experiencing network issues that impacts mobile traffic detection

The issue has been identified and a fix was implemented at 15:10 UTC

A fix has been implemented and we are monitoring the results.

The incident has been resolved, System is fully recovered

We're aware of an issue that might be affecting our system and are working on a resolution

We have identified the issue and we are actively working towards recovering the system.

We have identified the issue and we are actively working towards recovering the system.

Due to a cloud issue, we've experienced a small data loss between approximately 23:00 and 00:00 UTC. In addition we had a minor impact on detection in that window. Both issues are back to normal and we are continuously monitoring the cloud issue.

Detected issues are back to normal and we are continuously monitoring a minor cloud networking issue.

This incident has been resolved.

We are currently investigating this issue.

Due to a faulty backend configuration the system experienced partial data loss from 08:00 to 10:45 UTC along with temporary false negatives from 09:28 to 09:50 UTC Since the incident the system is back to normal and fully stable

We are experiencing network issues in several regions, which are being investigated.

We have identified and traced the network issue to our Cloud Vendor's load balancer. We are actively working towards recovering the system. In the meantime, customers should expect an elevated number of 5xx errors, which will cause the detection engine to operate in a fail-open mode

We have implemented a fix to restore the system and are currently monitoring its performance. The system should be fully operational within a few minutes.

The issues have been resolved, and the system is now fully operational

Our data processing infrastructure is running behind which is causing inaccuracies in the reporting tools

We are continuing to investigate this issue.

This incident has been resolved.

We're aware of a problem with admin console affecting a subset of customers

The issue has been identified and a fix is being implemented.

This incident has been resolved.

The issue has been identified and a fix is being implemented.

This incident has been resolved.

The PerimeterX external support chat system (Slack) is currently experiencing performance issues. [email protected] email ticketing system is unaffected, and may still be used for support communications.

At this time any issues our partners may have experienced related to access or logging into your dedicated Slack channel or space should be resolved. However, if you are still experiencing issues related to access or Slack channel login, please reach out to your customer success point of contact via email, or submit an escalation directly to [email protected]. Additionally, by phone at +1-650-620-7800 (press 2 for support). We sincerely appreciate your patience and support.

At this time any issues our partners may have experienced related to access or logging into your dedicated Slack channel or space should be resolved. However, if you are still experiencing issues related to access or Slack channel login, please reach out to your customer success point of contact via email, or submit an escalation directly to [email protected]. Additionally, by phone at +1-650-620-7800 (press 2 for support). We sincerely appreciate your patience and support.

The issue has been identified and a fix is being implemented.

This incident has been resolved.

We are currently investigating this issue.

This incident has been resolved.

There is an issue to login to the portal for customers using Okta OIDC SSO; Portal is showing partial results

We are continuing to work on a fix for this issue.

This incident has been resolved.

We are aware of an issue which started at 15:15 UTC that affected detection accuracy and resulted in an increase in false negatives. The issue was identified and we are actively working on remediating the situation.

The issue that affected detection accuracy and resulted in an increase in false negatives was resolved. There is a data lag in bot-defender portal. The team is currently working to reduce the data lag currently affecting the delayed or missing data in our console dashboards.

We are continuing to work on a fix for this issue.

This incident has been resolved.

We are currently investigating this issue.

This incident has been resolved.

We are aware of a backend configuration issue started 08:07 UTC, that affected detection accuracy and resulted in an increase in false positive and false negative cases. The issue was identified and fully remediated on 09:35 UTC (partial remediation to allow exoneration already achieved on 08:30 UTC).

We are currently investigating this issue.

The issue has been identified and a fix is being implemented.

This incident has been resolved.

We are currently investigating this issue.

We are continuing to investigate this issue.

Network outage. Customers should expect elevated amounts of 5xx responses from our backend and 502 errors in JS sensor requests

The issue has been identified and a fix is being implemented. The rate of 5xx is expected to start decreasing

A fix has been implemented and we are monitoring the results.

We are continuing to monitor for any further issues.

Customers should expect to receive 200 responses as usual on all systems.

Cloud detection is fully operational. Data is missing for reporting causing inaccurate data to be presented in the console.

All systems are fully operational

This incident has been resolved.

We have identified a security incident with one of our authentication vendors. As we analyze the impact, we have disabled the authentication with the portal. You can reach out to our support if needed in the meantime.

We are continuing to work on a fix for this issue.

Our analysis so far shows no compromise or risk to any of our users. We will issue any update on return to service plan within the hour.

We have completed the review. No risk was found. Everything was restored.

Earlier today (2022-03-01) our system experienced elevated challenge solve rates from 9:40 to 10:40 UTC The issue has been fully mitigated