Historia awarii Bugcrowd

We have identified an issue with Hacker / Researcher logins to the platform - currently you may receive a 500 error when logging in. We have applied a fix and are awaiting that to take effect.

Logins should be functioning as normal, but we will continue monitoring the situation.

This incident has been resolved.

There's an ongoing issue in Tracker that is resulting in users experiencing 500 HTTP responses from the server. We have found a potential cause for this and currently working towards fixing this asap.

We have applied a fix for this and currently monitoring the issue.

This incident has been resolved.

We're currently working with a vendor to resolve an issue in our authentication processes, at the moment new users, users who haven't haven't logged in for awhile, and users who have changed their password will be unable to login to the platform.

We've implemented a remediation and are now recovering.

This incident has been resolved.

We've identified another issue that is preventing us from sending platform emails and are working with our vendor to get it resolved but there is no ETA currently.

Email sending from the platform is functional again.

We've identified an issue that is preventing us from sending platform emails (e.g. program updates, password resets, etc). We are working on a fix but there is no ETA currently.

Email sending from the platform is functional again.

An issue with request rate-limiting is/was preventing POST requests from competing correctly. A fix is being deployed.

This incident has been resolved.

A dependency upgrade has caused unexpected errors in our production environment. The team is reverting the code change and we expect the error to be resolved in the next 20 minutes.

This incident has been resolved.

Currently the tracker portal, researcher portal and even the marketing site is down. We are investigating the issue. We will keep you posted on the progress.

We have identified the problem and applied a fix for the issue. The services are now back up and we are currently monitoring the platform.

Monitoring has alerted us to an issue with incoming webhook processing. The issue with the backend service has been identified and we are taking steps to remediate it, but while this is occurring there will be delays in inbound webhook processing.

The backend service that processes inbound webhooks has recovered, and is processing the backlog of webhook events.

Backlog processing has completed, systems have returned to normal.

One of our internal services is experiencing errors. We have initiated a workaround and meanwhile investigating the cause. You may notice several parts of the tracker and researcher portal might be resulting in errors as a result of this.

We have applied a temporary fix for the issue and the error rate has dropped. We are continuing to monitor and also investigating the root cause.

Root cause has been identified and the issue has been resolved. We will continue to monitor though.

We are currently experiencing issues with the outbound webhooks being broken. We are diagnosing this and will update the progress along the way.

Our proxy was experiencing problems due to unexpected behaviour of the dependant AWS service(s). The exact root cause is still unclear and we have filed a support ticket with AWS to understand the same.

The issue has resolved automatically. Will co-ordinate with the cloud provider support team for the RCA.

We've identified that there is an issue with the tracker and researcher portal currently that is causing the site to either not load, or load very slowly. We are working to rectify the problem.

A fix is rolling out and site performance should be returning to normal.

This incident has been resolved.

We are investigating an issue with indexed submissions and search currently - some functions involving submission retrieval are impacted at the moment.

We have identified a fix and are currently rolling it out.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

A mail provider is having issues which is leading to researcher emails going to bugcrowdninja.com addresses being delayed or not delivered. We are currently working with the provider to rectify the issue.

The mail provider has resolved the issue.

The previous maintenance was resolved early, and maintenance is still continuing. We will update on this new incident until the maintenance is complete.

Database maintenance has been completed.

We have identified a current issue with our CDN that fronts all of the Bugcrowd sites. While this is ongoing, Bugcrowd is currently unavailable. Incoming and outgoing webhooks are unaffected.

The CDN has recovered and our sites are accessible. We are monitoring the situation.

This incident has been resolved.

www.bugcrowd.com and numerous other sites on the internet are currently suffering from an external provider's outage. We are continuing to monitor the situation and will provide further updates as they become available.

The external provider issue is reportedly resolved and we are continuing to monitor availability.

The external provider has resolved the issue, and www.bugcrowd.com is continuing to function normally.

Our API is experiencing elevated error rates. We are investigating and will keep you posted.

This incident has been resolved.

Our hosting provider for our home page is having issues. We are investigating

Our provider appears to have resolved the issue. We will continue to monitor

This incident has been resolved.

We are currently performing some emergency database maintenance. Expected downtime is less than 10 minutes.

We are continuing to work on a fix for this issue.

This incident has been resolved.