Ιστορικό διακοπών του Doppler

1. 27 Οκτωβρίου 2026 στις 7:26 μ.μ. UTCinvestigating

   Αυτή τη στιγμή ερευνούμε αυτό το ζήτημα.

2. 27 Οκτωβρίου 2026 στις 7:30 μ.μ. UTCidentified

   Αναγνωρίσαμε την αιτία του αυξημένου χρονικού ορίου και προσπαθούμε να το επιλύσουμε.

3. 27 Οκτωβρίου 2026 στις 8:01 μ.μ. UTCidentified

   Συνεχίζουμε να δουλεύουμε προς την επίλυση. Μια επιδιόρθωση έχει προετοιμαστεί και θα αναπτυχθεί σύντομα.

4. 27 Οκτωβρίου 2026 στις 8:15 μ.μ. UTCmonitoring

   Έχει αναπτυχθεί μια λύση και παρακολουθούμε τα αποτελέσματα.

5. 27 Οκτωβρίου 2026 στις 8:25 μ.μ. UTCresolved

   Το περιστατικό λύθηκε.

Αυτόματη μετάφραση από την επίσημη ενημέρωση του συμβάντος.

1. 17 Ιουλίου 2026 στις 12:03 π.μ. UTCinvestigating

   Έχουμε λάβει αναφορές για λάθη όταν προσπαθούμε να κατεβάσουμε το Doppler CLI. Ερευνούμε το θέμα.

2. 17 Ιουλίου 2026 στις 12:08 π.μ. UTCresolved

   Το περιστατικό λύθηκε από την ομάδα μας.

Αυτόματη μετάφραση από την επίσημη ενημέρωση του συμβάντος.

1. 20 Νοεμβρίου 2025 στις 3:55 μ.μ. UTCinvestigating

   We are currently investigating an issue involving delays impacting integration sync, webhook and email delivery.

2. 20 Νοεμβρίου 2025 στις 4:43 μ.μ. UTCinvestigating

   We are continuing to investigate this issue.

3. 20 Νοεμβρίου 2025 στις 6:54 μ.μ. UTCmonitoring

   A fix has been implemented and we are monitoring the results.

4. 20 Νοεμβρίου 2025 στις 7:46 μ.μ. UTCresolved

   This incident has been resolved.

1. 18 Νοεμβρίου 2025 στις 12:35 μ.μ. UTCmonitoring

   Doppler is currently affected by an ongoing Cloudflare outage. This is impacting Dashboard and API access. Services accessing Doppler secrets through an integration (e.g., Kubernetes operator or syncs) should remain unaffected, but direct attempts to access the API will be unreliable until this issue resolves.

2. 18 Νοεμβρίου 2025 στις 12:51 μ.μ. UTCidentified

   Services are still impacted. Our team is continuing to monitor the situation.

3. 18 Νοεμβρίου 2025 στις 1:49 μ.μ. UTCmonitoring

   Cloudflare identified the issue and has been working on implementing a fix. Our dashboard and API are currently available and we're continuing to monitor the situation.

4. 18 Νοεμβρίου 2025 στις 2:59 μ.μ. UTCresolved

   This incident is now resolved.

1. 20 Αυγούστου 2025 στις 6:52 μ.μ. UTCinvestigating

   We're currently investigating an issue involving delays in background job processing that's impacting integration syncs, email delivery, activity log delivery, and webhook delivery (amongst other things). Impacted integration syncs will show as failed with "Out of sync". If expected emails aren't being received, it's likely related to this as well.

2. 20 Αυγούστου 2025 στις 7:33 μ.μ. UTCinvestigating

   We are continuing to investigate the issue.

3. 20 Αυγούστου 2025 στις 7:36 μ.μ. UTCidentified

   The issue has been identified and a fix is being implemented.

4. 20 Αυγούστου 2025 στις 7:38 μ.μ. UTCmonitoring

   A fix has been implemented and we're monitoring the situation. All unprocessed jobs have now been processed.

5. 20 Αυγούστου 2025 στις 7:58 μ.μ. UTCresolved

   This incident has been resolved.

1. 6 Αυγούστου 2025 στις 3:36 π.μ. UTCinvestigating

   We are currently investigating this issue.

2. 6 Αυγούστου 2025 στις 4:30 π.μ. UTCmonitoring

   We believe the issue is resolved and are monitoring for any additional issues.

3. 6 Αυγούστου 2025 στις 5:40 π.μ. UTCresolved

   The issue is resolved.

1. 12 Ιουνίου 2025 στις 6:29 μ.μ. UTCinvestigating

   We're currently investigating a system-wide outage related to disruptions with GCP and Cloudflare.

2. 12 Ιουνίου 2025 στις 8:19 μ.μ. UTCmonitoring

   GCP and Cloudflare have been gradually resolving their outages and Doppler services have been partially restored.

3. 12 Ιουνίου 2025 στις 9:07 μ.μ. UTCresolved

   This incident has been resolved.

1. 11 Απριλίου 2025 στις 2:23 π.μ. UTCinvestigating

   We're currently experiencing downtime due to extended database maintenance.

2. 11 Απριλίου 2025 στις 2:36 π.μ. UTCresolved

   This incident has been resolved.

1. 30 Οκτωβρίου 2024 στις 11:18 μ.μ. UTCinvestigating

   Certain users may be unable to fetch from personal configs.

2. 30 Οκτωβρίου 2024 στις 11:19 μ.μ. UTCinvestigating

   We are continuing to investigate this issue.

3. 30 Οκτωβρίου 2024 στις 11:41 μ.μ. UTCinvestigating

   A fix has been identified and should be deployed soon.

4. 30 Οκτωβρίου 2024 στις 11:53 μ.μ. UTCinvestigating

   Only users with access to personal configs via groups are affected. Users with direct access to personal configs are not affected.

5. 31 Οκτωβρίου 2024 στις 12:05 π.μ. UTCresolved

   This incident has been resolved.

1. 31 Μαΐου 2024 στις 6:04 μ.μ. UTCidentified

   Activity Logs stopped publishing to Splunk as of May 30th at 19:55 UTC. The issue has been identified and a fix is in progress. After the issue has been resolved and the connections reconnected, missed activity logs will be republished.

2. 31 Μαΐου 2024 στις 6:05 μ.μ. UTCidentified

   We are continuing to work on a fix for this issue.

3. 31 Μαΐου 2024 στις 7:51 μ.μ. UTCresolved

   A fix has been deployed and we are monitoring reconnections. We will backfill missed Activity Logs as needed after workplaces reconnect Splunk.

1. 16 Απριλίου 2024 στις 1:29 μ.μ. UTCresolved

   For a period of 18 minutes, secrets related API and dashboard actions saw increased error rates (503) and timeouts.

1. 29 Φεβρουαρίου 2024 στις 9:29 μ.μ. UTCidentified

   We are working on a fix.

2. 29 Φεβρουαρίου 2024 στις 9:38 μ.μ. UTCidentified

   We are continuing to work on a fix for this issue.

3. 29 Φεβρουαρίου 2024 στις 10:00 μ.μ. UTCmonitoring

   A fix has been implemented and we are monitoring the results.

4. 29 Φεβρουαρίου 2024 στις 10:10 μ.μ. UTCresolved

   This incident has been resolved.

1. 4 Αυγούστου 2023 στις 10:46 μ.μ. UTCinvestigating

   We are currently investigating this issue.

2. 4 Αυγούστου 2023 στις 10:46 μ.μ. UTCinvestigating

   We are continuing to investigate this issue.

3. 4 Αυγούστου 2023 στις 10:50 μ.μ. UTCresolved

   This incident has been resolved.

4. 4 Αυγούστου 2023 στις 11:18 μ.μ. UTCpostmortem

   This incident was caused by a faulty Kubernetes NetworkPolicy change. We’ll be evaluating how we can adjust our deployment procedures to catch these changes in the future.

1. 22 Μαρτίου 2023 στις 10:30 μ.μ. UTCinvestigating

   We're currently investigating this issue. If you need immediate availability, you can install Doppler directly from our published release artifacts instead of via your OS's package registry by running: ``` (curl -Ls --tlsv1.2 --proto "=https" --retry 3 https://cli.doppler.com/install.sh || wget -t 3 -qO- https://cli.doppler.com/install.sh) | sh ```

2. 22 Μαρτίου 2023 στις 11:13 μ.μ. UTCresolved

   An incident was identified at our provider for OS package registry downloads. They've fixed the issue and installs from OS package registries are available again. Impact: The incident affected all Doppler users attempting to install the Doppler CLI via their OS package registeries. This resulted in disruptions to their workflows relying on the Doppler CLI. Resolution: Cloudsmith resolved the incident after around an hour, and users were once again able to install the Doppler CLI from their OS package registries.

1. 2 Μαρτίου 2023 στις 6:55 μ.μ. UTCresolved

   For several hours on February 27, 2023, users intermittently encountered errors when attempting to install the Doppler CLI. This issue was due to a related incident on GitHub, where Doppler hosts all CLI binaries and signatures. According to GitHub's incident report, GitHub experienced degraded performance and increased error rates for their packages service. Impact: The incident affected all Doppler users attempting to install the Doppler CLI. This resulted in disruptions to their workflows and GitHub actions relying on the Doppler CLI. Resolution: GitHub resolved the incident after a few hours, and users were once again able to install the Doppler CLI.

1. 16 Ιανουαρίου 2023 στις 8:37 μ.μ. UTCresolved

   During this incident, changes to secrets could not be synced to integrations or trigger webhook updates. Additionally, activity log notifications could not be posted to Slack, Microsoft Teams, Sumo Logic, Splunk, or Datadog.

2. 17 Ιανουαρίου 2023 στις 1:31 π.μ. UTCpostmortem

   # Summary From 2023-01-16 19:11 UTC to 2023-01-16 20:04 UTC, Doppler experienced a partial outage which prevented sync integrations, webhooks, and activity log notifications from executing. The outage also prevented an internal job from firing which recomputes the version hashes for Doppler configs. This resulted in API clients \(e.g. the Doppler CLI and Kubernetes Operator\) failing to receive secrets updates which were made during this window. A recovery migration was run at 2023-01-17 00:37 UTC, re-triggering all syncs, webhooks, and activity log notifications — as well as recomputing config version hashes to restore the functionality for all clients to fetch secret updates. # Incident Details Doppler uses RabbitMQ to queue jobs which need to be executed as a result of secret updates. On 2023-01-13, Doppler’s security team rotated a RabbitMQ password, mistakenly identifying the credential as unused in production. It took several days for the RabbitMQ sessions in Doppler’s production services to expire and once they did, queue jobs could no longer be published. Once the incident was identified, Doppler’s security team created new RabbitMQ users to be used by our production services. The change was deployed and the incident was resolved at 2023-01-16 20:04 UTC. At 2023-01-17 00:37 UTC, Doppler ran a recovery migration to re-fire queue events for sync integrations, webhooks, activity log notifications, and secret version hash recomputations that were meant to fire during the incident window. # Next Steps Doppler has switched from using a single RabbitMQ credential to using one user per service. RabbitMQ users are now clearly named to mitigate the risk of accidental rotation in the future. We’ve also identified that the ability for API clients to fetch secrets should not be dependent on our application’s ability to connect to RabbitMQ. Our engineering team will move the config version hash computation to our atomic secrets write operation to ensure that the latest secrets are always fetched by clients. Lastly, our engineering team is reconfiguring the way we queue asynchronous jobs to ensure that if secrets are modified during a partial infrastructure failure, all post-update jobs will eventually be executed — without the need for manual recovery migrations.

1. 5 Ιανουαρίου 2023 στις 12:48 π.μ. UTCidentified

   New and existing Heroku sync integrations are currently not functioning. We have identified the issue and are working on resolving it. Secrets previously synced to Heroku apps will remain available but new secret changes in Doppler will not be synced until this is resolved.

2. 5 Ιανουαρίου 2023 στις 1:06 π.μ. UTCidentified

   This incident may persist for up to 24 hours. In the meantime, if you have an urgent secret change that needs to be made, please update the secret in your Doppler dashboard and then manually update the secret in Heroku directly via `heroku config:set`. Updating the secret in Doppler will ensure that the value in Heroku is not overwritten once the incident is resolved.

3. 5 Ιανουαρίου 2023 στις 6:23 μ.μ. UTCresolved

   This issue is now resolved. Additional action is required to re-enable existing Heroku syncs. All workplaces will need to reconnect their Heroku sync integrations from the [workplace Settings page](https://dashboard.doppler.com/workplace/settings). Once integrations are reconnected, all syncs will automatically be triggered to sync any pending changes in Doppler. A postmortem will be available soon with more details regarding what happened.

4. 5 Ιανουαρίου 2023 στις 9:30 μ.μ. UTCpostmortem

   **Root Cause** Doppler syncs secrets to Heroku via a Heroku OAuth application. This application is created in the Heroku dashboard and must be owned by a single Heroku user account. Doppler’s previous Heroku OAuth application was owned by a specific Doppler employee’s Heroku account without access to any additional resources. During a routine external account audit, this account was mistakenly identified as unused and manually deleted by our security team. This irrecoverably deleted Doppler’s existing Heroku OAuth application, thereby breaking any existing syncs and requiring the creation of a new OAuth application in a new account. **Resolution** Because users had authorized our previous Heroku OAuth application to their Heroku account\(s\), users need to authorize the new Heroku OAuth application. This involves reconnecting the integration from the [Doppler dashboard](https://dashboard.doppler.com/workplace/settings). Once the integration is reconnected, Doppler will re-enable all associated syncs that have been disabled and perform a fresh sync. Note that the previous OAuth application was deleted and therefore no action is required to remove its access to your Heroku account. **Next Steps** Internally, we’re reorganizing how shared accounts used for critical functionality are stored in 1Password. This new 1Password organization should help prevent this kind of accidental deletion in the future. We avoid shared accounts whenever possible, but this isn’t always feasible given third party implementations. We'll also be adding our individual integrations to our status page. This will allow customers to more easily see which integrations, if any, are currently experiencing issues.

1. 29 Αυγούστου 2022 στις 1:01 π.μ. UTCinvestigating

   We are currently investigating this issue.

2. 29 Αυγούστου 2022 στις 1:32 π.μ. UTCmonitoring

   A fix has been implemented and we are monitoring the results.

3. 29 Αυγούστου 2022 στις 1:38 π.μ. UTCresolved

   This issue was caused by a redis server running out of memory. We've increased the amount of memory allotted to this server. We'll also be setting up additional alerting on redis resource usage so that we can catch and prevent these issues in the future.

1. 1 Αυγούστου 2022 στις 4:45 μ.μ. UTCresolved

   An error was discovered while running a data backfill job that resulted in users being unable to update secrets for affected environments. The data that was generated triggered an unexpected assertion in the secrets validation code. Users that attempted to save secrets in these environments would have received a 500 response from the server with the message "An error has occurred but don't fret. Our team has been notified." The data has been fixed for all environments and secrets write operations are fully functional again.

1. 21 Ιουνίου 2022 στις 6:33 π.μ. UTCinvestigating

   We are currently investigating this issue.

2. 21 Ιουνίου 2022 στις 6:52 π.μ. UTCidentified

   This issue is due to a widespread Cloudflare outage. https://www.cloudflarestatus.com/incidents/xvs51y9qs9dj

3. 21 Ιουνίου 2022 στις 7:09 π.μ. UTCmonitoring

   A fix has been implemented and we are monitoring the results.

4. 21 Ιουνίου 2022 στις 7:10 π.μ. UTCmonitoring

   We are continuing to monitor for any further issues.

5. 21 Ιουνίου 2022 στις 7:22 π.μ. UTCresolved

   This incident has been resolved.