Ιστορικό διακοπών του Cratesio

1. 11 Ιουλίου 2026 στις 2:22 μ.μ. UTCinvestigating

   Βιώνουμε αυτό που μοιάζει με μια συντονισμένη επίθεση DOS εναντίον των διακομιστών API μας. Παρακολουθούμε την κατάσταση και αξιολογούμε την αντίδρασή μας. Οι μεταφορτώσεις και η ανάλυση της εξάρτησης μέσω του δείκτη αραιών/git δεν θα πρέπει να επηρεάζονται.

2. 11 Ιουλίου 2026 στις 2:22 μ.μ. UTCinvestigating

   Συνεχίζουμε να ερευνούμε αυτό το ζήτημα.

3. 11 Ιουλίου 2026 στις 3:06 μ.μ. UTCmonitoring

   Έχουμε αναπτύξει αντίμετρα και οι σέρβερ μας είναι σε μεγάλο βαθμό πίσω στο κανονικό και πάλι. Συνεχίζουμε να παρακολουθούμε την κατάσταση. Εάν αντιμετωπίσετε οποιοδήποτε πρόβλημα ενημερώστε μας!

4. 11 Ιουλίου 2026 στις 3:42 μ.μ. UTCresolved

   Το περιστατικό λύθηκε. Μεγάλες ευχαριστίες στο Fastly για την παροχή των υπηρεσιών τους CDN και το γρήγορο χρόνο απόκρισης τους!

Αυτόματη μετάφραση από την επίσημη ενημέρωση του συμβάντος.

1. 14 Ιουνίου 2026 στις 1:20 π.μ. UTCinvestigating

   crates.io responses are unusually slow due to database issues, with occasional 500s due to database timeouts. We are investigating.

2. 14 Ιουνίου 2026 στις 1:37 π.μ. UTCidentified

   The read-only replica database used for many of our common queries is under heavy load. We are evaluating options to deal with this.

3. 14 Ιουνίου 2026 στις 1:53 π.μ. UTCresolved

   The primary source of the load appears to have been a scraper abusing our API. We have blocked the scraper and the replica database has returned to its normal heavily loaded state instead of being overloaded.

1. 23 Μαΐου 2026 στις 6:47 μ.μ. UTCidentified

   The reverse dependencies pages are temporarily unavailable. Other crates.io functionality should continue to function normally.

2. 23 Μαΐου 2026 στις 7:19 μ.μ. UTCmonitoring

   Reverse dependencies remain disabled, but the rest of crates.io is functioning normally.

3. 26 Μαΐου 2026 στις 5:00 μ.μ. UTCresolved

   This incident has been resolved.

1. 20 Μαΐου 2026 στις 3:52 μ.μ. UTCmonitoring

   Crates.io is currently under a suspected denial of service effecting the web frontend. The reverse dependency endpoint has been disabled. You will not be able to browse reverse dependencies on crates.io for the duration of the incident. All other operations, publishes and the index are operating normally.

2. 20 Μαΐου 2026 στις 4:57 μ.μ. UTCresolved

   All endpoints are now operational and available again.

1. 14 Απριλίου 2026 στις 8:48 μ.μ. UTCinvestigating

   The crates.io background worker is having a bad morning, and a backlog of Git index updates has accumulated. We have given the worker some coffee and it's catching up, but index updates for newly-published crates are running about an hour behind right now. The sparse index (used by default by all modern Rust versions) is unaffected.

2. 14 Απριλίου 2026 στις 9:08 μ.μ. UTCmonitoring

   The background worker has been told to get itself together. The queue is now trending downwards, but git index updates are still running an hour behind. We are continuing to monitor, but we expect this to resolve over the next few hours.

3. 14 Απριλίου 2026 στις 11:44 μ.μ. UTCresolved

   The indexes are now back to normal operations.

1. 12 Φεβρουαρίου 2026 στις 7:36 μ.μ. UTCidentified

   The crates.io background worker is having a bad morning, and a backlog of Git index updates has accumulated. We have given the worker some coffee and it's catching up, but index updates for newly-published crates are running about an hour behind right now. The sparse index (used by default by all modern Rust versions) is unaffected.

2. 12 Φεβρουαρίου 2026 στις 7:46 μ.μ. UTCresolved

   The background worker has caught up, and crate publishes are now updating normally in both indexes.

1. 10 Νοεμβρίου 2025 στις 11:56 μ.μ. UTCinvestigating

   crates.io is currently experiencing degraded performance on any requests that hit the database (which is basically anything in the API except crate downloads). We are investigating.

2. 11 Νοεμβρίου 2025 στις 12:15 π.μ. UTCidentified

   The slowdown is definitely database related, and we are in contact with our upstream provider to see if we can track down what the root cause might be.

3. 11 Νοεμβρίου 2025 στις 1:04 π.μ. UTCmonitoring

   The 2025 version of "it's always DNS" might be "it's always AI scrapers pretending to be other things". Our database provider pointed out an expensive endpoint that was being hit just enough by a single IP to cause the incident, but not enough for our normal monitoring setup to detect it. We have blocked that IP and response times appear to be returning to normal. Continuing to monitor for a few minutes, but this looks good at the moment.

4. 11 Νοεμβρίου 2025 στις 1:14 π.μ. UTCresolved

   Everything continues to look stable, so we can consider this resolved.

1. 1 Νοεμβρίου 2025 στις 3:15 π.μ. UTCmonitoring

   Due to a large number of crate versions being published in the last few hours, the legacy Git crate index is currently approximately 245 crate versions (or about two hours) behind. The sparse index (which has been the default since Rust 1.70, released in June 2023) is unaffected.

2. 1 Νοεμβρίου 2025 στις 4:04 π.μ. UTCmonitoring

   The queue burndown has slowed due to an unusual number of crate publishes in the last hour, but it is slowly making progress and is now at 206 pending syncs. It could be anywhere from 2-4 hours from here until it catches up.

3. 1 Νοεμβρίου 2025 στις 5:01 π.μ. UTCmonitoring

   Now at 116 pending syncs. Probably 1½ hours before we're caught up, give or take.

4. 1 Νοεμβρίου 2025 στις 6:01 π.μ. UTCmonitoring

   57 pending syncs.

5. 1 Νοεμβρίου 2025 στις 6:38 π.μ. UTCresolved

   All published crates have been synced to the Git index, and there is now no backlog.

1. 2 Σεπτεμβρίου 2025 στις 10:48 μ.μ. UTCmonitoring

   A backlog of CDN invalidations is causing a backlog of approximately 350 crate backlog for sparse index updates and CDN updates. We are monitoring as the queue is worked through.

2. 2 Σεπτεμβρίου 2025 στις 11:08 μ.μ. UTCmonitoring

   The sparse index has synchronized and is now up-to-date. CDN updates are still processing.

3. 2 Σεπτεμβρίου 2025 στις 11:32 μ.μ. UTCmonitoring

   The backlog has cleared and all queues are nominal.

4. 2 Σεπτεμβρίου 2025 στις 11:33 μ.μ. UTCresolved

   This incident has been resolved.

1. 12 Αυγούστου 2025 στις 9:47 μ.μ. UTCidentified

   Due to a large number of crate versions being published in the last few hours, the legacy Git crate index is currently approximately 280 crate versions (or about two hours) behind. The sparse index (which has been the default since Rust 1.70, released in June 2023) is unaffected.

2. 12 Αυγούστου 2025 στις 10:21 μ.μ. UTCmonitoring

   The Git index sync job backlog is slowly burning down, and is anticipated to catch up in a few hours. We will continue to monitor the situation.

3. 13 Αυγούστου 2025 στις 1:55 π.μ. UTCresolved

   The Git index has caught up and is now up to date.

1. 13 Μαΐου 2025 στις 7:37 π.μ. UTCinvestigating

   Since approximately 05:20 UTC, index entries in the Git and sparse crate indexes have not been published. We are investigating.

2. 13 Μαΐου 2025 στις 7:39 π.μ. UTCidentified

   Background workers have been restarted.

3. 13 Μαΐου 2025 στις 7:41 π.μ. UTCmonitoring

   Indexes are up to date, and we are monitoring the background workers to ensure they continue to pick up new jobs.

4. 13 Μαΐου 2025 στις 7:44 π.μ. UTCresolved

   Crates published since the background workers were restarted are now appearing in the index, and the backlog of background jobs has been processed successfully.

1. 20 Μαΐου 2024 στις 11:17 π.μ. UTCidentified

   We are in the process of upgrading our database servers. This will require a short amount of time in read-only mode and a couple of minutes with degraded performance while our follower database is catching up with the new leader.

2. 20 Μαΐου 2024 στις 12:14 μ.μ. UTCmonitoring

   We noticed a small issue with our database indexes after the upgrade, resulting in several crates not being found anymore. We've reindexed the relevant tables which appears to have fixed the problem. The new follower database is catching up an will soon be available to improve the API response times again.

3. 20 Μαΐου 2024 στις 12:26 μ.μ. UTCmonitoring

   Database server upgrade is complete and crates.io should be running at full capacity again now. Hopefully even a bit faster than before due to increased memory on the new database servers.

4. 20 Μαΐου 2024 στις 12:26 μ.μ. UTCresolved

   This incident has been resolved.

1. 22 Μαρτίου 2024 στις 1:04 μ.μ. UTCinvestigating

   We are investigating the crates.io API and website being unreachable. Crate downloads should not be affected.

2. 22 Μαρτίου 2024 στις 1:16 μ.μ. UTCmonitoring

   The application is recovering, but we are still investigating trying to find the root cause. The incident seems to be unrelated to the one earlier today.

3. 22 Μαρτίου 2024 στις 1:25 μ.μ. UTCresolved

   The application is stable, marking the incident as resolved. We will continue investigating the root cause of this.

1. 22 Μαρτίου 2024 στις 7:11 π.μ. UTCinvestigating

   We are investigating the crates.io website and API being unavailable. Downloads from recent Cargo versions should be working, but older Cargo releases relying on the crates.io API are affected.

2. 22 Μαρτίου 2024 στις 7:26 π.μ. UTCmonitoring

   We have identified a problem with our read-only database replica and routed traffic away from it. We are monitoring the recovery, and we will continue investigating the cause of the problem.

3. 22 Μαρτίου 2024 στις 7:48 π.μ. UTCmonitoring

   Since the last update, crates.io fully recovered and we are serving traffic as usual. We're continuing to monitor the situation and we'll re-enable the read-only replica again soon.

4. 22 Μαρτίου 2024 στις 7:56 π.μ. UTCresolved

   This incident has been resolved.

1. 14 Μαρτίου 2024 στις 1:17 μ.μ. UTCidentified

   The crates.io team will perform a database maintenance on 2024-03-15 from 12:00 to 12:15 UTC. We expect this to take less than 15 minutes to complete. During maintenance, crates.io will only be available in read-only mode: downloading crates and visiting the website will still work, but logging in, publishing crates, yanking crates, or changing owners will not work.

2. 15 Μαρτίου 2024 στις 12:00 μ.μ. UTCidentified

   Scheduled maintenance on our database is starting. We expect this to take less than 15 minutes to complete. During maintenance, crates.io will only be available in read-only mode: downloading crates and visiting the website will still work, but logging in, publishing crates, yanking crates, or changing owners will not work.

3. 15 Μαρτίου 2024 στις 12:11 μ.μ. UTCresolved

   Scheduled maintenance finished successfully.

1. 14 Φεβρουαρίου 2024 στις 7:30 μ.μ. UTCinvestigating

   Some crates.io endpoints are timing out at present, including the summary route that drives the crates.io home page. We are investigating.

2. 14 Φεβρουαρίου 2024 στις 7:30 μ.μ. UTCinvestigating

   (If this looks suspiciously similar to https://status.crates.io/incidents/t49v2pfpv0vl, the same issue reappeared on the summary endpoint literally within seconds of resolving that incident. C'est la vie.)

3. 14 Φεβρουαρίου 2024 στις 10:13 μ.μ. UTCidentified

   We believe this issue is being caused by excess database load related to a bug fix deployed earlier today around download counting. This bug fix has caused the normal background processing of per-crate download count totals to take significantly longer and require more resources than usual. We will shortly be temporarily disabling the summary endpoint to alleviate some of the load on the database.

4. 15 Φεβρουαρίου 2024 στις 12:12 π.μ. UTCmonitoring

   The long running background job has completed, and response times for the summary endpoint have returned to normal. The next invocation of the relevant background job will be at 00:30 UTC (so in just under 20 minutes); we will be monitoring that closely to see if any problems resurface at that point.

5. 15 Φεβρουαρίου 2024 στις 12:12 π.μ. UTCmonitoring

   We are continuing to monitor for any further issues.

6. 15 Φεβρουαρίου 2024 στις 2:43 π.μ. UTCresolved

   crates.io has returned to normal service with the processing of the download count backlog from earlier and the completion of the long running background jobs. Investigations will continue during normal hours for the crates.io team to ascertain what is causing elevated database load.

1. 14 Φεβρουαρίου 2024 στις 6:45 μ.μ. UTCinvestigating

   Some crates.io endpoints are timing out at present, including the summary route that drives the crates.io home page. We are investigating. Crate downloads are unaffected.

2. 14 Φεβρουαρίου 2024 στις 6:50 μ.μ. UTCmonitoring

   Restarting the dynos has restored service, but we are still investigating the root cause of the issue, and continue to monitor crates.io performance.

3. 14 Φεβρουαρίου 2024 στις 7:25 μ.μ. UTCresolved

   crates.io continues to operate normally, and no further performance degradation or timeouts have been observed.

1. 27 Δεκεμβρίου 2023 στις 2:24 μ.μ. UTCinvestigating

   We are currently investigating this issue.

2. 27 Δεκεμβρίου 2023 στις 2:37 μ.μ. UTCmonitoring

   Download traffic has been temporarily redirected and the system is recovering. We are monitoring the situation.

3. 27 Δεκεμβρίου 2023 στις 3:45 μ.μ. UTCmonitoring

   Download traffic is being treated and counted regularly again. We are continuing to monitor for any further issues.

4. 27 Δεκεμβρίου 2023 στις 5:11 μ.μ. UTCresolved

   The incident has been resolved. Traffic and error rates are back to normal levels. The root cause has been identified and we are assessing solutions to avoid it from happening again in the future.

1. 20 Δεκεμβρίου 2023 στις 2:31 μ.μ. UTCinvestigating

   The crates.io team will perform a database maintenance on 2023-12-21 from 11:00 to 11:15 UTC. We expect this to take less than 15 minutes to complete. During maintenance, crates.io will only be available in read-only mode: downloading crates and visiting the website will still work, but logging in, publishing crates, yanking crates, or changing owners will not work.

2. 21 Δεκεμβρίου 2023 στις 10:59 π.μ. UTCinvestigating

   Scheduled maintenance on our database is starting. We expect this to take less than 15 minutes to complete. During maintenance crates.io will only be available in read-only mode: downloading crates and visiting the website will still work, but logging in, publishing crates, yanking crates or changing owners will not work.

3. 21 Δεκεμβρίου 2023 στις 11:13 π.μ. UTCresolved

   Scheduled maintenance finished successfully.

1. 16 Νοεμβρίου 2023 στις 7:37 μ.μ. UTCresolved

   At 19:04 UTC a faulty deployment caused the crates.io API to not be reachable anymore. We immediately initiated a rollback to the previous deployment and the issue was resolved at 19:06 UTC. We are currently in the process of figuring out the root cause.