Cockroach Labs 장애 기록

1. 2026년 10월 20일 오전 7:53 UTCinvestigating

   GCP europe-west2-b 및 가용성 영역은 현재 문제가 있습니다. 우리는 이 상황을 감시하고 일단 우리가 더 많은 정보가 있는 갱신 상태를 한 번 계속할 것입니다.

2. 2026년 10월 20일 오전 11:48 UTCidentified

   GCP는 유럽-서부2 지역의 이 붕괴를 초래하는 전력 사건을 mitigated, 충격적인 서비스에 일하고 있습니다. 우리는 일단 우리가 완화의 확인을 한 번 업데이트합니다.

3. 2026년 10월 20일 오후 12:57 UTCmonitoring

   GCP는 유럽-서부2 지역에서이 붕괴를 초래하는 전력 이벤트를 mitigated했으며 대부분의 GCP 서비스는 복구되었습니다. 우리는 계속 추적하고 해결하기 전에 정상적인 가동에 충격을 입힌 CRL 서비스 반환을 지킵니다.

4. 2026년 10월 20일 오후 2:53 UTCresolved

   GCP는 유럽-서부2 지역에 이 붕괴를 초래하는 전력 이벤트를 요약했습니다. 모든 GCP 및 CRL 서비스는 정상 작동으로 돌아 왔습니다. 더 이상 동작이 필요하지 않습니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 10월 6일 오후 9:21 UTCinvestigating

   우리는 현재 GCP us-central1 지역의 인스턴스에 대한 재고를 모니터링하고 있습니다. 가능한 경우, 고객은 새로운 클러스터를 만들 때 다른 지역을 사용하는 것이 좋습니다. 클러스터 생성 및 편집이 결국 용량으로 성공할 것으로 예상됩니다. GCP는 이러한 용량 문제가 해결될 때 타임라인을 제공하지 않았지만, 이 페이지를 계속 모니터링하고 업데이트할 것입니다.

2. 2026년 10월 6일 오후 9:22 UTCmonitoring

   우리는이 Stockout을 모니터링하고 새로운 정보로 업데이트를 게시합니다.

3. 2026년 10월 7일 오후 7:33 UTCresolved

   우리는이 문제를 모니터링하고 있습니다. 그러나 Google은 해결 될 때 ETA를 제공하지 않습니다. 필요한 용량을 얻기 위해 계속 노력할 것입니다. 고객은 여전히 새로운 클러스터를 회전 할 때 다른 지역을 선택할 것이 좋습니다. 용량 문제가 us-central1과 관찰됩니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 9월 17일 오후 7:08 UTCinvestigating

   CockroachDB Cloud (Basic) 클러스터는 GCP South America (São Paulo / southamerica-east1)에서 호스팅되었으며 현재 네트워크 연결 문제가 있습니다. 우리는 투자하고 곧 업데이트를 제공 할 것입니다

2. 2026년 9월 17일 오후 8:39 UTCresolved

   이 사건은 해결되었습니다. CockroachDB Cloud (Basic) 클러스터는 이제 완전히 복원 된 GCP South America (São Paulo / southamerica-east1)에서 호스팅했습니다. 커넥티비티는 정상으로 돌아와 모든 영향을 받은 클러스터가 예상대로 운영됩니다. 우리는이 발생 할 수있는 모든 혼란에 대해 사과합니다. 어떤 문제라도 계속 경험할 경우, 지원해 주세요.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 9월 1일 오후 3:39 UTCinvestigating

   Google Cloud는 Cockroach Cloud 고객에 영향을 미치는 네트워킹 정전을 경험하고 있습니다. Cockroach Cloud 고객은 고도화 된 대기 시간, 지연 또는 연결 문제를 경험할 수 있습니다. 우리의 팀은 이 사건에 완전히 관여됩니다. 현재 충격을 알고: us-central-1, 아시아-동-1

2. 2026년 9월 1일 오후 4:04 UTCinvestigating

   우리는 여전히 상황을 모니터링하고, 지난 15 분 동안 약간의 개선을 보이지만 완전히 해결되지 않습니다. 현재 충격을 알고: us-central1, asia-east1, 유럽-서부4-b

3. 2026년 9월 1일 오후 4:13 UTCmonitoring

   Google은 이제 us-central1 및 asia-east1이 해결되었지만 유럽-서부4-b는 여전히 적극적으로 영향을 미쳤습니다. 우리의 내부 팀은 우리에게 central1 및 asia-east1 지구에 대한 마지막 15 분에 더 이상 오류를 확인하고 있습니다. 현재 충격을 알고: 유럽-서부4-b

4. 2026년 9월 1일 오후 4:37 UTCmonitoring

   우리는 Google이 us-central1-b를 보고하는 것을 배우는 것은 또한 degradation에 충격을 주었습니다. 네트워크 수준의 완화 노력에 따라 네트워크 인프라가 완전히 회복되었습니다. Cloud의 Google 제품 엔지니어링 팀은 현재 개별 서비스의 복구 상태를 검증하고 있습니다.

5. 2026년 9월 1일 오후 7:45 UTCresolved

   Google 영향은 이제 완전히 해결 된 것으로보고됩니다. 내부 팀은이 보고서를 확인했습니다. 모든 서비스는 복원됩니다. 더 많은 도움이 필요하면 지원 티켓에 도달하고 제출하십시오.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 8월 4일 오후 11:46 UTCinvestigating

   We are investigating an issue affecting a subset of Free Tier clusters in eu-west-1, where some clusters may be unable to connect or execute queries. Teams have been notified and are actively working on mitigation.

2. 2026년 8월 4일 오후 11:47 UTCinvestigating

   We are investigating an issue affecting a subset of Basic clusters in eu-west1, where some clusters may be unable to connect or execute queries. Teams have been notified and are actively working on mitigation.

3. 2026년 8월 4일 오후 11:52 UTCinvestigating

   We are investigating an issue affecting a subset of Basic and Standard clusters in eu-west1, where some clusters may be unable to connect or execute queries. Teams have been notified and are actively working on mitigation.

4. 2026년 8월 5일 오전 3:55 UTCmonitoring

   Teams have mitigated the immediate active impact, but work continues to ensure the problem does not recur and to fully understand the root cause. For now we will continue to monitor the situation. If you continue to experience impacts, please reach out to Support.

5. 2026년 8월 5일 오후 7:50 UTCresolved

   Following an extended period of monitoring, the team has confirmed that the cluster remains healthy and stable.

1. 2026년 7월 12일 오후 8:08 UTCinvestigating

   We are currently aware of and investigating a cluster creation failure. Teams are engaged and actively working to mitigate as quickly as possible.

2. 2026년 7월 12일 오후 9:24 UTCresolved

   The cluster creation issue has been restored. Issue is resolved.

1. 2026년 6월 9일 오후 9:26 UTCidentified

   We have identified a minor issue affecting Cloud customers who are unable to accept invites, preventing them from inviting new users to their organizations. Internal teams are aware and a fix is in flight. ETA: 30min - 1hour. We will provide the next update at the top of the hour or prior if the fix is deployed before that time.

2. 2026년 6월 9일 오후 10:02 UTCresolved

   Fix deployment is complete, tested and verified.

1. 2026년 5월 8일 오전 2:13 UTCmonitoring

   AWS is experiencing elevated error rates and degraded performance in a single availability zone in the us-east-1 region. CockroachDB clusters in us-east-1 remain available, and we are monitoring the situation and taking corrective action as appropriate to ensure SQL availability of CockroachDB clusters in this region.

2. 2026년 5월 8일 오전 3:24 UTCmonitoring

   AWS's latest status update reports that they are continuing to work on the situation, and progress is slow. AWS has advised a next update time at 05:00 UTC. We continue to monitor the situation and take appropriate action to ensure uptime and availability for Cockroach DB clusters. We'll next update by about 05:00 UTC when we learn more from AWS.

3. 2026년 5월 8일 오전 8:25 UTCmonitoring

   AWS reports continued progress towards resolving the issue in the US-EAST-1 Region and is working towards full recovery. Affected services may continue to experience elevated error rates and latencies until recovery is complete. We continue to monitor the situation and take appropriate action to ensure uptime and availability for CockroachDB clusters. We'll provide another update as we learn more from AWS.

4. 2026년 5월 8일 오후 9:53 UTCresolved

   All CockroachDB clusters in the affected AWS availability zone have been restored to a fully operational state. We will continue to monitor the upstream incident from AWS, but do not expect any impact to CockroachDB clusters.

1. 2026년 3월 18일 오후 3:23 UTCinvestigating

   Social logins for customers using personal/non-SSO Microsoft accounts is currently unavailable. Our teams are aware and currently investigating.

2. 2026년 3월 18일 오후 4:02 UTCresolved

   Social login for Microsoft accounts is now available. Please reach out to support if you continue to experience issues.

1. 2026년 3월 1일 오후 3:11 UTCmonitoring

   A power outage at an AWS facility in the me-central-1 (Mid-East) region may cause failures or slower than usual progress on cluster operations. AWS is aware of the issue and is working to restore power. We are monitoring the situation, and will provide updates as we receive them from AWS.

2. 2026년 3월 1일 오후 8:43 UTCmonitoring

   Current conflict in the region has led to objects impacting the AWS data center creating a fire earlier today. Power was shut down to prevent further damage. AWS is working on multiple mitigation paths to restore services as quickly as possible. We will provide the next update by 4PM PST or sooner if we have more to share.

3. 2026년 3월 1일 오후 11:09 UTCmonitoring

   AWS is reporting positive signs of recovery at this time, but are still working on multiple mitigation paths to fully restore services. We continue to keep a close eye on the situation and will provide another update by 6PM PST, or sooner if we have additional information to share.

4. 2026년 3월 2일 오전 1:51 UTCmonitoring

   AWS is reporting significant signs of recovery for specific APIs and requests, but several still remain to be mitigated and will likely take several hours more to complete, if not into tomorrow. Our teams have begun testing with this update but have reported we are still impacted at this time. We will continue to monitor the situation closely and will provide one more update this evening by 7PM PST, or sooner if we have additional information to share.

5. 2026년 3월 2일 오전 2:47 UTCmonitoring

   AWS continues to work on mitigation efforts, including stabilizing impacted APIs and implementing interim workarounds. Our teams are actively reviewing and validating these mitigations; however, this effort will continue into tomorrow. We will provide our next update at 7:00 AM PST, or sooner if additional information becomes available.

6. 2026년 3월 2일 오전 9:01 UTCmonitoring

   me-central-1 region is impacted more with two AZs out and the clusters in that region are unavailable. In addition to this, me-south-1 region is also experiencing an AZ outage, which causes degraded performance of clusters in me-south-1. We are closely tracking AWS's status. Impacted customers can track AWS status here https://health.aws.amazon.com/health/status

7. 2026년 3월 2일 오후 5:50 UTCmonitoring

   AWS me-central-1 region is impacted with two AZs out. Clusters in that region are currently unavailable. AWS me-south-1 region is also experiencing a single AZ outage, which causes degraded performance of clusters in that region. While no impact has been observed on GCP or Azure in those regions, customers with single-region deployments are advised to have their disaster recovery plans ready should the need to restore from a backup to another region be required. We are closely tracking AWS's status. Impacted customers can track AWS status here https://health.aws.amazon.com/health/status.

8. 2026년 3월 11일 오후 7:51 UTCresolved

   Beginning March 1, 2026, a power outage caused by physical damage to AWS infrastructure affected multiple Availability Zones in the AWS me-central-1 (UAE) region, resulting in the unavailability of CockroachDB Cloud clusters in that region. A single Availability Zone in me-south-1 was also impacted, causing degraded performance for clusters in that region. While AWS has not yet fully restored power to all affected Availability Zones, Cockroach Labs has directly engaged all customers impacted by this incident, providing resolution guidance and hands-on remediation assistance, including cross-region data restoration where applicable. As remediation steps have been completed or provided to all directly impacted customers, we are resolving this incident from Cockroach Labs' perspective. Cluster creation in the affected regions remains restricted pending full AWS recovery. Customers with outstanding questions or concerns are encouraged to contact Cockroach Labs Support.

1. 2026년 2월 9일 오전 3:04 UTCinvestigating

   We are investigating connectivity issues to Standard and Basic clusters on AWS.

2. 2026년 2월 9일 오전 4:15 UTCmonitoring

   We have resolved an internal misconfiguration, and see that Standard and Basic clusters are functioning normally again. We will continue to monitor to ensure that the error does not recur.

3. 2026년 2월 9일 오후 3:20 UTCresolved

   Standard and Basic clusters continue to function normally.

1. 2026년 2월 2일 오후 4:56 UTCidentified

   We have identified an issue that is causing Advanced cluster creation on AWS to take longer than usual. We are preparing a fix, and will update this status page with more information when we have an ETA for resolution.

2. 2026년 2월 2일 오후 5:57 UTCidentified

   We are preparing a fix for deployment, which should restore AWS cluster creation times to normal levels. We will update the status page when the fix is deployed, or by 14:30 US/Eastern with more information.

3. 2026년 2월 2일 오후 7:31 UTCidentified

   We are continuing to work on the fix for this issue. Next update will be at 15:00 US/Eastern.

4. 2026년 2월 2일 오후 7:46 UTCresolved

   We have deployed and verified a fix. Advanced clusters should now create within normal time frames.

1. 2025년 10월 31일 오후 3:54 UTCinvestigating

   We are investigating errors that are causing delays in creation of Basic and Standard clusters on AWS. Basic and Standard cluster creation on GCP is operating normally at this time.

2. 2025년 10월 31일 오후 4:21 UTCinvestigating

   We are investigating errors with Basic and Standard clusters on AWS. Cluster creation, certain cluster edit operations (such as adding a region), and certain CockroachDB Cloud Console functionality (such as listing databases or users) are also impacted. Basic and Standard clusters on GCP are operating normally at this time.

3. 2025년 10월 31일 오후 6:22 UTCidentified

   We have identified the cause of cluster creation failures impacting Basic and Standard AWS clusters. As a precautionary measure, while we apply a fix, we have disabled all cluster creation in Cockroach Cloud. We will restore cluster creation when it is safe to do so. There is no SQL availability impact to any existing clusters. We will update here by 3:00 PM US/Eastern with more information.

4. 2025년 10월 31일 오후 6:59 UTCidentified

   We are continuing to work on a fix for this issue. Next update will be by 4:00 PM US/Eastern time.

5. 2025년 10월 31일 오후 7:58 UTCidentified

   We are continuing to work on a fix for this issue. Next update will be by 4:30 PM US/Eastern time.

6. 2025년 10월 31일 오후 8:18 UTCmonitoring

   We have resolved the underlying issue that was preventing cluster creation and other cluster operations. We have re-enabled cluster creation on all clouds. We will monitor the situation to ensure that everything continues to operate normally.

7. 2025년 10월 31일 오후 10:09 UTCresolved

   All cluster operations are working normally.

1. 2025년 10월 30일 오후 2:50 UTCinvestigating

   We are investigating errors that cause creation for Advanced clusters to take longer than usual. Standard and Basic cluster creation is operating normally.

2. 2025년 10월 30일 오후 3:25 UTCmonitoring

   We have applied a mitigation and observe that cluster creations are proceeding again. We will monitor all impacted cluster creations to ensure they succeed.

3. 2025년 10월 30일 오후 4:05 UTCresolved

   Cluster creation is operating normally, and all delayed clusters have finished creation.

1. 2025년 10월 20일 오후 1:28 UTCidentified

   We’re currently impacted by an ongoing AWS outage in the us-east-1 region. Customers may experience failures or delays when creating new clusters, adding nodes, or running backups in this region. Existing clusters remain operational, but backup operations may be delayed or fail intermittently. We’re continuing to monitor AWS recovery efforts and will provide updates as more information becomes available.

2. 2025년 10월 20일 오후 10:52 UTCmonitoring

   AWS has resolved the underlying outage in the us-east-1 region, and we’re seeing recovery across CockroachDB Cloud operations. Cluster creation, scaling, and backups are now succeeding again. We’re continuing to monitor for full stability before marking this incident as resolved.

3. 2025년 10월 21일 오후 2:56 UTCresolved

   The earlier AWS outage in the us-east-1 region has been fully resolved. All CockroachDB Cloud operations, including cluster creation, scaling, and backups, are functioning normally again. We’ve verified recovery across affected systems and customer clusters, and normal operations have resumed.

1. 2025년 9월 24일 오후 12:31 UTCidentified

   We are currently experiencing an incident impacting certain cluster operations on Google Cloud Platform due to an known ongoing Google Kubernetes-related issue. Customers may encounter errors when creating or editing clusters. Our engineering teams are actively monitoring the situation and communicating with Google regarding resolution.

2. 2025년 9월 24일 오후 4:00 UTCresolved

   Google has reported the issue has been resolved and we've verified cluster operations are functional.

1. 2025년 7월 29일 오후 11:08 UTCinvestigating

   We are observing an incident impacting SQL availability across multiple Serverless clusters. We are currently investigating the issue.

2. 2025년 7월 29일 오후 11:17 UTCmonitoring

   Correction from the original status update. This issue appears to have been isolated to cluster crl-prod-6rw. We have identified the cause, mitigated and confirmed that cluster availability has returned. We are currently completing additional validations.

3. 2025년 7월 29일 오후 11:35 UTCresolved

   We have confirmed resolution of the incident impacting SQL availability for the affected cluster. SQL availability is restored and the associated cluster is now healthy.

1. 2025년 7월 21일 오후 9:39 UTCidentified

   We are aware of an ongoing Incident with dependent service "Let's Encrypt": https://letsencrypt.status.io/, which may impact cluster creation and add region operations for CockroachDB Cloud Advanced clusters. We will update this status page with more information as it becomes available.

2. 2025년 7월 22일 오전 4:22 UTCresolved

   LetsEncrypt has indicated the issue has been mitigated. We have validated cluster creation behaviour is now returning to normal.

1. 2025년 7월 3일 오후 4:58 UTCinvestigating

   We are investigating an issue that may prevent or delay cluster operations from succeeding.

2. 2025년 7월 3일 오후 5:39 UTCresolved

   We have corrected a misconfiguration which was preventing cluster operations from succeeding. We are monitoring the success of cluster operations already in progress and new operations.

1. 2025년 7월 2일 오후 5:42 UTCinvestigating

   We are investigating cluster operations, including creating or editing clusters, that are taking longer than usual.

2. 2025년 7월 2일 오후 6:50 UTCidentified

   We have identified the issue preventing cluster operations from proceeding, and we see that operations are moving forward. We will next update this incident by 16:00 EDT.

3. 2025년 7월 2일 오후 7:32 UTCmonitoring

   We continue to see cluster operations succeeding, and will actively monitor to ensure the issue does not reoccur.

4. 2025년 7월 2일 오후 8:46 UTCresolved

   All cluster operations are succeeding normally at this time.