Apollo 장애 기록

1. 2026년 10월 6일 오전 12:26 UTCinvestigating

   새로운 그래프 artifacts 빌드를 배포하는 문제입니다. 우리는 현재 뿌리 원인을 조사하고있다

2. 2026년 10월 6일 오전 12:37 UTCmonitoring

   클라우드 구성에 문제가 있습니다. 우리는 그것을 수정하고 이제 복구를보고

3. 2026년 10월 6일 오전 12:45 UTCresolved

   Graph artifacts 파이프라인의 문제점은 해결되었습니다. 우리는 문제가 해결 된 동안 만 지연되는 동안 데이터가 손실되지 않았습니다. 우리는이 붕괴에 대한 사과

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 9월 1일 오후 3:41 UTCinvestigating

   Google Cloud Platform (GCP)에서 진행되는 정전으로 여러 Apollo 서비스에서 서비스 중단을 경험하고 있습니다. 우리는 적극적으로 투자하고 더 많은 정보로 업데이트를 공유 할 것입니다.

2. 2026년 9월 1일 오후 3:48 UTCinvestigating

   AWS 업링크 엔드포인트는 사용할 수 있습니다. GraphOS Routers 및 Gateways를 사용하여 기본 멀티 클라우드 구성을 계속 시작하고이 아웃시 구성을 끌어낼 수 있지만 새로운 스키마가 게시 될 수 없습니다.

3. 2026년 9월 1일 오후 4:39 UTCidentified

   Google Cloud Platform (GCP)에서 진행되는 정전으로 여러 Apollo 서비스에서 서비스 중단을 경험하고 있습니다. 우리는 적극적으로 투자하고 더 많은 정보로 업데이트를 공유 할 것입니다.

4. 2026년 9월 1일 오후 5:23 UTCmonitoring

   우리의 제 3 자 공급 업체는 문제가 us-central1-b 영역의 일부에 격리되었음을 확인했습니다. 지역에있는 나머지 영역은 비공개되었습니다. 대부분의 Apollo 제품은 복구 및 엔지니어링 팀은 나머지 영향을받는 서비스를 검증합니다. 우리는 이 사건을 해결하기 전에 밀접하게 감시할 것입니다.

5. 2026년 9월 1일 오후 8:18 UTCresolved

   우리의 제 3 자 공급 업체는 문제가 해결되고 모든 Apollo 제품은 복구되었습니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 8월 26일 오후 10:12 UTCidentified

   일부 조직 및 사용자 수정 (사용자가 초대, 사용자 권한 변경, OAuth 변경)은 간헐적으로 실패합니다. 우리는 수정에서 일하고 있습니다.

2. 2026년 8월 26일 오후 10:18 UTCmonitoring

   우리는 더 이상 어떤 요구 실패를 본지 않습니다, 그러나 우리는 뿌리 원인 고침이 배치될 때까지 계속 감시할 것입니다.

3. 2026년 8월 26일 오후 11:36 UTCresolved

   수정이 배포되었습니다. 이러한 실패 작업은 속도 제한 오류로 등장하지만, 속도 제한은 잘못 적용되지 않았습니다. 우리는 예상대로 속도 제한 작업을 보장하기 위해 변경했습니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 8월 11일 오후 7:11 UTCinvestigating

   Registry service workflow latency has spiked, affecting mulitple internal services serving long-running workflows.

2. 2026년 8월 11일 오후 8:23 UTCmonitoring

   The issue has been resolved and we're continuing to monitor.

3. 2026년 8월 11일 오후 11:19 UTCresolved

   The issue has been resolved.

1. 2026년 8월 5일 오전 9:43 UTCidentified

   Embedded Explorer and Embedded Sandbox are temporarily down. The issue has been identified and a fix is being deployed.

2. 2026년 8월 5일 오전 10:08 UTCresolved

   The issue has been fixed.

1. 2026년 7월 9일 오후 3:41 UTCinvestigating

   We’re currently investigating a rise in error rates for our Proposals service.

2. 2026년 7월 9일 오후 3:54 UTCinvestigating

   We are still investigating elevated error rates for Proposals. We appreciate your patience.

3. 2026년 7월 9일 오후 4:05 UTCidentified

   We’ve identified the issue and are working on resolution.

4. 2026년 7월 9일 오후 4:21 UTCmonitoring

   We have applied a fix and have begun to see recovery in our metrics. We will continue to monitor and update the status page once it’s resolved.

5. 2026년 7월 9일 오후 6:09 UTCresolved

   The error rate for Proposals has returned to normal, no further impact expected.

1. 2026년 6월 28일 오전 3:13 UTCinvestigating

   We are investigating suspected data loss for field and coordinate usage metrics for all customers.

2. 2026년 6월 28일 오전 4:01 UTCmonitoring

   Field and coordinate usage metric ingestion is functional and lag is trending downwards. We estimate 8-10 hours for full recovery.

3. 2026년 6월 28일 오전 4:21 UTCmonitoring

   The backlog of object field, input object field, and enum value metrics are now being processed but we expect the backlog to take 8-10 hours to be fully processed. For a period of time between 4 am and 9pm on June 27 UTC we expect there to be data loss for input object field and enum value metrics, as well as the client and operation breakdown of object field metrics. No other metrics (operation, trace, etc.) are impacted.

4. 2026년 6월 28일 오후 8:16 UTCresolved

   The backlock of schema coordinate (object field, input object field, and enum value) metrics has now cleared and there is no more metric ingestion delay. The metrics for input object fields, enum values, and the operation and client breakdown of object fields is inaccurate for June 27 4-5am and 8-9pm, and missing between 5am to 8pm (all times in UTC). All other metrics were not impacted.

1. 2026년 6월 15일 오후 2:38 UTCinvestigating

   We are investigating a case where metric ingestion is delayed for some customers. There is no loss of data.

2. 2026년 6월 15일 오후 4:18 UTCinvestigating

   We have identified the cause though customers continue to experience metric ingestion delays while we identify the appropriate mitigation. Data backfill after remediation is still expected after mitigation is applied. We appreciate your patience.

3. 2026년 6월 15일 오후 5:11 UTCidentified

   We are incrementally applying mitigations which are showing positive results. We feel we are confidently iterating toward recovery. At this time, no data is believed to be lost and metric backfill will continue with the oldest metrics being populated first, though we currently have a substantial backlog which we will scale to process quickly. We will update this status page as things continue and we appreciate your patience.

4. 2026년 6월 15일 오후 5:42 UTCmonitoring

   We are confidently seeing recovery. Some metric delay will continue for the next couple hours, though many metric types have already recovered. We will keep this status page open in Monitoring state and update it when we have eliminated our metric lag. Thank you again for your patience.

5. 2026년 6월 15일 오후 10:16 UTCresolved

   The metric ingestion backlog has been fully processed and all metrics have resumed normal ingestion. We do not believe any data was lost as a result of this incident. All metric types should now be operating as expected. Thank you for your patience throughout this incident.

1. 2026년 6월 1일 오전 8:35 UTCinvestigating

   Some Router deployments are experiencing issues coming online due to a message around licensing. We are investigating.

2. 2026년 6월 1일 오전 9:12 UTCinvestigating

   We are continuing to investigate.

3. 2026년 6월 1일 오전 9:54 UTCmonitoring

   We have seen recovery for about 50 minutes now and we are continuing to monitor the recovery. If you find that you are impacted by a licensing message on Router startup, please open a ticket with our support team.

4. 2026년 6월 1일 오전 11:39 UTCresolved

   After extended monitoring, we have confirmed that the issue is resolved. Post-incident action items will continue after the end of the incident. As with all incidents, we will be conducting an internal post-mortem. We appreciate your patience.

1. 2026년 5월 25일 오후 7:50 UTCinvestigating

   We’re currently investigating an issue where the “Submit a Request” and “My Requests” buttons are not appearing properly in the Apollo Support Portal. As a temporary workaround, tickets can still be created and accessed through the “Submit a Request” button available in the header of the portal.

2. 2026년 5월 26일 오전 1:48 UTCinvestigating

   We’re continuing to investigate the issue affecting the visibility of the “Submit a Request” and “My Requests” buttons in the Apollo Support Portal. At this time, we are working with our external provider to identify the root cause and restore normal functionality.

3. 2026년 5월 26일 오후 12:41 UTCmonitoring

   We’ve implemented changes to address the issue affecting the visibility of the “Submit a Request” and “My Requests” buttons in the Apollo Support Portal. At this time, the portal appears to be functioning normally, and we are actively monitoring the situation to ensure stability. If you continue experiencing issues, please try refreshing the page or signing out and back into the portal. We’ll continue to monitor closely and provide additional updates if needed.

4. 2026년 5월 26일 오후 12:58 UTCresolved

   The issue affecting the visibility of the “Submit a Request” and “My Requests” buttons in the Apollo Support Portal has been resolved. Portal functionality has been restored, and users should now be able to access these options normally. Thank you for your patience while we worked to resolve this issue.

1. 2026년 4월 27일 오전 8:34 UTCinvestigating

   Opening a ticket using the Contact Support button on the [studio.apollographql.com](http://studio.apollographql.com "studio.apollographql.com") site itself is currently failing. Support tickets can still be created through our [support.apollographql.com](http://support.apollographql.com "support.apollographql.com") portal, or via our `support@` email address on the `apollographql.com` domain.

2. 2026년 4월 27일 오전 9:23 UTCmonitoring

   Support ticket opening functionality has been restored to Studio’s “Contact Support” functionality in the UI. We have mitigated the failures and will monitor before marking as resolved.

3. 2026년 4월 27일 오전 10:29 UTCresolved

   We have observed sustained restored functionality in our Studio-initiated support ticket form and are marking this as fully resolved.

1. 2026년 4월 24일 오후 7:27 UTCinvestigating

   We're seeing issues with StudioUI and are investigating to find the root cause

2. 2026년 4월 24일 오후 8:09 UTCinvestigating

   We're continuing to investigate to find the root cause and resolve the issue

3. 2026년 4월 24일 오후 8:33 UTCidentified

   We have identified a possible root cause, and we are testing changes to confirm

4. 2026년 4월 24일 오후 8:49 UTCmonitoring

   We confirmed with internal testing that our fix is working as expected, and customers should be seeing recovery. We're continuing to monitor to confirm.

5. 2026년 4월 24일 오후 8:53 UTCresolved

   We have resolved the issue. Customers should see full functionality.

1. 2026년 4월 21일 오후 5:05 UTCinvestigating

   We’re investigating the availability of our platform after a recent deploy.

2. 2026년 4월 21일 오후 5:20 UTCmonitoring

   We have restored functionality to impacted users SSO access and are monitoring.

3. 2026년 4월 21일 오후 5:47 UTCresolved

   SSO login is now fixed.

1. 2026년 4월 16일 오후 12:45 UTCinvestigating

   We are aware of an issue affecting GraphOS Studio where users may encounter errors when attempting to view the Changes or Schema Reference tabs within graph proposal. We are investigating and will give an update soon.

2. 2026년 4월 16일 오후 1:27 UTCmonitoring

   We have identified the root cause of the failures affecting the Changes and References tabs in Schema Proposals. A fix has been deployed and we continue to monitor and investigate.

3. 2026년 4월 16일 오후 3:43 UTCresolved

   Proposals errors have been clear for ~2 hours since the rollback reached prod. Marking resolved.

1. 2026년 3월 31일 오전 6:59 UTCinvestigating

   We’re monitoring an interruption in the transmission of transactional emails (e.g., email verification, welcome emails, proposals approvals). We are investigating.

2. 2026년 3월 31일 오전 7:26 UTCmonitoring

   We’ve identified the root cause of the delays, applied mitigations and recent delivery attempts are succeeding. Some previously attempted deliveries may take time to be retried.

3. 2026년 3월 31일 오전 7:39 UTCresolved

   Retries of delayed transaction emails have been successful. If you have not received an email you were expecting for account reasons, we suggest retrying the workflow which triggered the email.

1. 2026년 2월 12일 오전 6:06 UTCresolved

   We've identified an issue where our metrics ingestion pipeline was falling behind for several hours. Systems have now fully recovered and no data was lost, however some customers may have experienced delays viewing insights data in Studio and / or slower page load times during this event. Apologies for the interruption.

1. 2025년 11월 21일 오후 6:05 UTCinvestigating

   Platform API is also seeing degraded performance, which we are investigating

2. 2025년 11월 21일 오후 6:05 UTCinvestigating

   We're seeing increases in latency across a few different Apollo services. You may be seeing latency issues with proposals, checks, and studio.

3. 2025년 11월 21일 오후 7:06 UTCinvestigating

   We're still investigating the root cause. We're seeing some recovery, but not to full recovery

4. 2025년 11월 21일 오후 8:06 UTCinvestigating

   We're continuing to investigate the root cause of the issue. Some customers are reporting significant improvements in performance.

5. 2025년 11월 21일 오후 9:05 UTCmonitoring

   We've implemented a fix and are monitoring the system.

6. 2025년 11월 21일 오후 10:05 UTCresolved

   The issue has been resolved.

1. 2025년 11월 18일 오후 7:06 UTCresolved

   There was a period of degraded performance on schema publishing. The root cause has been identified and service restored.

1. 2025년 11월 18일 오후 1:05 UTCinvestigating

   We're experiencing elevates error rates due to an upstream DNS provider major incident.

2. 2025년 11월 18일 오후 2:06 UTCmonitoring

   We're continuing the monitor the affected services while the upstream provider remediates the issue.

3. 2025년 11월 18일 오후 2:06 UTCmonitoring

   Our DNS provider has provided an update: "The issue has been identified and a fix is being implemented." We will be monitoring for recovery of our systems.

4. 2025년 11월 18일 오후 2:44 UTCmonitoring

   At this time, our monitors indicate that only Cloud Routers hosted on Apollo infrastructure are impacted as a result of an upstream service provider incident. Studio UI has already recovered. Beyond these two impacted services, there has been no impact to APIs (including Uplink).

5. 2025년 11월 18일 오후 3:02 UTCmonitoring

   The upstream provider has indicated recovery on their status page (https://www.cloudflarestatus.com/incidents/8gmgl950y3h7). At the same time, we have seen our own monitors return to non-triggered states. We will continue to monitor for the next 15 minutes and then declare this Resolved. While Cloud Routers were impacted throughout the incident, the Studio UI impact was during two distinct periods: Nov 18 11:52 am – Nov 18, 12:10 pm UTC (18 minutes) and again from Nov 18, 12:51 pm – Nov 18, 1:09 pm UTC (another 18 minutes). Studio UI was otherwise available in times outside of that range.

6. 2025년 11월 18일 오후 3:38 UTCresolved

   We are marking this incident resolved on the Apollo side. For any continuing matters, please consult our support portal: https://support.apollographql.com/

1. 2025년 11월 18일 오후 1:05 UTCresolved

   Cloud Routers are experiencing elevated errors due to an upstream external provider incident.