Fullstory 장애 기록

1. 2026년 9월 24일 오전 10:29 UTCresolved

   변경은 9 월 23 일에 캡처 된 자산에 대한 URL이 해결되는 방법; 이 inadvertently led는 EU1 데이터 센터의 세션에 대해 깨진 재생. 이것은 9:45am UTC에서이 변경을 롤링하여 수정되었습니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 9월 7일 오후 10:32 UTCmonitoring

   우리는 일부 세션 데이터 가용성에 지연을 발생시키는 문제에 대한 수정을 시행하고 검색 및 대시보드 성능에 기여하고 잠재적으로 풀스토리의 알림을 연기했습니다. 우리는 적극적으로 결과를 모니터링하여 전체 안정성을 보장합니다.

2. 2026년 9월 7일 오후 10:34 UTCmonitoring

   우리는 더 많은 문제를 모니터링하고 있습니다.

3. 2026년 9월 8일 오전 5:46 UTCresolved

   이 사건은 이제 해결되었습니다. 지난 6:15 PM (ET)에서 문제가 발생하면 [email protected]으로 이메일을 보내주십시오.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 9월 1일 오후 3:19 UTCinvestigating

   저희 클라우드 인프라 제공업체(Google Cloud Platform)가 NA1 데이터 센터의 여러 Fullstory 서비스에 영향을 미치는 이슈를 알고 있습니다. Affected 서비스는 데이터 캡처 (웹 및 네이티브 모바일), Fullstory 웹 응용 프로그램, API, Webhooks & Alerts, Anywhere : 창고, 가이드 및 설문 조사, StoryAI 및 인력이 포함됩니다. 데이터 캡처는 이 기간 동안 중단 될 수 있습니다. 우리는 우리의 공급자와 활발히 관여되고, 상황을 감시하고, 그들이 유효한 것과 같이 갱신을 제공할 것입니다.

2. 2026년 9월 1일 오후 4:32 UTCmonitoring

   클라우드 인프라 제공업체(Google Cloud Platform)에 의해 보고된 문제점에 대한 복구를 보고하고, 전액적 안정성을 확보하기 위해 적극적으로 모니터링하고 있습니다. 또한 제공 업체의 상태를 직접 따라 할 수 있습니다. https://status.cloud.google.com/incidents/J5ia5t9p3g9Q5Wi7r8Ev

3. 2026년 9월 1일 오후 8:36 UTCresolved

   클라우드 인프라 제공업체 (Google Cloud Platform)이 보고한 문제는 이제 해결되고 Fullstory 서비스는 9월 1일 오후 4시 32분에 2026년 9월 1일로 완전 운영됩니다. 모든 백업 데이터는 현재 처리되고 곧 사용할 수 있습니다. 질문이 있거나 지속적인 문제가 발생하면 [email protected]으로 연락하십시오.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 8월 18일 오후 5:57 UTCinvestigating

   10:40AM ET에서, NA1의 Silver Schema 사용자는 11:10am ET의 주위에 영향을받은 모든 사용자와 창고 동기화 지연을 경험하기 시작했다. 우리는 여전히이 문제를 조사하고있다.

2. 2026년 8월 18일 오후 6:42 UTCmonitoring

   우리는 NA1 지역의 실버 슈마 창고 고객에게 지연을 일으키는 원인이 된 10 : 40 AM ET 주위에 시작된 문제의 해결을 구현했습니다. 시스템은 이제 잡아. 우리의 팀은 모든 자료가 정확하게 처리된다는 것을 보증하기 위하여 단계를 가지고 가고, 우리는 일단 모든 가공이 정상에 다시 한 번 다른 갱신을 포스트할 것입니다.

3. 2026년 8월 18일 오후 8:24 UTCresolved

   NA1 지역의 Silver Schema 창고 고객을위한 지연이 발생 한 문제는 해결되고 서비스가 8 월 18 일, 2026에서 4 : 23 PM ET으로 전체 운영으로 돌아갑니다. 궁금한 점이 있거나 지속적인 문제가 발생하면 [email protected]에서 팀에 도달하지 마십시오.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 8월 7일 오후 1:40 UTCresolved

   From 7:58pm UTC on the 6th of August, users in NA1 & EU1 organisations with Guides & Surveys enabled were unable to create or edit tours and smart tips. There was also some disruption of end-user content delivery, though this was mitigated by edge caching serving cached content. The issue was resolved by 8:30am UTC on the 7th of August.

1. 2026년 7월 23일 오후 3:46 UTCmonitoring

   우리는 7 월 23 일 오후 7시 45 분 (ET)에서 7시 45 분 (ET) 주변을 시작 BigQuery와 문제가 있습니다 : BigQuery를 활용 창고 연결. 우리는 상황을 밀접하게 감시하고 그(것)들의 개정을 제공할 것입니다.

2. 2026년 7월 23일 오후 5:16 UTCmonitoring

   이 사건의 범위를 명확하게하려면 : Fullstory는 BigQuery를 내부적으로 사용하여 최종 목적지 유형과 관계없이 NA1 데이터 센터에 대한 데이터를 변환합니다. 결과적으로 BigQuery degradation은 BigQuery, Redshift, Databricks 및 Snowflake Destinations를 포함한 NA1의 소규모 테이블의 동기화를 지연시킬 수 있습니다. BigQuery, Redshift, Databricks 및 Snowflake Destinations뿐만 아니라 BigQuery와 직접 동기화 할 수 있습니다. 우리는 모니터링을 계속하고 복구 동기화로 더 많은 업데이트를 제공합니다.

3. 2026년 7월 23일 오후 8:58 UTCresolved

   이 사건은 해결됩니다. 아래에서 BigQuery degradation는 mitigated이고, Fullstory가 어디에나 사용했던 데이터 변환 층: NA1의 창고는 정상적인 가동으로 돌아 왔습니다. 이 기간 동안 지연 된 모든 동기화는 이제 잡았고, 영향을받는 테이블은 모든 목적지 (BigQuery, Redshift, Databricks 및 Snowflake)에서 최신 상태로 유지됩니다. 이 문제는 7 월 23 일 오전 7시 45 분부터 오전 11시 20 분까지 발생했습니다. 우리는 혼란에 대해 사과하고 당신의 인내에 감사드립니다. [email protected]으로 이메일을 보내주십시오. 문제가 계속되는 경우.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 7월 16일 오후 8:38 UTCinvestigating

   Dashboards, Segments, Session Replay 내에서 직접 임베디드되는 StoryAI 기능 뒤에 모델은 Google Outage로 인해 손상된 가용성을 볼 수 있습니다. 우리는이 문제를 해결하기 위해 팀과 긴밀하게 협력하고 있습니다. Fullstory 내의 다른 AI 기능은이 시간에 영향을 미치지 않습니다.

2. 2026년 7월 16일 오후 10:41 UTCresolved

   이 사건은 해결되었습니다

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 7월 8일 오후 8:15 UTCresolved

   Between July 6, 2026 at 8:47 AM EDT and July 7, 2026 at 12:00 PM EDT, incorrect filter configurations were intermittently applied to the default canned metrics (Users Today, Sessions Today, Error Rate, and Avg Page Load Time), which could have caused them to display unexpected filter values or report zero results due to no matching data. If you have any questions or are experiencing any continued issues with canned metrics, please reach out to [email protected].

1. 2026년 6월 11일 오후 1:58 UTCmonitoring

   Beginning at approximately 04:00 CEST (22:00 EDT Jun 10), end users on sites where Guides & Surveys is installed may have experienced guides, tours, surveys, and smart tips not displaying correctly, and interaction data was delayed. This only is impacting customers in our NA data center. This issue was caused by resource exhaustion in the service responsible for delivering and recording end-user activity, not in the FullStory application itself. The FullStory app, Guides & Surveys editor, and theme/targeting configuration were not affected. No data was lost. Content delivery may have been delayed, and whether queued interactions are replayed will depend on whether the end user's session re-qualifies against targeting rules. We have identified the root cause of this issue, implemented a fix, and are actively monitoring the service. We will provide a further update once the service has been fully restored.

2. 2026년 6월 11일 오후 5:08 UTCresolved

   This incident has been resolved. Guides, tours, surveys, and smart tips are now delivering normally for end users in the NA data center. We apologize for the disruption and are implementing additional safeguards to prevent recurrence. If you are still experiencing issues, please contact support at [email protected].

1. 2026년 6월 11일 오후 1:28 UTCinvestigating

   We are investigating an issue where Metric Alerts are triggering at thresholds that do not match the underlying metric data for customers in both our EU and NA data centers. Customers may be receiving alert notifications that appear to fire incorrectly or more frequently than expected. Our engineering team has been notified and is actively investigating. We will provide an update as soon as we have more information. We apologize for the disruption and appreciate your patience.

2. 2026년 6월 11일 오후 4:45 UTCresolved

   As of 9:36 AM ET on June 11, 2026, this issue is resolved. A feature flag introduced as part of our multi-metric support rollout caused alert evaluations to diverge from actual metric values, resulting in Metric Alerts firing incorrectly even when the underlying metric had not breached its threshold. Start time of this issue: Multi-Metric Beta customers: June 4, 2026 at 10:27 AM ET All other customers using Metrics with division: June 10, 2026 at 4:25 PM ET If you continue to receive alerts you believe are incorrect, please reach out to us at [email protected]. We apologize for the disruption.

1. 2026년 6월 8일 오후 5:15 UTCmonitoring

   We are monitoring an issue that began on June 6 at 10:00am ET causing data processing delays in our EU1 data center. New sessions are appearing normally, but analytics for recent sessions and data sent from Fullstory to connected warehouses may be delayed. A fix has been implemented today and we are monitoring the system's recovery. We will provide another update as soon as possible.

2. 2026년 6월 8일 오후 6:05 UTCresolved

   This issue is now resolved. If you notice any further delays in analytics indexing or data warehouse syncing reach out to [email protected].

1. 2026년 6월 5일 오후 4:44 UTCresolved

   On April 6th, 2026, we introduced a limit on the number of pages that can be recorded in a single session to ensure optimal performance. An unexpected side effect caused our systems to continue collecting excess session data without properly saving it, creating a processing backlog. To resolve this, we quickly paused the limit and significantly increased our system's processing capacity to clear the backlog. Unfortunately, because of the size of the delay, a portion of session data was permanently deleted before it could be successfully archived. Due to ongoing metric degradation as a result of that incident, we experienced a similar build up in our processing backlog on May 18th 2026. While we were able to capture the events in these sessions, the affected sessions will appear in a blank state. This issue impacted a small subset of sessions in some customer accounts in our NA data center from May 7th, 04:00 UTC to May 19th, 07:00 UTC when the issue was ultimately resolved.

1. 2026년 5월 19일 오후 9:31 UTCresolved

   On April 6th, 2026, we introduced a limit on the number of pages that can be recorded in a single session to ensure optimal performance. An unexpected side effect caused our systems to continue collecting excess session data without properly saving it, creating a backlog in our database. To resolve this, we quickly paused the limit and significantly increased our system's processing capacity to clear the backlog. Unfortunately, because of the size of the delay, a portion of session data was permanently deleted before it could be successfully archived. While we were able to capture the events in these sessions, the affected sessions will appear in a blank state. This issue impacted a small subset of sessions in some customer accounts in our NA data center from April 6th, 11:00 UTC to April 14th, 14:00 UTC when the issue was ultimately resolved.

1. 2026년 3월 17일 오후 7:19 UTCresolved

   Some customers using Fullstory Anywhere: Warehouse may have observed a significant decrease in the number of events being transported to their data warehouse between February 24, 2026 to March 13, 2026. No data was lost during this time and all data has been backfilled to the impacted customers' warehouses. This incident is now resolved.

2. 2026년 3월 17일 오후 7:19 UTCpostmortem

   2026/02/24 Data transport to data warehouse significantly delayed for some customers This postmortem details the customer impact, the root cause of what happened, how we addressed the problem, and how we will prevent it from happening in the future. # Customer Impact Some customers using Fullstory Anywhere: Warehouse may have observed a significant decrease in the number of events being transported to their data warehouse between Feb 24, 2026 to Mar 13, 2026. The impact here is limited to object storage destinations such as Amazon S3, Azure Blob Storage, and Google Cloud Storage. # Root Cause The Fullstory platform has two internal pipelines that generate event data and notify downstream systems. There is a streaming pipeline for real time event processing, and a batch pipeline for everything else. Object storage data destinations receive notifications from the latter whenever it processes session and page entities that are deemed inactive i.e. no longer receiving event data. On Feb 24, 2026 the team made a change to the streaming pipeline to track additional metadata about page entities that were being processed. Unfortunately this change had an unintended effect on the batch pipeline which caused notifications to not be fired upon successful processing for a subset of events. Thus event data that should have been transported to object storage to instead remain buffered indefinitely. # Resolution On Mar 11, 2026 the team investigated the issue and pinpointed it to a code change that was deployed on Feb 24, 2026. A fix was deployed on the same day as the issue was found. From there events from newly processed page entities began flowing to affected customer’s data warehouses again. Once the fix was in place the team focused on impact analysis and remediation. A backfill process was kicked off on Mar 12, 2026 to transport event data that did not originally make it to affected customer’s data warehouses. As of Mar 13, 2026 the backfill is complete and any gaps in event data have been filled. No further action is required from those impacted. # Process Changes and Prevention Our team has completed a thorough review of the incident, fixed the relevant bug, and implemented a backfill process for remediation here. We have also identified several areas of improvement with our metrics and monitoring, as well as system changes to prevent this kind of unintended cross-pipeline impact. We deeply regret this incident and invite any Fullstory customer who was materially affected to contact [[email protected]](mailto:[email protected]). We stand by ready to fully address all of your concerns.

1. 2026년 2월 27일 오후 9:53 UTCinvestigating

   We are currently experiencing delays in data processing in our EU1 data center. New sessions may not appear until several minutes later. Additionally, data passed to connected warehouses from Fullstory will be delayed. We are investigating the issue.

2. 2026년 2월 28일 오전 12:14 UTCmonitoring

   Our team has implemented a fix and the system is now catching up. We are addressing delays that began at the following times: - Analytics: 2026-02-27 20:00 UTC - Warehouse: 2026-02-25 03:00 UTC We are taking steps to ensure all data is processed accurately and will post another update once processing is back to normal.

3. 2026년 2월 28일 오전 12:52 UTCmonitoring

   The delay in processing for new sessions has been resolved and this system is now operating normally. We are continuing to work through the delays affecting data passed to connected warehouses from Fullstory. We will provide another update once all processing is back to normal.

4. 2026년 2월 28일 오전 4:23 UTCresolved

   This incident in the EU1 region has been resolved. All data processing delays have been cleared, and all systems are now operating normally.

1. 2026년 2월 25일 오후 4:20 UTCmonitoring

   We are currently monitoring an issue that is causing delays for warehouse customers located in our EU1 region. Customers may experience slower than normal processing times. We have identified the root cause and will continue to monitor the situation. We will provide another update as soon as we have more information.

2. 2026년 2월 25일 오후 7:36 UTCmonitoring

   We have implemented a fix for the issue that began around 18:40 UTC, which was causing delays for our warehouse customers in the EU1 region. The system is now catching up. Our team is taking steps to ensure all data is processed accurately, and we will post another update once all processing is back to normal.

3. 2026년 2월 25일 오후 10:06 UTCresolved

   Correction: The issue began at approximately 18:40 ET, not 18:40 UTC as previously stated. We have resolved the issue that was causing delays for our warehouse customers in the EU1 region, and all systems are now fully operational. Our team will continue to ensure all data is processed correctly. We appreciate your patience and apologize for any inconvenience this may have caused.

1. 2026년 2월 18일 오전 9:55 UTCresolved

   We experienced a service disruption in our EU1 region that impacted playback functionality from 8:45 AM to 9:37 AM UTC. We apologize for any inconvenience this may have caused.

1. 2026년 2월 13일 오후 8:54 UTCresolved

   From 1:04 PM to 1:26 PM EST, Fullstory experienced elevated error rates for session capture and some areas of the app. This also caused slight processing delays that lasted until about 2:00 PM EST. There was minimal to no data loss, and most failed requests were successful on retry.

1. 2026년 1월 23일 오후 5:09 UTCresolved

   From January 22, 2026 at 12:54 PM ET through January 23, 2026 at 11:26 AM ET, the Get Session API was unintentionally unavailable. Any requests to Get Session API may have incorrectly responded with HTTP 403 for valid calls. Fullstory customers that attempted to send an API request during the outage window will need to process these requests again. If you believe your account was impacted or have any questions, please reach out to [email protected].

1. 2025년 12월 19일 오후 5:19 UTCinvestigating

   Certain organizations are experiencing elevated server API error rates, specifically for the create event batch API.

2. 2025년 12월 19일 오후 8:09 UTCresolved

   This incident has been resolved.