Learnosity 장애 기록

1. 2026년 10월 26일 오후 6:05 UTCinvestigating

   17:50 UTC로, 우리는 모든 지역에서 아이템 풀에 영향을 미치는 문제를 경험하고 있습니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제점을 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

2. 2026년 10월 26일 오후 6:34 UTCinvestigating

   18:30 UTC 기준으로, 우리는 아이템 풀을 만들고 편집 할 때 등급의 성능을 조사하고 있습니다. 우리는 또한 로컬 장치 평가 패키지의 생성 및 편집에 영향을 미치는 as-of-yet unconfirmed tangential 문제점을 조사하고 있습니다. 저자, 평가 및 분석 스택의 다른 모든 API는 일반적으로 작용합니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제를 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

3. 2026년 10월 26일 오후 7:01 UTCinvestigating

   19:00 UTC 기준으로, Item Pool orx Local Device Assessment Package를 만들거나 편집할 때 성능이 향상되었습니다. 저자, 평가 및 분석 스택의 다른 모든 API는 일반적으로 작용합니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제를 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

4. 2026년 10월 26일 오후 7:31 UTCmonitoring

   19:30 UTC 기준, Item Pool 및 Local Device Assessment 생성은 약 30 분 동안 정상적인 처리량을 보였습니다. 우리는 이제 안정성을 보장하기 위해 모니터링의 추가 기간을 입력합니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제를 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

5. 2026년 10월 26일 오후 8:10 UTCresolved

   20:10 UTC의 경우, 우리는 Item Pool 또는 Local Device Assessment Package 생성 또는 업데이트로 더 많은 문제를 볼 수 없습니다. 우리는이 사건을 해결하고 급성 성능의 뿌리 원인을 조사하기 위해 계속됩니다. 학습 지원 및 시스템 엔지니어링 팀은 한 번에 게시물 mortem에 따라 수행됩니다. 루트 원인 분석 및 다음 단계 또는 예방 조치를 완료했습니다. 자주 묻는 질문.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 10월 20일 오후 2:16 UTCinvestigating

   13:20 UTC로, 우리는 미국 동부 1 지역의 향상된 수학 득점 서비스로 스코링 지연을 보았습니다. 데이터 손실이 관찰되지 않고 모든 세션은 궁극적으로 점수를 받았습니다. 노크에 효과로 Data API Scoring endpoint는 간헐적인 가용성 문제를 경험하고 있습니다. 승인 또는 평가 배송은 영향을받지 않습니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제점을 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

2. 2026년 10월 20일 오후 3:04 UTCinvestigating

   15:00 UTC 기준으로, 우리는 여전히 우리의 향상된 Math Scoring 서비스의 부족한 degraded 성능을 조사하지만 중요한 오류 또는 지연을 볼 수 없습니다. 스코링을 기다리는 세션의 backlog는 신속하고, 제출은 일반적으로 점수가됩니다. 다른 모든 API는 부패하지 않습니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제를 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

3. 2026년 10월 20일 오후 3:51 UTCmonitoring

   15:45 UTC의 경우, 완화 조치는 성공적으로 제출 queue를 명확하게 유지하고, scoring은 일반적으로 작동하도록 계속되었습니다. 고급 성능의 기간 동안받은 제출은 캡처되었으며 데이터가 손실되지 않았습니다. 다른 모든 API는 작동 상태를 유지. 우리는 뿌리 원인을 조사하고 우리의 조사가 완료되면 사건 postmortem을 공유합니다. 그 동안, 우리는 사건을 폐쇄하기 전에 지속적인 안정성을 확인하기 위해 추가 모니터링 기간을 입력하고 있습니다.

4. 2026년 10월 20일 오후 6:17 UTCresolved

   18:00 UTC로, 추가 문제없이 추가 모니터링의 긴 기간 후, 우리는 미국 동부 1에서 향상된 수학 득점에 영향을 미치는이 사건을 해결하고 있습니다. 학습 지원 및 시스템 엔지니어링 팀은 한 번에 게시물 mortem에 따라 수행됩니다. 루트 원인 분석 및 다음 단계 또는 예방 조치를 완료했습니다. 자주 묻는 질문.

5. 2026년 10월 27일 오후 1:27 UTCpostmortem

   ## Affected 시스템 및 지역 20 10월 2026일, Learnosity는 미국 동부 1에서 향상된 수학 득점에 영향을 미치는 지연 및 간헐적 인 요청 실패를 경험했습니다. 고객 영향은 대략 13:06 UTC에 시작하고 대략 41 분을 지속하는 13:47 UTC에서 끝납니다. 정상적인 처리 시간 보다는 더 긴 경험된 응어리를 빼고, Data API 득점 엔드포인트에 몇몇 직접적인 요구는 과실을 돌려주었습니다. 항목 API를 통해 저장 된 세션에 대해, 영향을받는 득점 요청은 누락하고 재조정되었으며, 점수 가용성을 스코링 실패보다는 지연 된 점수로 결과. 다른 지역의 수학 질문 및 서비스없이 평가는 비범했다. ## 투자 우리의 조사는 예상치 못한 높은 처리 시간에 결과 scoring 요청의 특정 패턴을 확인했습니다. 이것은 다른 득점 요구, 지연 및 간헐적인 timeouts를 가공하기 위하여 유효한 수용량을 감소시켰습니다. 이 문제는 특정 scoring 입력이 처리되었는지, 오히려 전반적인 요청 볼륨의 예상치 못한 증가. ## 해결책 Scoring 성능은 13:47 UTC에 의해 정상으로 반환됩니다. 15:12 UTC에서, Learnosity 엔지니어는 다른 고객에게 영향을 미치는 득점 작업 부하를 방지하기 위해 추가 트래픽 격리를 구현했습니다. ## 예방 영어 회화를 단기간에 완성하세요. 800 페이지가 넘는 레슨과 8000 개 이상의 오디오 클립을 다운로드 하시고, 자신을 영어 환경에 항상 노출 시키세요. 인터넷이나 컴퓨터에 상관없이 MP3 플레이어를 통해 언제나 영어를 듣고 따라할 수 있습니다. 영어회화 다운로드 페이지로 이동하시면 좀 더 자세한 내용을 알려드립니다. 더 많은 개선은 scoring 요청 검증을 강화하고, 더 넓은 서비스에 영향을 미치는 개인 scoring 워크로드를 방지하기 위해 노력합니다. 추가적인 고립 조치는 이러한 개선이 완료되고 그 효과는 확인됩니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 10월 8일 오후 1:40 UTCinvestigating

   UTC의 경우, 우리는 모든 지역에서 최신 LTS 릴리스에 일몰 API 버전 v2024.1.LTS를 다시 매핑하는 문제를 경험하고 있습니다. 이것은 여전히 3 월 2026 년 생활의 끝을 도달 한 v2024.1.LTS에 핀으로 핑 된 고객의 매우 작은 수에 영향을줍니다. 다른 API 버전이 영향을 미칩니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제점을 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

2. 2026년 10월 8일 오후 1:47 UTCinvestigating

   13:40> UTC로, 우리는 v2024.1.LTS 버전과 API를 캐시 업데이트로 접근 할 수 있습니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제를 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

3. 2026년 10월 8일 오후 2:20 UTCinvestigating

   14:00 UTC 기준으로 v2024.1.LTS는 미국 동부 1 (VA) 및 미국-서부-2 (OR)를 포함한 AMER 지역의 최신 LTS 버전으로 성공적으로 재 매핑됩니다. 모든 API는이 지역에서 액세스해야합니다. 우리는 현재 EMEA 및 APAC 지구에 재 매핑하고 있습니다. Active API 버전에 대한 직접 액세스는 작동 상태를 유지합니다. 이 문제는 이제 EMEA 및 APAC 지역의 마이크로 세트에 대한 버전의 리밍에만 영향을 미칩니다. 일몰 v2024.1.LTS API 버전에 핀이 붙은 EMEA 및 APAC 지역은 3 월 2026에서 EOL에 도달했습니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제를 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

4. 2026년 10월 8일 오후 3:15 UTCresolved

   15:00 UTC 기준으로 v2024.1.LTS는 모든 지역에서 성공적으로 재 매핑됩니다. 이 EOL/sunset 버전 끈을 사용하여 모든 API에 액세스해야합니다. 학습 지원 및 시스템 엔지니어링 팀은 한 번에 게시물 mortem에 따라 수행됩니다. 루트 원인 분석 및 다음 단계 또는 예방 조치를 완료했습니다. 자주 묻는 질문.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 10월 2일 오후 6:08 UTCinvestigating

   16:45 UTC로, 우리는 미국 동부 1 지역의 향상된 Math Scoring 서비스로 스코링 지연의 매우 낮은 숫자를 경험하기 시작했습니다. 모든 세션은 점수를 얻었으며 데이터 손실이 발생하지 않았습니다. 다른 API는 영향을받지 않습니다. 우리는 이 서비스에 한정된 지속적인 지연을 투자하고 있습니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제점을 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

2. 2026년 10월 2일 오후 6:30 UTCidentified

   18:30 UTC로, 우리는 향상된 Math Scoring 서비스에 대한 사기 문제를 확인하고 문제를 해결하기 위해 노력하고 있습니다. 모든 세션은 데이터 손실없이 캡처되고 다른 모든 API는 정상적으로 작동됩니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제를 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

3. 2026년 10월 2일 오후 7:10 UTCmonitoring

   19:00 UTC로, 우리는 몇 시간 동안 향상된 Math Scoring 서비스에 지연을 보았고 이제 모니터링 단계에 들어갑니다. 우리는 문제를 해결하기 전에 30 분 동안 계속 모니터링 할 것입니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제를 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

4. 2026년 10월 2일 오후 7:52 UTCresolved

   19:45 UTC로, 우리는 미국 동부 1 지역에서 향상된 수학 득점에 영향을 미치는 문제를 해결했습니다. 모든 API와 서비스는 일반적으로 작용하고 있습니다. 학습 지원 및 시스템 엔지니어링 팀은 한 번에 게시물 mortem에 따라 수행됩니다. 루트 원인 분석 및 다음 단계 또는 예방 조치를 완료했습니다. 자주 묻는 질문.

5. 2026년 10월 27일 오전 1:24 UTCpostmortem

   ## **시스템 및 지역** 에 2 10월 2026, Learnosity 경험있는 간헐적인 득점 오류 및 미국 동부 1의 수학 질문 유형에 영향을 미치는 타임 아웃. 고객 영향은 약 16:34 UTC로 시작하여 18:47 UTC로 종료되며 약 2 시간 및 13 분 동안 지속됩니다. 완벽한 요청은 실패하기 전에 최대 40 초의 지연을 경험할 수 있습니다. 다른 지역은 불완전했습니다. ##**투자** 이 사건은 수학 득점 서비스에 수요가 유효했던 수용량을 초과할 때 일어났습니다. 용량 및 스케일링 개선은 이전 사건에 따라 도입되었지만, 이 서비스는 규정 된에서 더 많은 인스턴스를 방지하는 네트워크 구성에 추가 한계를 발생했습니다. 영향은 계산 시간 제한이 무거운 부하에서 안정적으로 시행되지 않았기 때문에 예를 더 이상 점유하는 느린 계산에 의해 합성되었다. 이러한 조건은 수요를 처리하기 위해 충분한 스케일링 서비스를 방지했습니다. ## **해설 ** Learnosity 엔지니어는 추가 스케일링 용량과 네트워크 구성에 scoring 서비스를 이동하여 서비스를 복원하고 각 인스턴스에 할당 된 컴퓨팅 리소스를 증가시킵니다. 이러한 변경은 18:47 UTC에 의해 삭제 된 오류와 함께 복구 할 수있는 서비스를 허용했습니다. ##**이전** Learnosity는 무거운 짐의 밑에 천천히 계산을 종결하기 위하여 계산 시간 한계 강제적인 기계장치를 새롭게 했습니다. 건강 검사는 불필요한 인스턴스 교체를 방지하기 위해 계산 처리에서 분리되었습니다. 이 서비스는 추가적인 compute 및 네트워크 용량으로 작동합니다. 더 많은 건축 개선은 미래 스케일링을 제한하여 네트워크 용량을 방지하기 위해 하부로, 우리는 비정상적으로 느린 계산의 영향을 줄이기 위해 더 짧은 계산 시간 제한을 평가하고 있습니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 9월 11일 오후 6:19 UTCinvestigating

   17 : 00 UTC로, 우리는 현재 미국 동부 1 지구에 scoring과 미성년자 간담 문제를 조사하고 있습니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제점을 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

2. 2026년 9월 11일 오후 6:30 UTCinvestigating

   18:30 UTC로, 모든 API는 작동을 유지하고 우리는 효율성을 모니터링하고 있습니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제를 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

3. 2026년 9월 11일 오후 7:42 UTCidentified

   19:30 UTC 기준으로, 우리는 3개의 강화한 수학 질문 유형의 득점을 가진 sporadic 문제이기 위하여 이것을 확인했습니다. 이 득점 시도는 재량에 성공하고 다른 APIs는 영향을 미치지 않습니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제를 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

4. 2026년 9월 11일 오후 8:53 UTCmonitoring

   20:45 UTC로, 우리는 초기 EMS scoring 요청과 retries의 취급을 개량하고 지금 60 분 감시 단계에 들어갈 것입니다. 모든 API는 정상적으로 기능을 계속합니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제를 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

5. 2026년 9월 11일 오후 9:44 UTCresolved

   21:45 UTC로, 향상된 Math Scoring 질문 유형 응답은 미국 동부 1 지역에서 일반적으로 득점하고, 모든 API는 일반적으로 작용하고, 우리는이 사건을 닫고 있습니다. 학습 지원 및 시스템 엔지니어링 팀은 한 번에 게시물 mortem에 따라 수행됩니다. 루트 원인 분석 및 다음 단계 또는 예방 조치를 완료했습니다. 자주 묻는 질문.

6. 2026년 10월 27일 오전 1:23 UTCpostmortem

   ## **시스템 및 지역** 에 11 9월 2026, Learnosity 경험 간헐적인 득점 오류 및 타임 아웃은 미국 동부 1의 수학 질문 유형에 영향을 미치는. 고객 영향은 약 17시 UTC로 시작하여 21시 45분 UTC로 종료되며 약 4시간 45분 동안 지속됩니다. 다른 질문 유형은 일반적으로 점수를 계속하고 다른 지역은 비공식적이었습니다. 가장 실패한 요청은 retried 때 성공했습니다. ##**투자** 이 사건은 추후에 수요 증가를 따르는 수학 득점 서비스의 충분한 수용량에 기인했습니다. 이 서비스는 구성 된 최대 인스턴스 수에 도달하고 부하를 수용하기 위해 더 스케일 할 수 없습니다. 이것은 인스턴스당 제한된 compute 리소스에 의해 합성되고 더 이상 대상을 차지하는 값이 느린 계산. 무거운 짐의 밑에, 계산 시간 한계는 믿을 수 없을 것이고, 바쁜 인스턴스는 건강 검사를 실패하고, 더 유효한 수용량을 감소시킬 수 있었습니다. ## **해설 ** 학습 엔지니어는 실행 인스턴스의 수를 늘리고 최대 인스턴스 제한을 올리면서 서비스를 복원했습니다. 정상 득점 작업은 21:45 UTC에 의해 재개, 그리고 모든 나머지 누적 득점 요청은 23:17 UTC에 의해 처리되었습니다. 9 월 14 일, 추가 구성 변경은 계산 자원을 증가시키기 위해 배포되었으며 자동 스케일링을 개선하고 불필요한 인스턴스 교체를 줄일 수 있었습니다. ##**이전** Learnosity는 스코링 용량을 증가시키고, 인스턴스 제한을 올리고, 프로세서 활용을 기반으로 자동 스케일링을 개선했습니다. 건강 검사는 잘못된 것으로 식별 된 잘못된 실수로 바쁜 인스턴스를 방지하기 위해 계산 처리에서 분리되었습니다. 우리는 또한 자원 활용, 응답 시간 및 득점 오류에 대한 계산 시간 제한 시행 및 확장 모니터링 및 경고를 개선했습니다. 더 많은 작업은 네트워크 스케일링 용량을 강화하고 더 짧은 계산 시간 제한을 평가 할 수 있습니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 7월 24일 오후 12:16 UTCinvestigating

   우리는 미국-서부-2(OR) 지역에 영향을 미치는 지속적인 AWS 문제점을 알고 있으며, 이는 Learnosity Authoring 환경에 접근할 때 간헐적인 연결 문제를 일으킬 수 있습니다. 우리는 적극적으로 상황을 모니터링하고 그들은 사용할 수로 업데이트를 제공 할 것입니다. 이 시간에, Learnosity 고객에 영향을 미치는 영향은 간결하고 권한으로 제한됩니다. 평가 및 기타 API는 일반적으로 behaving입니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제점을 조사하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

2. 2026년 7월 24일 오후 12:28 UTCmonitoring

   11:02 UTC로, Learnosity는 Learnosity 저자 사이트에 액세스 할 때 간헐적 인 타임 아웃의 모니터 경고를보고 시작했습니다. 다른 기능은 영향을 받지 않고, Learnosity 경고는 11시 11분 UTC로 중단되었습니다. 11:40 UTC에서 AWS는 US-West-2 (OR)에 degraded 서비스를 보고했습니다. 11:56 UTC에서 AWS는 시스템 복구를보고했습니다. 학습 지원 및 시스템 엔지니어링 팀은 적극적으로 문제를 모니터링하고 가능한 한 빨리 업데이트 및 해결과 함께 수행됩니다.

3. 2026년 7월 24일 오후 12:35 UTCmonitoring

   12:30 UTC에서 AWS는 그들이 확인하고 문제점을 수정했다고보고, 그 복구가 계속된다. 학습 지원 및 시스템 엔지니어링 팀은 30 분 이상의 문제를 모니터링하고 가능한 한 빨리 업데이트 및 해결과 함께 따라질 것입니다.

4. 2026년 7월 24일 오후 1:02 UTCresolved

   UTC의 경우, 우리는이 문제를 해결합니다. AWS 시스템은 복구 및 Learnosity 인증은 50 분 동안 비공개되었습니다. 학습 지원 및 시스템 엔지니어링 팀은 한 번에 게시물 mortem에 따라 수행됩니다. 루트 원인 분석 및 다음 단계 또는 예방 조치를 완료했습니다. 자주 묻는 질문.

5. 2026년 7월 24일 오후 1:03 UTCpostmortem

   ## Affected 시스템 및 지역 2026-07-24에서 Learnosity는 AWS US-West-2 \ (OR\) 지역의 네트워킹 사고로 인해 Learnosity 저자 사이트에 액세스하는 간단한 간헐적 인 타임 아웃을 경험했습니다. (https://health.aws.amazon.com/health/status?eventID=arn:aws:health:us-west-2::event/MULTIPLE SERVICES/PLEAWS MULTI SERVICES PLE PLEAWS MULTI SERVICES E E VICESERVICES E VICESERVICES. Learnosity 고객 영향은 대략 11:02 UTC에 시작하고 11:11 UTC에 끝납니다. AWS는 그 후 12시 12분 UTC에서 지역 사건을 일시적으로 폐쇄했습니다. Learnosity 고객에게 미치는 영향의 전체 기간은 약 9 분입니다. ###투자 Learnosity는 OR 지역에서 서비스를 제공하는 11:02 UTC의 내부 경고를 즉시 조사를 시작했다. 초기 조사는 Learnosity 서비스 평가의 증거가 없습니다. 더 많은 조사는 Learnosity 저자 사이트에 액세스하는 간헐적 인 타임 아웃을 식별했습니다. AWS에서 네트워크 인프라에 미치는 영향에 대한 문제점이 확인 및 구제가 시작되었습니다. ### 해결책 Learnosity 저자 사이트에 영향을 미치는 간헐적 인 타임 아웃은 11 : 11 UTC에 의해 중단되었으며 AWS는 11 : 15 UTC의 문제를 효과적으로 해결했습니다. AWS는 11:59 UTC에 의해 해결 된 간단한 재구성 이벤트를보고 공식적으로 12:12 UTC의 사건을 닫습니다. ### 예방 이 사건에 기여하지 않는 학습 시스템. 내부 감시 및 경고는 예상대로 행동한다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 4월 17일 오후 4:06 UTCinvestigating

   As of 16:00 UTC, we are currently experiencing a slowdown in availability of newly submitted submitted session results in the us-east-1 region. There is no data loss, all saves and submissions are being queued as designed, and the assessments stack remains operational. Delays may currently be experienced using the Data API to fetch session results, and the Reports API to display results. Other workflows that rely on the synchronous use of Data API results may also be affected. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

2. 2026년 4월 17일 오후 4:21 UTCidentified

   As of 16:20 UTC, Learnosity has increased database performance and we are processing queued sessions rapidly. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

3. 2026년 4월 17일 오후 4:28 UTCmonitoring

   As of 16:25 UTC, we have resolved the slowdown in processing session results and all sessions are being scored normally. We will continue to monitor for a further 30 minutes before closing this incident report. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

4. 2026년 4월 17일 오후 5:58 UTCresolved

   As of 16:55 UTC, we are have resolved the degraded performance issue affecting availability of session data in us-east-1. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required. Please reach out if you have any questions or concerns.

5. 2026년 5월 4일 오후 5:23 UTCpostmortem

   ### Affected Systems and Regions On 2026-04-17, Learnosity experienced a service degradation impacting availability of newly submitted session results for a small subset of customers in US-East-1. The issue began at approximately 13:54 UTC and was resolved at 16:24 UTC. The total duration of customer impact was approximately 2.5 hours. ### Investigation Service degradation was detected following the accumulation of submitted sessions in the async scoring queue, This was caused by elevated CPU utilization when heavy load coincided with a background data migration task that had been running for several days on the affected EC2 instance. This reduced message processing throughput, leading to queueing of sessions.  ### Resolution Learnosity CloudOps engineers responded by scaling session processing node EC2 instances, and corresponding database proxies, to maximum allowable capacity. The migration task was also stopped, which further reduced CPU pressure and allowed message processing throughput to recover. The session scoring backlog was fully cleared by 16:24 UTC. ### Prevention Learnosity is implementing the following measures to mitigate: * Pre-flight health checks and safeguards are being introduced to prevent migration tasks from running during busy periods of production usage. * Enhanced monitoring is being applied to detect backlog growth and database contention, and apply remedial action more quickly

1. 2026년 4월 16일 오후 2:08 UTCinvestigating

   As of 13:30 UTC, we are currently experiencing slow downs affecting the Live Progress report in the us-east-1 region. Atypical use of eventbus may also be affected, such as custom reports/implementations. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

2. 2026년 4월 16일 오후 3:23 UTCresolved

   As of 14:30 UTC, the Live Progress report event degradation issue in the us-east-1 region has been resolved. Additional capacity processed the event load and these systems are operating normally. Brief additional latency during this scaling may have been introduced for users of Learnosity's premium Firehose feature, but we've seen no evidence that this affected customers. This, too, was resolved once scaling was complete. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required. Please reach out if you have any questions or concerns.

3. 2026년 5월 4일 오후 2:39 UTCpostmortem

   ### Affected Systems and Regions On 2026-04-16, Learnosity experienced a service degradation impacting the Live Progress report for a small subset of customers in the us-east-1 region. The issue began at approximately 13:12 UTC and was resolved at 14:40 UTC. The total duration of customer impact was approximately 88 minutes. ### Investigation The issue was detected following elevated error rates on the load balancer serving the eventbus service. Investigation determined that an unhealthy condition within the EC2 instances led to elevated CPU utilization and memory pressure within the Auto Scaling Group, resulting in application instability and repeated service restarts. Inconsistent recovery caused traffic to concentrate on a subset of hosts, further elevating error rates. A secondary effect of the instability was increased request pressure on downstream dependencies. ### Resolution Service was restored by stabilizing the affected EC2 instance and restoring consistent application availability across the Auto Scaling Group. Load distribution normalized once all instances returned to a healthy state. ### Prevention Learnosity is implementing the following measures to mitigate: * Improve service startup and dependency handling to ensure consistent recovery behavior * Review resource thresholds to reduce the likelihood of similar instability * Enhance monitoring to detect and respond more quickly to similar conditions

1. 2026년 2월 4일 오후 6:34 UTCidentified

   As of 18:30 UTC, we are currently experiencing an issue affecting assessment delivery in VA (us-east-1). Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

2. 2026년 2월 4일 오후 6:55 UTCmonitoring

   As of 18:52 UTC, the Learnosity System Engineering team has put a fix in place. We will continue to monitor for some time to ensure a stable recovery, but all services are operational again. Learnosity Support and Systems Engineering teams will continue to actively monitor the issue, and will follow on with an update and resolution as soon as possible.

3. 2026년 2월 4일 오후 7:15 UTCresolved

   As of 19:12 UTC, traffic has returned to normal and everything remains stable. We have resolved the incident affecting assessments in VA (us-east-1). Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required. Please reach out if you have any questions or concerns.

4. 2026년 2월 10일 오후 5:08 UTCpostmortem

   ### **Affected Systems and Regions** On 2026-02-04, Learnosity experienced an availability incident impacting Questions API access for customers in the AMER region. The issue began at approximately 18:02 UTC and was formally declared resolved at 18:51 UTC. The total duration of customer impact was approximately 49 minutes. ### **Investigation** The issue was first detected when the Items API attempted to initialize the dependent Questions API. The incident was traced to a misconfiguration within the content delivery network servicing the AMER region, causing a SSL certificate error.. During a certificate management task, the distribution responsible for serving [_questions.learnosity.com_](http://questions.learnosity.com) was configured with an SSL certificate that did not match the _\*.learnosity.com_ domain nomenclature. This caused browsers and client systems to reject connections to the affected subdomain. ### **Resolution** Once the issue was identified, the response team updated the affected distribution to use the correct SSL certificate. Service was fully restored after the change was completed. ### **Prevention** Learnosity is strengthening certificate deployment and validation processes to ensure that domain mappings are verified prior to rollout. Additional safeguards have been added to prevent cross-domain certificate assignments and to improve monitoring for sudden traffic drops that may indicate certificate-related failures.

1. 2026년 1월 12일 오후 5:06 UTCinvestigating

   As of 4:30 UTC, we are currently investigating an issue that may result in temporary session unavailability for a small number of US-only customers. We can confirm that no session data has been lost, and we are working to restore the affected sessions as quickly as possible. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

2. 2026년 1월 12일 오후 5:52 UTCidentified

   As of 5:30 UTC, we've identified and corrected a load balancing configuration affecting database reads and writes for a very small number of US-only customers with consumers in the US-East-1 region. - All new sessions will behave as expected. - Read requests seeking data on sessions created prior to 23:00 UTC yesterday will also now behave correctly. - We are working on restoring sessions recently affected While all APIs continue to function normally, this update identifies service areas that may temporarily have limited access to recently saved or submitted sessions. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

3. 2026년 1월 12일 오후 7:10 UTCresolved

   As of 7:00 UTC, we've concluded our monitoring and confirmed that all new requests are behaving as expected. We are closing this record and will continue to work with affected customers directly to confirm that all recent sessions are accessible. Please reach out if you have any questions or concerns.

1. 2025년 10월 20일 오후 2:53 UTCmonitoring

   Learnosity services in VA are continuing to be partially impacted by the major AWS outage. AWS first reported the incident at 7:11 UTC As of 14:50 UTC they are continuing to work on its full resolution, but have again set their impact level to "Degraded". The AWS Health Status dashboard where updates are being posted by the AWS team can be found here: https://health.aws.amazon.com/health/status Overall, the error rate has fallen and has stayed below what we saw earlier in the day and we have been successful in slowly scaling up the infrastructure, but we're still seeing some issues stemming from the the ongoing AWS outage as we attempt to bring online some new instances, and we have begun to see an uptick in errors. These have included errors associated with Data API, Feedback Aide, and the delivery of adaptive assessments, delays in scoring and video encoding, as well as other smaller spikes across different systems. While the overall error rate remains low, Learnosity Support and Systems Engineering teams are actively monitoring the outage and manually intervening where possible to mitigate the issues we're facing. In spite of our best efforts it is likely that the APIs may not be fully operational in VA until AWS's service is fully restored to normal operation, but we are taking every step within our power to try to minimize the disruption to our services.

2. 2025년 10월 20일 오후 3:30 UTCinvestigating

   As of 15:25 UTC, we are seeing increasing error rates. Data API is the most affected service at the moment and we are seeing an increase in scoring delays, but we are continuing to score incoming messages to the best rate our current capacity allows. AWS is still reporting a severity of "Degraded", but now with an increased count of 81 affected services. Learnosity Support and Systems Engineering teams are continuing to monitor the issue, and will continue our mitigation efforts wherever possible.

3. 2025년 10월 20일 오후 6:13 UTCmonitoring

   As of 19:10 UTC, we are beginning to see errors rates diminish and some Learnosity services are beginning to scale again. AWS is still reporting a severity of "Degraded" with problems to 104 of their services in us-east-1. Learnosity Support and Systems Engineering teams are continuing to monitor and manually intervene when possible to mitigate the effects of the outage.

4. 2025년 10월 20일 오후 7:02 UTCmonitoring

   As of 20:00 UTC, the error rate has continued to drop, systems remain scaled and the backlogs of scoring messages continues to drop. AWS is still reporting 109 impacted services and a severity of "Degraded" Learnosity Support and Systems Engineering teams are continuing to monitor the issue and are manually adjusting the infrastructure where needed to ensure stability.

5. 2025년 10월 20일 오후 7:32 UTCmonitoring

   As of 19:30 UTC, the error rate has fallen to 0.1% and continues to fall, we continue to steadily process the backlog of scoring messages. As of their latest update AWS has downgraded the severity of outage to "Impacted" but still list 109 services as affected. Learnosity Support and Systems Engineering teams are continuing to actively monitor the issue, and will follow on with an update and resolution as soon as possible.

6. 2025년 10월 20일 오후 8:44 UTCmonitoring

   As of 20:40 UTC, the error rate remains low, the infrastructure is scaled to meet demand, and the scoring queues are empty and new submissions are being processed as they come in. All services are operational and we are processing the remaining backlog of messages from the incident. AWS still lists 93 affected services and a severity of "Impacted". Learnosity Support and Systems Engineering teams are continuing to monitor the issue, but at the moment all systems are fully operational.

7. 2025년 10월 20일 오후 9:35 UTCresolved

   As of 21:30 UTC, all systems keep operating normally, so we are resolving the status page about the AWS outage affecting Learnosity Services in the VA (us-east-1) region. Learnosity Support and Systems Engineering teams will continue to monitor the systems closely while AWS continues to work on the issues on their side. Please reach out if you have any questions or concerns.

1. 2025년 10월 20일 오전 10:11 UTCmonitoring

   AWS is going through late-stage recovery of a major outage in US-East-1 bringing down most of their services. Learnosity APIs were stable and unaffected in isolation, but some impact cascaded to Learnosity (such as availability of assets from CloudFront). The full extent of the incident is unknown at this time. NOTE: Up until the time of this writing (10:05 UTC), this included impact to sign-in services that prevented us from accessing our StatusPage. We immediately posted banner notifications in our Support Portal, and those banners will remain visible until we've been able to verify that service has been restored for sufficient time. Incident History: As of 7:00 UTC, AWS experienced a major outage. While Learnosity APIs were fully operational, some Learnosity services use AWS and were affected. These included access to our Support ticketing system (the Learnosity-hosted Help guides were unaffected), and availability of CloudFront-hosted assets, but other systems may have been impacted as well. As of 10:00 UTC, AWS has restored services and access to Learnosity-adjacent features is becoming available relatively quickly. Learnosity Support will monitor AWS for a further 30 minutes before calling this issue resolved. We will follow on with an update and resolution at that time, or sooner.

2. 2025년 10월 20일 오전 11:23 UTCresolved

   As of 11:20 UTC, AWS services appear to have been stable for approximately an hour. Learnosity APIs--including services affected by the AWS outage (such as CloudFront)--appear to be fully operational. At this time we are labeling this issue resolved. Learnosity Support and Systems Engineering teams will continue to investigate impact from this incident and follow up with a post mortem once we hear from AWS. Please reach out if you have any questions or concerns.

1. 2025년 10월 2일 오후 9:34 UTCinvestigating

   As of 21:15 UTC, Learnosity APIs are being impacted by a temporary outage from our Math partner, Desmos. Authoring or Editing content with Desmos questions or features, or initializing assessments that contain Desmos questions or features, can prevent the Author API or Items API from initializing. Specific reports that display question content, such as the Session Detail by Item report, can also fail to initialize Learnosity APIs are not otherwise impacted, so content that does not include Desmos features are not affected. Until we learn more, we will list APIs that may experience knock-on effects from this temporary Desmos outage. However, as most customers will not experience any issues, we will not list these APIs as down. Desmos and Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

2. 2025년 10월 2일 오후 10:14 UTCmonitoring

   As of 22:12 UTC, Desmos services have been restored and Learnosity APIs are initializing without issue. Desmos and Learnosity Support and Systems Engineering teams will continue to monitor for a further 30 minutes before declaring this issue resolved. We will follow on with an update and resolution as soon as possible.

3. 2025년 10월 2일 오후 10:43 UTCresolved

   As of 22:42 UTC, all Learnosity APIs are continuing to function as expected in all regions, and we are declaring this issue resolved. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required. Please reach out if you have any questions or concerns.

1. 2025년 8월 19일 오후 4:11 UTCinvestigating

   As of 3:40 UTC, we are experiencing degraded performance in our analytics stack affecting the availability of session data in US-East-1 for a subset of customers. This is affecting the Reports API and Data API. Neither authoring nor assessment stacks are affected, and there is no data loss. Submitted sessions are queueing for processing. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

2. 2025년 8월 19일 오후 4:35 UTCinvestigating

   As of 4:30 UTC, we are continuing to investigate degraded performance in our Data and Reports APIs. Only availability of recently submitted session data is affected. Historic session data, as well as all other API stacks, remain unaffected. New submissions are persisting correctly with no data loss and these submission are being queued for scoring. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

3. 2025년 8월 19일 오후 5:34 UTCidentified

   (Note: We're correcting cited UTC times to the 24 hr format and will include both forms in this update only.) As of 5:30pm/17:30 UTC, we've identified a possible contributing cause for the degraded performance in our analytics stack. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

4. 2025년 8월 19일 오후 6:42 UTCidentified

   As of 18:30 UTC, initial remediation efforts have tripled the rate of queued session processing and we are continuing to work toward a full resolution. Access to recently submitted session results via the Data API and Reports API remains the only affected part of the Learnosity ecosystem. New submissions continue to be safely queued for scoring while the degraded performance remains. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

5. 2025년 8월 19일 오후 7:56 UTCidentified

   As of 19:30 UTC, we are now processing queued sessions rapidly and more than half of the backlog has already cleared.. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

6. 2025년 8월 19일 오후 8:35 UTCidentified

   As of 20:30 UTC, the scoring queue backlog is almost empty and new sessions will soon be processed without delay. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

7. 2025년 8월 19일 오후 9:05 UTCmonitoring

   As of 21:00 UTC, all sessions have been cleared from the scoring backlog queue and all submissions are being processed normally. Learnosity Support and Systems Engineering teams will monitor this situation for a further 30 minutes before calling it resolved.

8. 2025년 8월 19일 오후 9:31 UTCresolved

   As of 21:30 UTC, we are have resolved the issue affecting availability of session data in US-East-1. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required. Please reach out if you have any questions or concerns.

9. 2025년 9월 19일 오후 7:14 UTCpostmortem

   ### Affected Systems and Regions On 19 August 2025, Learnosity experienced degraded performance in our analytics stack, affecting session data availability in US-East-1 for a subset of customers. This affected the Reports API and Data API, with no other stacks affected, and no data loss.  ### Investigation Monitoring detected a rapid increase in unprocessed and retried messages, along with elevated lock contention in the sessions database. The root cause was traced to a customer implementation issue generating an extraordinarily high number of submissions. This drove excessive retries, magnifying actual traffic volume.  The use of time-ordered v7 UUIDs for session IDs, normally handled without issue, became problematic under this contention. Uniqueness checks on each session ID required more resources and triggered a succession of temporary deadlocks. These deadlocks would usually self-resolve, but the amplified traffic prevented recovery, turning a minor issue into a sustained queue blockage. ### Resolution Once the issue was identified, Learnosity moved the customer to a dedicated, isolated sync queue, preventing cross‑tenant impact while we investigated. We applied targeted rate limits for the isolated service to protect the database, and drained the backlog. Where safe, long‑running queries were terminated to free locks and allow forward progress.  To support faster diagnosis, Learnosity enabled detailed deadlock logging and expanded metrics around message retries, abandonment, and per‑session activity. Learnosity also worked with the customer to adjust implementation settings, reducing combined saves and submits by two orders of magnitude. Session IDs were also switched to v4 UUIDs which simplified uniqueness checks further preventing deadlocks. Immediately after these changes were put into use, queues began to rapidly recover, and normal processing resumed. Most sessions for the subset of affected customers saw short delays, while the most significantly delayed session took ~6 hours before final persistence. Throughout, we identified no data loss. ### Prevention To prevent recurrence, we are: * Implementing targeted load tests and contention simulations to replicate high-parallelism patterns. * Reviewing customer identifier schemes for session IDs and auditing usage across all customers \(initial checks confirm none of our Top 50 customers currently use v7 UUIDs\). * Analyzing adoption of per-tenant fair-use queues \(or equivalent fair-share policies\) to cap burst throughput from a single tenant and protect shared infrastructure.

1. 2025년 8월 8일 오후 1:55 UTCinvestigating

   As of 13:40 UTC, we began investigating a possible issue affecting manual update jobs to session response scores, sessions statuses, and session metadata in the US-East-1 (VA) region. We saw a brief buildup in the queue for these jobs, the queue processed as expected, and is now empty. However, we're investigating further, out of an abundance of caution, to see if there are any unexpected causes for the short queue buildup. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

2. 2025년 8월 8일 오후 1:57 UTCinvestigating

   As of 13:55 UTC, all manual update jobs have been executed and the jobs queue is empty. Requests to manually update session response scores, session statuses, and session metadata should execute as expected. We are still investigating possible reasons for the blip. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

3. 2025년 8월 8일 오후 2:19 UTCresolved

   As of 14:10 UTC, we've found no negative behavior in the manual update processes and attribute the short burst of jobs to uncharacteristic testing of the new manual grading workflow. The jobs queue functioned as expected, ensuring no loss of requests, and the jobs were processed in a timely fashion. Learnosity will continue to monitor this activity for a further 60 minutes, but we are calling this resolved at this time. We apologize for the unwarranted alert. Please reach out if you have any questions or concerns.

4. 2025년 8월 8일 오후 2:19 UTCpostmortem

   At 13:40 UTC on 8 August 2025, we noticed a brief backup of asynchronous jobs affecting manual session updates in the US-East-1 \(VA\) region. Possible jobs including manual updates to response scores, session statuses, and session metadata. The queue mechanism worked as expected, and all requests were processed in short order. Out of an abundance of caution, we raised an alert while systems remained operational, but found nothing of concern during our investigation. We hope the unwarranted alert was not a significant distraction.

1. 2025년 3월 3일 오후 1:48 UTCinvestigating

   As of 13:45 UTC, we are experiencing delays in availability of session information in US-East-1 Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

2. 2025년 3월 3일 오후 2:00 UTCinvestigating

   As of 14:00 UTC, we are still investigating an issue affecting the availability of session information in US-East-1. Both Data API and Reports API are experiencing extended delays and intermittent failures in returning session results. Authoring and Assessment APIs appear to be unaffected at this time and all session submissions are being successfully queued. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

3. 2025년 3월 3일 오후 3:06 UTCinvestigating

   As of 15:00 UTC, we are continuing to investigate the issue. Data and Reports API are still impacted. Assessment and Authoring APIs remain unaffected. Assessment submissions are being queued and will be processed as the system works through any backlog queue. Note that if the Data API or Reports API is used in a preliminary/synchronous assessment delivery workflow, this will likely have a knock-on affect when initializing assessments. Initializing any assessment API directly is unaffected at present. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

4. 2025년 3월 3일 오후 3:20 UTCmonitoring

   As of 15:15 UTC, all services have been restored. All queued messages have been processed and session information should now be available. Users of the Learnosity Firehose service will still see slight delays as messages are sent out for recently processed events. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

5. 2025년 3월 3일 오후 3:54 UTCresolved

   As of 15:45 UTC, after a further 30 minutes of monitoring, we are resolving this issue. All services remain fully operational. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalised any next steps or preventative measures required. Please reach out if you have any questions or concerns.

6. 2025년 3월 14일 오후 4:31 UTCpostmortem

   ### Affected Systems and Regions On March 3, 2025, we experienced a temporary service slowdown impacting the availability of session information in the US-East-1 region. The issue began at 13:19 UTC and was resolved by 15:15 UTC. The Data API and Reports API experienced delays and intermittent failures when returning session results. Authoring and Assessment APIs were unaffected with no data loss. ### Investigation We identified a recently introduced inefficient database query was causing unnecessary strain when handling extremely large data sets. This led to delayed responses to the Data API sessions endpoints. This further led to delays or unavailability in select reports via the Reports API. Additionally, some timeouts and safeguards that should have triggered earlier did not activate as expected. ### Resolution To quickly restore performance: * We optimized the affected queries, improving efficiency and reducing database load. * We scaled up the impacted systems to immediately process the backlog. * We fine-tuned system timeouts and connection limits to prevent similar issues in the future. Following these fixes, all services resumed normal operations. ### Prevention To prevent future occurrences: * We have permanently updated our database handling methods to ensure more efficient query execution. * We are enhancing automated monitoring to detect and respond to database slowdowns before they impact customers. We appreciate your patience and remain committed to delivering a seamless experience.

1. 2025년 1월 9일 오후 1:33 UTCinvestigating

   As of 13:30 UTC, We are currently investigating a possible performance degradation affecting Reports API and Data API results in US-East-1. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

2. 2025년 1월 9일 오후 2:02 UTCinvestigating

   As of 14:00 UTC, Learnosity is working on identifying the cause of degraded performance in our Analytics APIs, Reports and Data. Authoring and Assessment stacks remain unaffected Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

3. 2025년 1월 9일 오후 2:13 UTCmonitoring

   As of 14:10 UTC, Learnosity has restored full service for all new requests to all Analytics APIs. We are monitoring stability and analyzing all queued requests. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

4. 2025년 1월 9일 오후 2:50 UTCresolved

   As of 14:50 UTC, after a further 60 minutes of issue-free monitoring, we are resolving this issue affecting availability of session information via our Analytics stack. Reports API and Data API are fully operational. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required. Please reach out if you have any questions or concerns.

5. 2025년 1월 14일 오후 5:25 UTCpostmortem

   **Affected Systems and Regions** On 2025-01-09, Learnosity experienced a brief partial outage affecting our analytics stacks, specifically the Reports API and Data API in the AMER region. The issue began at 13:30 UTC and was resolved at 14:06 UTC, lasting 36 minutes. All other API's were unaffected and there was no loss of data. **Investigation** We discovered that a large number of atypical, inefficient Data API queries requested by customers were taking too long to complete. This prevented other queries from running in a timely manner, creating a backlog. It was determined that an additional database index would significantly improve the response times of these types of queries. **Resolution** Immediately upon discovering this issue, the impacted database instances were successfully scaled up to ease the backlog of requests. The additional index was implemented and all remaining queries were processed quickly. Affected APIs returned to normal operations, and further monitoring ensured the issue was fully resolved. **Prevention** Following further testing, the new index is working well and has become a permanent part of the system. We are also adding new automated monitoring and regression testing to ensure similar requests perform as expected.

1. 2024년 10월 1일 오후 3:09 UTCinvestigating

   As of 15:00 UTC, we are currently investigating an issue affecting the event delivery that powers the live progress report (live-activitystatus-by-user) in VA (us-east-1). Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

2. 2024년 10월 1일 오후 3:25 UTCidentified

   As of 15:20 UTC, the Systems Engineering team has identified the cause for the slower than normal delivery of events and is proceeding with mitigation steps. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

3. 2024년 10월 1일 오후 3:40 UTCmonitoring

   As of 15:35 UTC, the Systems Engineering team has deployed a fix, the errors have stopped, and currently all systems are functional. We will continue to monitor for a time before resolving the incident but we are currently seeing no errors. We will follow on with an update and resolution as soon as possible.

4. 2024년 10월 1일 오후 4:07 UTCresolved

   As of 16:05 UTC, We are have resolved the issue affecting live progress reports in VA (us-east-1). As the recovery remains stable, we are now resolving the incident that was affecting live progress reports. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalised any next steps or preventative measures required. Please reach out if you have any questions or concerns.

1. 2024년 9월 13일 오후 2:35 UTCinvestigating

   As of 14:30 UTC, We are investigating a potential issue affecting the delivery of some Events API events (used in the live proctoring / live-activitystatus-by-user report) in us-east-1 (VA). Assessments are otherwise proceeding normally and can be saved and submitted without any issue. This issue if confirmed would only affect the delivery of Events API events. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

2. 2024년 9월 13일 오후 3:09 UTCidentified

   As of 15:00 UTC, the Learnosity Systems Engineering team has identified the issue preventing the assessment events from being delivered and are working on the implementation of remediation steps. Learnosity Support and Systems Engineering teams are continuing to work on the issue, and will follow on with an update and resolution as soon as possible.

3. 2024년 9월 13일 오후 3:34 UTCmonitoring

   As of 15:30 UTC, events are flowing normally once again and all systems are operational. We will continue to monitor for some time before we resolve the issue fully to ensure we have a stable recovery.

4. 2024년 9월 13일 오후 4:03 UTCresolved

   As of 16:00 UTC, after monitoring and confirming the stable state of the fixes put into place, we have resolved the Events API events delivery issue affecting proctored assessments in us-east-1 (VA). Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalised any next steps or preventative measures required. Please reach out if you have any questions or concerns.

1. 2024년 5월 22일 오후 6:27 UTCinvestigating

   As of 18:10 UTC, we are trying to determine if an issue exists causing a very small number of adaptive sessions to not trigger a Firehose event after scoring in US-East-1. This is just an internal investigation at present, without confirmation, and we have no evidence that any impact outside the described implementation is affected. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

2. 2024년 5월 22일 오후 6:33 UTCinvestigating

   As of 18:30 UTC, Learnosity Support and Systems Engineering teams are continuing to investigate this unconfirmed issue and will follow on with an update as soon as possible.

3. 2024년 5월 22일 오후 6:47 UTCinvestigating

   As of 18:45 UTC, Learnosity Support and Systems Engineering teams are continuing to investigate an unconfirmed issue, examining less than 500 potentially affected sessions out of 4.93 million submitted over the same period. We will follow on with an update as soon as possible.

4. 2024년 5월 22일 오후 7:01 UTCmonitoring

   As of 19:00 UTC, we have been unable to find any systemic problems, while the number of potentially impacted sessions remains at one hundredth of one percent ( 0.01%) of total submissions over the period investigated, likely due to a content issue. Learnosity Support and Systems Engineering teams will continue to monitor for 30 minutes and expect to close this investigation at that time.

5. 2024년 5월 22일 오후 7:42 UTCresolved

   As of 19:40 UTC, we believe we've identified content to be the root cause of this tiny number of sessions not passing through scoring and triggering a Firehose event. We've reached out to customer proactively and will help address this issue directly. We are now calling this issue resolved and apologize for any distractions. Please reach out if you have any questions or concerns.