Pricing
Status historyRSS updatesStatus API

Powered by Uptimus

    Störungsverlauf von Learnosity

    56 Learnosity incidents · August 2021 — official updates, affected components, duration and resolution details.

    Zurück zum aktuellen Status

    Problem, das Artikelpools in allen Regionen betrifft.

    Beginn 26. Oktober 2026 um 18:05 UTC · 2h 5m

    IssuesGeringfügiger Vorfall
    Betroffene Komponenten
    Use of Item PoolsUse of Item PoolsCreation and editing of Local Device Assessment packagesCreation and editing of Local Device Assessment packagesCreation and editing of Local Device Assessment packagesUse of Item Pools
    1. 26. Oktober 2026 um 18:05 UTCinvestigating

      Ab 17:50 UTC haben wir Probleme mit Artikelpools in allen Regionen. Learnosity Support- und Systems Engineering-Teams untersuchen das Problem aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    2. 26. Oktober 2026 um 18:34 UTCinvestigating

      Ab 18:30 UTC untersuchen wir die verminderte Leistung beim Erstellen und Bearbeiten von Item Pools. Wir untersuchen auch ein bisher unbestätigtes tangentiales Problem, das möglicherweise die Erstellung und Bearbeitung von Paketen zur Bewertung lokaler Geräte beeinflusst. Alle anderen APIs in den Autoren-, Bewertungs- und Analyse-Stacks funktionieren normal. Die Teams von Learnosity Support und Systems Engineering untersuchen das Problem weiterhin aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    3. 26. Oktober 2026 um 19:01 UTCinvestigating

      Ab 19:00 UTC untersuchen wir weiterhin die beeinträchtigte Leistung beim Erstellen oder Bearbeiten von Paketen zur lokalen Gerätebewertung von Item Pools. Alle anderen APIs in den Autoren-, Bewertungs- und Analyse-Stacks funktionieren normal. Die Teams von Learnosity Support und Systems Engineering untersuchen das Problem weiterhin aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    4. 26. Oktober 2026 um 19:31 UTCmonitoring

      Ab 19:30 UTC hat die Erstellung von Item Pool und Local Device Assessment einen normalen Durchsatz für etwa 30 Minuten gesehen. Wir werden nun in eine zusätzliche Kontrollperiode eintreten, um Stabilität zu gewährleisten. Die Teams von Learnosity Support und Systems Engineering untersuchen das Problem weiterhin aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    5. 26. Oktober 2026 um 20:10 UTCresolved

      Ab 20:10 UTC haben wir keine weiteren Probleme mit der Erstellung oder Aktualisierung von Paketen für den Item Pool oder die lokale Gerätebewertung gesehen. Wir lösen diesen Vorfall und werden weiterhin die Ursache der verschlechterten Leistung untersuchen. Learnosity Support- und Systems Engineering-Teams werden mit einem Post-Mortem fortfahren, sobald wir die Ursachenanalyse abgeschlossen und die nächsten Schritte oder erforderlichen vorbeugenden Maßnahmen abgeschlossen haben. Bitte kontaktieren Sie uns, wenn Sie Fragen oder Bedenken haben.

    Automatisch aus der offiziellen Störungsmeldung übersetzt.

    Verbessertes Mathe-Scoring mit Verzögerungen in der US-East-1-Region

    Beginn 20. Oktober 2026 um 14:16 UTC · 4h 1m

    IssuesGeringfügiger Vorfall
    Betroffene Komponenten
    Scoring endpointScoring of Enhanced Math question types
    1. 20. Oktober 2026 um 14:16 UTCinvestigating

      Ab 13:20 UTC begannen wir, Punkteverzögerungen mit unserem Enhanced Math Scoring-Service in der US-East-1-Region zu sehen. Es wurde kein Datenverlust beobachtet und alle Sitzungen wurden letztendlich bewertet. Als Folgeeffekt tritt beim Data API Scoring-Endpunkt intermittierende Verfügbarkeitsprobleme auf. Keine Autorisierung oder Bewertung Lieferung wurde betroffen. Learnosity Support- und Systems Engineering-Teams untersuchen das Problem aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    2. 20. Oktober 2026 um 15:04 UTCinvestigating

      Ab 15:00 Uhr UTC untersuchen wir immer noch die kurzlebige verschlechterte Leistung unseres Enhanced Math Scoring-Service, haben aber keine signifikanten Fehler oder Verzögerungen für mehr als eine Stunde gesehen. Der rückstau der sitzungen, die auf das scoring warten, wurde schnell gelöscht, und einsendungen werden weiterhin normal bewertet. Alle anderen APIs bleiben unberührt. Die Teams von Learnosity Support und Systems Engineering untersuchen das Problem weiterhin aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    3. 20. Oktober 2026 um 15:51 UTCmonitoring

      Ab 15:45 UTC haben Minderungsmaßnahmen die Einreichungswarteschlange erfolgreich freigehalten, und das Scoring funktionierte weiterhin normal. Beiträge, die während des Zeitraums der beeinträchtigten Leistung eingegangen sind, wurden erfasst, und es gingen keine Daten verloren. Alle anderen APIs bleiben funktionsfähig. Wir untersuchen weiterhin die Ursache und werden einen Vorfall postmortem teilen, sobald unsere Untersuchung abgeschlossen ist. In der Zwischenzeit treten wir in einen zusätzlichen Überwachungszeitraum ein, um die anhaltende Stabilität zu bestätigen, bevor der Vorfall abgeschlossen wird.

    4. 20. Oktober 2026 um 18:17 UTCresolved

      Ab 18:00 UTC, nach einer langen Zeit der zusätzlichen Überwachung ohne weitere Probleme, sind wir diesen Vorfall mit Auswirkungen auf Enhanced Math Scoring in US-East-1 gelöst. Learnosity Support- und Systems Engineering-Teams werden mit einem Post-Mortem fortfahren, sobald wir die Ursachenanalyse abgeschlossen und die nächsten Schritte oder erforderlichen vorbeugenden Maßnahmen abgeschlossen haben. Bitte kontaktieren Sie uns, wenn Sie Fragen oder Bedenken haben.

    5. 27. Oktober 2026 um 13:27 UTCpostmortem

      ## Betroffene Systeme und Regionen Am 20. Oktober 2026 erlebte Learnosity Scoring-Verzögerungen und intermittierende Anforderungsfehler, die das Enhanced Math-Scoring in US-East-1 beeinflussten. Der Kundeneinfluss begann um ca. 13:06 UTC und endete um 13:47 UTC und dauerte ca. 41 Minuten. Betroffene Scoring-Anfragen wurden länger als normal verarbeitet, und einige direkte Anfragen an den Data API-Scoring-Endpunkt gaben Fehler zurück. Bei Sitzungen, die über die Items-API gespeichert wurden, wurden betroffene Scoring-Anfragen in die Warteschlange gestellt und erneut getestet, was zu einer verzögerten Score-Verfügbarkeit und nicht zu Scoring-Ausfällen führte. Bewertungen ohne mathematische Fragen und Dienstleistungen in anderen Regionen blieben unberührt. ## Untersuchung Unsere Untersuchung identifizierte ein spezifisches Muster von Scoring-Anfragen, das zu unerwartet hohen Bearbeitungszeiten führte. Dies reduzierte die verfügbare Kapazität zur Verarbeitung anderer Scoring-Anfragen, was zu Verzögerungen und zeitweiligen Timeouts führte. Das Problem war damit verbunden, wie bestimmte Scoring-Eingaben verarbeitet wurden, und nicht mit einem unerwarteten Anstieg des Gesamtanforderungsvolumens. ## Resolution Die Scoring-Leistung wurde um 13:47 UTC wieder normal. Um 15:12 UTC implementierten die Learnosity-Ingenieure eine zusätzliche Verkehrsisolation, um zu verhindern, dass sich die betroffene Scoring-Arbeitslast auf die Scoring-Operationen anderer Kunden auswirkt. ## Prävention Learnosity hat Änderungen implementiert, um unnötigen Ressourcenverbrauch zu reduzieren und die Wiederherstellung betroffener Scoring-Anfragen zu verbessern. Weitere Verbesserungen sind im Gange, um die Validierung von Scoring-Anfragen zu stärken, die Verarbeitungssicherheit zu verbessern und zu verhindern, dass einzelne Scoring-Workloads den breiteren Dienst beeinträchtigen. Zusätzliche Isolationsmaßnahmen bleiben bestehen, solange diese Verbesserungen abgeschlossen sind und ihre Wirksamkeit überprüft wird.

    Automatisch aus der offiziellen Störungsmeldung übersetzt.

    Problem mit LTS Remapping von Sunset Version v2024.1.LTS

    Beginn 8. Oktober 2026 um 13:40 UTC · 1h 35m

    OutageSchwerwiegender Vorfall
    Betroffene Komponenten
    LTS Remapping of Sunset Versions
    1. 8. Oktober 2026 um 13:40 UTCinvestigating

      Ab 13:00 Uhr UTC haben wir Probleme mit der Neuzuordnung der Sunset-API-Versionen v2024.1.LTS zu den neuesten LTS-Versionen in allen Regionen. Dies betrifft nur eine sehr kleine Anzahl von Kunden, die immer noch an v2024.1.LTS gebunden sind, das Ende des Lebens im März 2026 erreicht hat. Andere API-Versionen sind nicht betroffen. Learnosity Support- und Systems Engineering-Teams untersuchen das Problem aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    2. 8. Oktober 2026 um 13:47 UTCinvestigating

      Ab 13:40> UTC leiten wir v2024.1.LTS-Versionen um und APIs werden als Caches-Update zugänglich. Die Teams von Learnosity Support und Systems Engineering untersuchen das Problem weiterhin aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    3. 8. Oktober 2026 um 14:20 UTCinvestigating

      Ab 14:00 Uhr UTC wird v2024.1.LTS erfolgreich auf die neueste LTS-Version in AMER-Regionen umgestellt, einschließlich US-East-1 (VA) und US-West-2 (OR). Alle APIs sollten in diesen Regionen zugänglich sein. Wir remapping derzeit in EMEA und APAC Regionen. Direkter Zugriff auf aktive API-Versionen bleibt funktionsfähig. Dieses Problem betrifft nun nur noch die Neuzuordnung von Versionen für einen Mikrosatz von Kunden in den Regionen EMEA und APAC, die an die verfallene Version der v2024.1.LTS-API angeheftet wurden, die im März 2026 EOL erreichte. Die Teams von Learnosity Support und Systems Engineering untersuchen das Problem weiterhin aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    4. 8. Oktober 2026 um 15:15 UTCresolved

      Ab 15:00 Uhr UTC ist v2024.1.LTS in allen Regionen erfolgreich remapping. Mit diesem EOL / Sunset-Versionsstring sollte auf alle APIs zugegriffen werden. Learnosity Support- und Systems Engineering-Teams werden mit einem Post-Mortem fortfahren, sobald wir die Ursachenanalyse abgeschlossen und die nächsten Schritte oder erforderlichen vorbeugenden Maßnahmen abgeschlossen haben. Bitte kontaktieren Sie uns, wenn Sie Fragen oder Bedenken haben.

    Automatisch aus der offiziellen Störungsmeldung übersetzt.

    Problem mit dem Enhanced Math Scoring in der US-East-1-Region

    Beginn 2. Oktober 2026 um 18:08 UTC · 1h 44m

    IssuesGeringfügiger Vorfall
    Betroffene Komponenten
    Scoring of Enhanced Math question types
    1. 2. Oktober 2026 um 18:08 UTCinvestigating

      Ab 16:45 UTC begannen wir, mit unserem Enhanced Math Scoring-Service in der US-East-1-Region sehr geringe Punkteverzögerungen zu verzeichnen. Alle Sitzungen wurden bewertet und es kam zu keinem Datenverlust. Andere APIs sind nicht betroffen. Wir untersuchen laufende Verzögerungen, die auf diesen Dienst beschränkt sind. Learnosity Support- und Systems Engineering-Teams untersuchen das Problem aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    2. 2. Oktober 2026 um 18:30 UTCidentified

      Ab 18:30 UTC haben wir ein Skalierungsproblem für den Enhanced Math Scoring Service identifiziert und arbeiten daran, das Problem zu lösen. Alle Sitzungen werden weiterhin ohne Datenverlust erfasst und alle anderen APIs funktionieren normal. Die Teams von Learnosity Support und Systems Engineering untersuchen das Problem weiterhin aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    3. 2. Oktober 2026 um 19:10 UTCmonitoring

      Ab 19:00 UTC haben wir seit einiger Zeit keine Verzögerungen im Enhanced Math Scoring-Service mehr gesehen und treten nun in eine Überwachungsphase ein. Wir werden weiterhin 30 Minuten überwachen, bevor wir das Problem lösen. Die Teams von Learnosity Support und Systems Engineering untersuchen das Problem weiterhin aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    4. 2. Oktober 2026 um 19:52 UTCresolved

      Ab 19:45 UTC haben wir das Problem mit dem Enhanced Math Scoring in der US-East-1-Region behoben. Alle APIs und Dienste funktionieren normal. Learnosity Support- und Systems Engineering-Teams werden mit einem Post-Mortem fortfahren, sobald wir die Ursachenanalyse abgeschlossen und die nächsten Schritte oder erforderlichen vorbeugenden Maßnahmen abgeschlossen haben. Bitte kontaktieren Sie uns, wenn Sie Fragen oder Bedenken haben.

    5. 27. Oktober 2026 um 01:24 UTCpostmortem

      ## **Betroffene Systeme und Regionen** Am 2. Oktober 2026 erlebte Learnosity intermittierende Scoring-Fehler und Timeouts, die mathematische Fragetypen in US-East-1 beeinflussten. Der Kundeneinfluss begann um ca. 16:34 UTC und endete um 18:47 UTC und dauerte ca. 2 Stunden und 13 Minuten. Betroffene Anfragen können zu Verzögerungen von bis zu 40 Sekunden führen, bevor sie fehlschlagen. Andere Regionen waren nicht betroffen. ## **Untersuchung** Der Vorfall ereignete sich, als die Nachfrage nach dem Mathe-Scoring-Service die verfügbare Kapazität überstieg. Obwohl Kapazitäts- und Skalierungsverbesserungen nach einem früheren Vorfall eingeführt wurden, stieß der Dienst auf eine zusätzliche Grenze in seiner Netzwerkkonfiguration, die die Bereitstellung weiterer Instanzen verhinderte. Die Auswirkungen wurden durch langsame Berechnungen verschlimmert, die Instanzen länger als beabsichtigt belegten, da Berechnungszeitlimits bei starker Belastung nicht zuverlässig durchgesetzt wurden. Zusammengenommen verhinderten diese Bedingungen, dass der Dienst ausreichend skaliert wurde, um die Nachfrage zu bewältigen. ## **Resolution** Learnosity-Ingenieure stellten den Dienst wieder her, indem sie den Scoring-Dienst in eine Netzwerkkonfiguration mit zusätzlicher Skalierungskapazität verlagerten und die Berechnungsressourcen für jede Instanz erhöhten. Diese Änderungen ermöglichten es dem Dienst, sich zu erholen, wobei die Bewertungsfehler um 18:47 UTC gelöscht wurden. ## **Vorbeugung** Learnosity hat den Mechanismus zur Durchsetzung des Berechnungszeitraums aktualisiert, um langsame Berechnungen unter schwerer Last zuverlässig zu beenden. Gesundheitschecks wurden auch von der Berechnungsverarbeitung getrennt, um einen unnötigen Instanzersatz zu verhindern. Der Dienst arbeitet jetzt mit zusätzlicher Rechen- und Netzwerkkapazität. Weitere architektonische Verbesserungen sind im Gange, um zu verhindern, dass die Netzwerkkapazität die zukünftige Skalierung einschränkt, und wir evaluieren kürzere Berechnungszeitgrenzen, um die Auswirkungen ungewöhnlich langsamer Berechnungen zu reduzieren.

    Automatisch aus der offiziellen Störungsmeldung übersetzt.

    Untersuchung möglicher Probleme mit Scoring in US-East-1

    Beginn 11. September 2026 um 18:19 UTC · 3h 26m

    IssuesGeringfügiger Vorfall
    Betroffene Komponenten
    Scoring endpointScoring of Enhanced Math question types
    1. 11. September 2026 um 18:19 UTCinvestigating

      Ab 17:00 UTC untersuchen wir derzeit kleinere intermittierende Probleme mit dem Scoring in der US-East-1-Region. Learnosity Support- und Systems Engineering-Teams untersuchen das Problem aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    2. 11. September 2026 um 18:30 UTCinvestigating

      Ab 18:30 UTC bleiben alle APIs in Betrieb und wir überwachen die Effizienz der Bewertung. Die Teams von Learnosity Support und Systems Engineering untersuchen das Problem weiterhin aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    3. 11. September 2026 um 19:42 UTCidentified

      Ab 19:30 UTC haben wir bestätigt, dass dies ein sporadisches Problem mit der Bewertung von nur drei erweiterten mathematischen Fragetypen ist. Diese Scoring-Versuche sind bei Wiederholungen erfolgreich und keine anderen APIs sind betroffen. Die Teams von Learnosity Support und Systems Engineering untersuchen das Problem weiterhin aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    4. 11. September 2026 um 20:53 UTCmonitoring

      Ab 20:45 UTC haben wir die Handhabung von anfänglichen EMS-Scoring-Anfragen und Retries verbessert und treten nun in eine 60-minütige Überwachungsphase ein. Alle APIs funktionieren weiterhin normal. Die Teams von Learnosity Support und Systems Engineering untersuchen das Problem weiterhin aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    5. 11. September 2026 um 21:44 UTCresolved

      Ab 21:45 utc werden die antworten auf den fragetyp "enhanced math score" in der us-ost-1-region normal bewertet, alle apis funktionieren normal und wir schließen diesen vorfall ab. Learnosity Support- und Systems Engineering-Teams werden mit einem Post-Mortem fortfahren, sobald wir die Ursachenanalyse abgeschlossen und die nächsten Schritte oder erforderlichen vorbeugenden Maßnahmen abgeschlossen haben. Bitte kontaktieren Sie uns, wenn Sie Fragen oder Bedenken haben.

    6. 27. Oktober 2026 um 01:23 UTCpostmortem

      ## **Betroffene Systeme und Regionen** Am 11. September 2026 erlebte Learnosity intermittierende Scoring-Fehler und Timeouts, die mathematische Fragetypen in US-East-1 beeinflussten. Der Kundeneinfluss begann um ca. 17:00 UTC und endete um 21:45 UTC und dauerte ca. 4 Stunden und 45 Minuten. Andere Fragetypen punkten weiterhin normal und andere Regionen waren nicht betroffen. Die meisten fehlgeschlagenen Anfragen waren erfolgreich, wenn sie wiederholt wurden. ## **Untersuchung** Der Vorfall wurde durch unzureichende Kapazitäten im Mathe-Scoring-Service nach einem Anstieg der Nachfrage in den vorangegangenen Wochen verursacht. Der Dienst erreichte seine konfigurierte maximale Anzahl von Instanzen und konnte nicht weiter skalieren, um die Last aufzunehmen. Dies wurde durch begrenzte Rechenressourcen pro Instanz und langsame Berechnungen, die Instanzen länger als beabsichtigt belegten, noch verstärkt. Bei starker Last wurde das Berechnungslimit nicht zuverlässig durchgesetzt, und beschäftigte Instanzen konnten die Gesundheitskontrollen nicht bestehen und ersetzt werden, was die verfügbare Kapazität weiter reduziert. ## **Resolution** Learnosity-Ingenieure stellten den Dienst wieder her, indem sie die Anzahl der laufenden Instanzen erhöhten und das maximale Instanzlimit erhöhten. Normale Scoring-Operationen wurden von 21:45 UTC wieder aufgenommen, und alle verbleibenden in der Warteschlange stehenden Scoring-Anfragen wurden von 23:17 UTC verarbeitet. Am 14. September wurden zusätzliche Konfigurationsänderungen eingeführt, um die Rechenressourcen zu erhöhen, die automatische Skalierung zu verbessern und unnötigen Instanzersatz zu reduzieren. ## **Vorbeugung** Learnosity hat die Scoring-Kapazität erhöht, die Instanzlimits erhöht und die automatische Skalierung basierend auf der Prozessorauslastung verbessert. Gesundheitschecks wurden von der Berechnungsverarbeitung getrennt, um zu verhindern, dass geschäftige Instanzen falsch als ungesund identifiziert werden. Wir haben auch die Durchsetzung der Berechnungszeitbegrenzung verbessert und die Überwachung und Warnung auf Ressourcenauslastung, Reaktionszeiten und Bewertungsfehler erweitert. Weitere Arbeiten sind im Gange, um die Netzwerkskalierungskapazität zu stärken und kürzere Berechnungsfristen zu bewerten.

    Automatisch aus der offiziellen Störungsmeldung übersetzt.

    Mögliches Problem mit Auswirkungen auf AWS in US-West-2 (OR)

    Beginn 24. Juli 2026 um 12:16 UTC · 46m

    Pending
    Betroffene Komponenten
    Logging in (author site)Logging in (author site)Logging in (author site)
    1. 24. Juli 2026 um 12:16 UTCinvestigating

      Uns ist ein laufendes AWS-Problem bekannt, das sich auf die Region US-West-2 (OR) auswirkt und beim Zugriff auf die Learnosity Authoring-Umgebung zu intermittierenden Verbindungsproblemen führen kann. Wir beobachten die Situation aktiv und werden Updates bereitstellen, sobald sie verfügbar sind. Zu diesem Zeitpunkt sind die Auswirkungen auf Learnosity-Kunden kurz und intermittierend und auf das Authoring beschränkt. Assessments und andere APIs verhalten sich normal. Learnosity Support- und Systems Engineering-Teams untersuchen das Problem aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    2. 24. Juli 2026 um 12:28 UTCmonitoring

      Ab 11:02 UTC begann Learnosity, Monitorwarnungen von kurzlebigen, intermittierenden Timeouts beim Zugriff auf die Learnosity-Autorenseite zu sehen. Keine andere Funktionalität war betroffen, und Learnosity-Benachrichtigungen wurden um 11:11 UTC eingestellt. Um 11:40 UTC meldete AWS einen degradierten Service in US-West-2 (OR). Um 11:56 UTC meldete AWS, dass sich ihre Systeme erholten. Die Teams von Learnosity Support und Systems Engineering überwachen das Problem weiterhin aktiv und werden so schnell wie möglich ein Update und eine Lösung durchführen.

    3. 24. Juli 2026 um 12:35 UTCmonitoring

      Um 12:30 UTC berichtete AWS, dass sie das Problem identifiziert und korrigiert haben und dass die Wiederherstellung fortgesetzt wird. Learnosity Support- und Systems Engineering-Teams werden das Problem für weitere 30 Minuten überwachen und so schnell wie möglich ein Update und eine Lösung durchführen.

    4. 24. Juli 2026 um 13:02 UTCresolved

      Ab 13:00 UTC nennen wir dieses Problem gelöst. AWS-Systeme erholen sich und Learnosity Authoring ist seit 50 Minuten nicht betroffen. Learnosity Support- und Systems Engineering-Teams werden mit einem Post-Mortem fortfahren, sobald wir die Ursachenanalyse abgeschlossen und die nächsten Schritte oder erforderlichen vorbeugenden Maßnahmen abgeschlossen haben. Bitte kontaktieren Sie uns, wenn Sie Fragen oder Bedenken haben.

    5. 24. Juli 2026 um 13:03 UTCpostmortem

      ### Betroffene Systeme und Regionen Am 2026-07-24 erlebte Learnosity kurze zeitweilige Timeouts, die den Zugang zur Learnosity-Autorenseite aufgrund eines Netzwerkvorfalls in der AWS-Region US-West-2 \(OR\) (https://health.aws.amazon.com/health/status?eventID=arn:aws:health:us-west-2::event/MULTIPLE SERVICES/AWS MULTIPLE SERVICES OPERATIONAL ISSUE/AWS MULTIPLE SERVICES OPERATIONAL ISSUE/AWS MULTIPLE SERVICES OPERATIONAL ISSUE 83A36 24C89B493A6) beeinträchtigten. Learnosity Customer Impact begann um ca. 11:02 UTC und endete um 11:11 UTC. AWS schloss daraufhin die Behebung ab und schloss den regionalen Vorfall um 12:12 UTC offiziell ab. Die Gesamtdauer der möglichen Auswirkungen auf Learnosity-Kunden betrug etwa 9 Minuten. ### Untersuchung Learnosity erhielt eine interne Warnung um 11:02 UTC, dass der Dienst in der Region OR betroffen war, und begann sofort eine Untersuchung. Erste Untersuchungen fanden keine Hinweise auf eine Learnosity Service Degradation. Weitere Untersuchungen identifizierten intermittierende Timeouts, die den Zugriff auf die Learnosity-Autorenseite beeinflussten. Learnosity bestätigte mit AWS, dass ein Problem identifiziert wurde, das sich auf die Netzwerkinfrastruktur in der Region auswirkt, und die Behebung begonnen hatte. ### Resolution Intermittierende Timeouts, die sich auf die Learnosity-Autorenseite auswirkten, wurden um 11:11 UTC eingestellt, und AWS löste das Problem effektiv um 11:15 UTC. AWS meldete ein kurzes Rekonvergenz-Ereignis, das bis 11:59 UTC gelöst wurde, und schloss den Vorfall offiziell um 12:12 UTC. ### Prävention Keine Learnosity-Systeme trugen zu diesem Vorfall bei. Die interne Überwachung und Alarmierung verhielt sich wie erwartet.

    Automatisch aus der offiziellen Störungsmeldung übersetzt.

    Degraded performance affecting availability of newly submitted session results in US-East-1

    Beginn 17. April 2026 um 16:06 UTC · 48m

    IssuesGeringfügiger Vorfall
    Betroffene Komponenten
    Availability of session informationLoading and rendering of reports
    1. 17. April 2026 um 16:06 UTCinvestigating

      As of 16:00 UTC, we are currently experiencing a slowdown in availability of newly submitted submitted session results in the us-east-1 region. There is no data loss, all saves and submissions are being queued as designed, and the assessments stack remains operational. Delays may currently be experienced using the Data API to fetch session results, and the Reports API to display results. Other workflows that rely on the synchronous use of Data API results may also be affected. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

    2. 17. April 2026 um 16:21 UTCidentified

      As of 16:20 UTC, Learnosity has increased database performance and we are processing queued sessions rapidly. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    3. 17. April 2026 um 16:28 UTCmonitoring

      As of 16:25 UTC, we have resolved the slowdown in processing session results and all sessions are being scored normally. We will continue to monitor for a further 30 minutes before closing this incident report. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    4. 17. April 2026 um 17:58 UTCresolved

      As of 16:55 UTC, we are have resolved the degraded performance issue affecting availability of session data in us-east-1. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required. Please reach out if you have any questions or concerns.

    5. 4. Mai 2026 um 17:23 UTCpostmortem

      ### Affected Systems and Regions On 2026-04-17, Learnosity experienced a service degradation impacting availability of newly submitted session results for a small subset of customers in US-East-1. The issue began at approximately 13:54 UTC and was resolved at 16:24 UTC. The total duration of customer impact was approximately 2.5 hours. ### Investigation Service degradation was detected following the accumulation of submitted sessions in the async scoring queue, This was caused by elevated CPU utilization when heavy load coincided with a background data migration task that had been running for several days on the affected EC2 instance. This reduced message processing throughput, leading to queueing of sessions.  ### Resolution Learnosity CloudOps engineers responded by scaling session processing node EC2 instances, and corresponding database proxies, to maximum allowable capacity. The migration task was also stopped, which further reduced CPU pressure and allowed message processing throughput to recover. The session scoring backlog was fully cleared by 16:24 UTC. ### Prevention Learnosity is implementing the following measures to mitigate: * Pre-flight health checks and safeguards are being introduced to prevent migration tasks from running during busy periods of production usage. * Enhanced monitoring is being applied to detect backlog growth and database contention, and apply remedial action more quickly

    Degraded performance affecting Live Progress Report in US-East-1

    Beginn 16. April 2026 um 14:08 UTC · 1h 16m

    IssuesGeringfügiger Vorfall
    Betroffene Komponenten
    Live Progress (Live Activity by User) report
    1. 16. April 2026 um 14:08 UTCinvestigating

      As of 13:30 UTC, we are currently experiencing slow downs affecting the Live Progress report in the us-east-1 region. Atypical use of eventbus may also be affected, such as custom reports/implementations. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

    2. 16. April 2026 um 15:23 UTCresolved

      As of 14:30 UTC, the Live Progress report event degradation issue in the us-east-1 region has been resolved. Additional capacity processed the event load and these systems are operating normally. Brief additional latency during this scaling may have been introduced for users of Learnosity's premium Firehose feature, but we've seen no evidence that this affected customers. This, too, was resolved once scaling was complete. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required. Please reach out if you have any questions or concerns.

    3. 4. Mai 2026 um 14:39 UTCpostmortem

      ### Affected Systems and Regions On 2026-04-16, Learnosity experienced a service degradation impacting the Live Progress report for a small subset of customers in the us-east-1 region. The issue began at approximately 13:12 UTC and was resolved at 14:40 UTC. The total duration of customer impact was approximately 88 minutes. ### Investigation The issue was detected following elevated error rates on the load balancer serving the eventbus service. Investigation determined that an unhealthy condition within the EC2 instances led to elevated CPU utilization and memory pressure within the Auto Scaling Group, resulting in application instability and repeated service restarts. Inconsistent recovery caused traffic to concentrate on a subset of hosts, further elevating error rates. A secondary effect of the instability was increased request pressure on downstream dependencies. ### Resolution Service was restored by stabilizing the affected EC2 instance and restoring consistent application availability across the Auto Scaling Group. Load distribution normalized once all instances returned to a healthy state. ### Prevention Learnosity is implementing the following measures to mitigate: * Improve service startup and dependency handling to ensure consistent recovery behavior * Review resource thresholds to reduce the likelihood of similar instability * Enhance monitoring to detect and respond more quickly to similar conditions

    Issue affecting assessment delivery in VA (us-east-1)

    Beginn 4. Februar 2026 um 18:34 UTC · 40m

    OutageSchwerwiegender Vorfall
    Betroffene Komponenten
    Loading and rendering of Item/Activity Edit viewAMER Assessment (Summary)AMER Analytics (Summary)Loading and rendering of Question Edit viewAMER Authoring (Summary)Saving of student responsesLoading and rendering of assessment UILoading and rendering of Items/Questions/FeaturesLoading and rendering of reports
    1. 4. Februar 2026 um 18:34 UTCidentified

      As of 18:30 UTC, we are currently experiencing an issue affecting assessment delivery in VA (us-east-1). Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

    2. 4. Februar 2026 um 18:55 UTCmonitoring

      As of 18:52 UTC, the Learnosity System Engineering team has put a fix in place. We will continue to monitor for some time to ensure a stable recovery, but all services are operational again. Learnosity Support and Systems Engineering teams will continue to actively monitor the issue, and will follow on with an update and resolution as soon as possible.

    3. 4. Februar 2026 um 19:15 UTCresolved

      As of 19:12 UTC, traffic has returned to normal and everything remains stable. We have resolved the incident affecting assessments in VA (us-east-1). Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required. Please reach out if you have any questions or concerns.

    4. 10. Februar 2026 um 17:08 UTCpostmortem

      ### **Affected Systems and Regions** On 2026-02-04, Learnosity experienced an availability incident impacting Questions API access for customers in the AMER region. The issue began at approximately 18:02 UTC and was formally declared resolved at 18:51 UTC. The total duration of customer impact was approximately 49 minutes. ### **Investigation** The issue was first detected when the Items API attempted to initialize the dependent Questions API. The incident was traced to a misconfiguration within the content delivery network servicing the AMER region, causing a SSL certificate error.. During a certificate management task, the distribution responsible for serving [_questions.learnosity.com_](http://questions.learnosity.com) was configured with an SSL certificate that did not match the _\*.learnosity.com_ domain nomenclature. This caused browsers and client systems to reject connections to the affected subdomain. ### **Resolution** Once the issue was identified, the response team updated the affected distribution to use the correct SSL certificate. Service was fully restored after the change was completed. ### **Prevention** Learnosity is strengthening certificate deployment and validation processes to ensure that domain mappings are verified prior to rollout. Additional safeguards have been added to prevent cross-domain certificate assignments and to improve monitoring for sudden traffic drops that may indicate certificate-related failures.

    Issue affecting availability of recent session data for a minority of customers in the US-East-1 region.

    Beginn 12. Januar 2026 um 17:06 UTC · 2h 4m

    Pending
    Betroffene Komponenten
    Availability of session informationUpdating session response scoresFirehoseLoading and rendering of reports
    1. 12. Januar 2026 um 17:06 UTCinvestigating

      As of 4:30 UTC, we are currently investigating an issue that may result in temporary session unavailability for a small number of US-only customers. We can confirm that no session data has been lost, and we are working to restore the affected sessions as quickly as possible. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

    2. 12. Januar 2026 um 17:52 UTCidentified

      As of 5:30 UTC, we've identified and corrected a load balancing configuration affecting database reads and writes for a very small number of US-only customers with consumers in the US-East-1 region. - All new sessions will behave as expected. - Read requests seeking data on sessions created prior to 23:00 UTC yesterday will also now behave correctly. - We are working on restoring sessions recently affected While all APIs continue to function normally, this update identifies service areas that may temporarily have limited access to recently saved or submitted sessions. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    3. 12. Januar 2026 um 19:10 UTCresolved

      As of 7:00 UTC, we've concluded our monitoring and confirmed that all new requests are behaving as expected. We are closing this record and will continue to work with affected customers directly to confirm that all recent sessions are accessible. Please reach out if you have any questions or concerns.

    AWS outage continuing to have some impact on Learnosity services in VA (us-east-1)

    Beginn 20. Oktober 2025 um 14:53 UTC · 6h 42m

    OutageSchwerwiegender Vorfall
    Betroffene Komponenten
    Branching/Adaptive servicesAMER Assessment (Summary)Creation of report data setsAMER Analytics (Summary)Availability of session informationProcessing and availability of rich student responses (audio/images/files)Scoring endpointSelf-Hosted Adaptive servicesUpdating session response scoresCreation of sessionsAMER Data Centric (Summary)Creation of Item PoolsFirehose
    1. 20. Oktober 2025 um 14:53 UTCmonitoring

      Learnosity services in VA are continuing to be partially impacted by the major AWS outage. AWS first reported the incident at 7:11 UTC As of 14:50 UTC they are continuing to work on its full resolution, but have again set their impact level to "Degraded". The AWS Health Status dashboard where updates are being posted by the AWS team can be found here: https://health.aws.amazon.com/health/status Overall, the error rate has fallen and has stayed below what we saw earlier in the day and we have been successful in slowly scaling up the infrastructure, but we're still seeing some issues stemming from the the ongoing AWS outage as we attempt to bring online some new instances, and we have begun to see an uptick in errors. These have included errors associated with Data API, Feedback Aide, and the delivery of adaptive assessments, delays in scoring and video encoding, as well as other smaller spikes across different systems. While the overall error rate remains low, Learnosity Support and Systems Engineering teams are actively monitoring the outage and manually intervening where possible to mitigate the issues we're facing. In spite of our best efforts it is likely that the APIs may not be fully operational in VA until AWS's service is fully restored to normal operation, but we are taking every step within our power to try to minimize the disruption to our services.

    2. 20. Oktober 2025 um 15:30 UTCinvestigating

      As of 15:25 UTC, we are seeing increasing error rates. Data API is the most affected service at the moment and we are seeing an increase in scoring delays, but we are continuing to score incoming messages to the best rate our current capacity allows. AWS is still reporting a severity of "Degraded", but now with an increased count of 81 affected services. Learnosity Support and Systems Engineering teams are continuing to monitor the issue, and will continue our mitigation efforts wherever possible.

    3. 20. Oktober 2025 um 18:13 UTCmonitoring

      As of 19:10 UTC, we are beginning to see errors rates diminish and some Learnosity services are beginning to scale again. AWS is still reporting a severity of "Degraded" with problems to 104 of their services in us-east-1. Learnosity Support and Systems Engineering teams are continuing to monitor and manually intervene when possible to mitigate the effects of the outage.

    4. 20. Oktober 2025 um 19:02 UTCmonitoring

      As of 20:00 UTC, the error rate has continued to drop, systems remain scaled and the backlogs of scoring messages continues to drop. AWS is still reporting 109 impacted services and a severity of "Degraded" Learnosity Support and Systems Engineering teams are continuing to monitor the issue and are manually adjusting the infrastructure where needed to ensure stability.

    5. 20. Oktober 2025 um 19:32 UTCmonitoring

      As of 19:30 UTC, the error rate has fallen to 0.1% and continues to fall, we continue to steadily process the backlog of scoring messages. As of their latest update AWS has downgraded the severity of outage to "Impacted" but still list 109 services as affected. Learnosity Support and Systems Engineering teams are continuing to actively monitor the issue, and will follow on with an update and resolution as soon as possible.

    6. 20. Oktober 2025 um 20:44 UTCmonitoring

      As of 20:40 UTC, the error rate remains low, the infrastructure is scaled to meet demand, and the scoring queues are empty and new submissions are being processed as they come in. All services are operational and we are processing the remaining backlog of messages from the incident. AWS still lists 93 affected services and a severity of "Impacted". Learnosity Support and Systems Engineering teams are continuing to monitor the issue, but at the moment all systems are fully operational.

    7. 20. Oktober 2025 um 21:35 UTCresolved

      As of 21:30 UTC, all systems keep operating normally, so we are resolving the status page about the AWS outage affecting Learnosity Services in the VA (us-east-1) region. Learnosity Support and Systems Engineering teams will continue to monitor the systems closely while AWS continues to work on the issues on their side. Please reach out if you have any questions or concerns.

    AWS Outage Impacted some Learnosity Services

    Beginn 20. Oktober 2025 um 10:11 UTC · 1h 12m

    Pending
    Betroffene Komponenten
    Support ticketing systemProcessing and availability of rich student responses (audio/images/files)
    1. 20. Oktober 2025 um 10:11 UTCmonitoring

      AWS is going through late-stage recovery of a major outage in US-East-1 bringing down most of their services. Learnosity APIs were stable and unaffected in isolation, but some impact cascaded to Learnosity (such as availability of assets from CloudFront). The full extent of the incident is unknown at this time. NOTE: Up until the time of this writing (10:05 UTC), this included impact to sign-in services that prevented us from accessing our StatusPage. We immediately posted banner notifications in our Support Portal, and those banners will remain visible until we've been able to verify that service has been restored for sufficient time. Incident History: As of 7:00 UTC, AWS experienced a major outage. While Learnosity APIs were fully operational, some Learnosity services use AWS and were affected. These included access to our Support ticketing system (the Learnosity-hosted Help guides were unaffected), and availability of CloudFront-hosted assets, but other systems may have been impacted as well. As of 10:00 UTC, AWS has restored services and access to Learnosity-adjacent features is becoming available relatively quickly. Learnosity Support will monitor AWS for a further 30 minutes before calling this issue resolved. We will follow on with an update and resolution at that time, or sooner.

    2. 20. Oktober 2025 um 11:23 UTCresolved

      As of 11:20 UTC, AWS services appear to have been stable for approximately an hour. Learnosity APIs--including services affected by the AWS outage (such as CloudFront)--appear to be fully operational. At this time we are labeling this issue resolved. Learnosity Support and Systems Engineering teams will continue to investigate impact from this incident and follow up with a post mortem once we hear from AWS. Please reach out if you have any questions or concerns.

    Partner Service Outage May Disrupt Learnosity API Initialization in All Regions

    Beginn 2. Oktober 2025 um 21:34 UTC · 1h 9m

    Pending
    Betroffene Komponenten
    Loading and rendering of Items/Questions/FeaturesLoading and rendering of Question Edit viewLoading and rendering of Question Edit viewLoading and rendering of Items/Questions/FeaturesLoading and rendering of Items/Questions/FeaturesLoading and rendering of reportsLoading and rendering of Question Edit viewLoading and rendering of reportsLoading and rendering of reports
    1. 2. Oktober 2025 um 21:34 UTCinvestigating

      As of 21:15 UTC, Learnosity APIs are being impacted by a temporary outage from our Math partner, Desmos. Authoring or Editing content with Desmos questions or features, or initializing assessments that contain Desmos questions or features, can prevent the Author API or Items API from initializing. Specific reports that display question content, such as the Session Detail by Item report, can also fail to initialize Learnosity APIs are not otherwise impacted, so content that does not include Desmos features are not affected. Until we learn more, we will list APIs that may experience knock-on effects from this temporary Desmos outage. However, as most customers will not experience any issues, we will not list these APIs as down. Desmos and Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

    2. 2. Oktober 2025 um 22:14 UTCmonitoring

      As of 22:12 UTC, Desmos services have been restored and Learnosity APIs are initializing without issue. Desmos and Learnosity Support and Systems Engineering teams will continue to monitor for a further 30 minutes before declaring this issue resolved. We will follow on with an update and resolution as soon as possible.

    3. 2. Oktober 2025 um 22:43 UTCresolved

      As of 22:42 UTC, all Learnosity APIs are continuing to function as expected in all regions, and we are declaring this issue resolved. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required. Please reach out if you have any questions or concerns.

    Issue affecting availability of recently submitted session analytics in US-East-1

    Beginn 19. August 2025 um 16:11 UTC · 5h 21m

    IssuesGeringfügiger Vorfall
    Betroffene Komponenten
    Availability of session information
    1. 19. August 2025 um 16:11 UTCinvestigating

      As of 3:40 UTC, we are experiencing degraded performance in our analytics stack affecting the availability of session data in US-East-1 for a subset of customers. This is affecting the Reports API and Data API. Neither authoring nor assessment stacks are affected, and there is no data loss. Submitted sessions are queueing for processing. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

    2. 19. August 2025 um 16:35 UTCinvestigating

      As of 4:30 UTC, we are continuing to investigate degraded performance in our Data and Reports APIs. Only availability of recently submitted session data is affected. Historic session data, as well as all other API stacks, remain unaffected. New submissions are persisting correctly with no data loss and these submission are being queued for scoring. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    3. 19. August 2025 um 17:34 UTCidentified

      (Note: We're correcting cited UTC times to the 24 hr format and will include both forms in this update only.) As of 5:30pm/17:30 UTC, we've identified a possible contributing cause for the degraded performance in our analytics stack. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    4. 19. August 2025 um 18:42 UTCidentified

      As of 18:30 UTC, initial remediation efforts have tripled the rate of queued session processing and we are continuing to work toward a full resolution. Access to recently submitted session results via the Data API and Reports API remains the only affected part of the Learnosity ecosystem. New submissions continue to be safely queued for scoring while the degraded performance remains. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    5. 19. August 2025 um 19:56 UTCidentified

      As of 19:30 UTC, we are now processing queued sessions rapidly and more than half of the backlog has already cleared.. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    6. 19. August 2025 um 20:35 UTCidentified

      As of 20:30 UTC, the scoring queue backlog is almost empty and new sessions will soon be processed without delay. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    7. 19. August 2025 um 21:05 UTCmonitoring

      As of 21:00 UTC, all sessions have been cleared from the scoring backlog queue and all submissions are being processed normally. Learnosity Support and Systems Engineering teams will monitor this situation for a further 30 minutes before calling it resolved.

    8. 19. August 2025 um 21:31 UTCresolved

      As of 21:30 UTC, we are have resolved the issue affecting availability of session data in US-East-1. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required. Please reach out if you have any questions or concerns.

    9. 19. September 2025 um 19:14 UTCpostmortem

      ### Affected Systems and Regions On 19 August 2025, Learnosity experienced degraded performance in our analytics stack, affecting session data availability in US-East-1 for a subset of customers. This affected the Reports API and Data API, with no other stacks affected, and no data loss.  ### Investigation Monitoring detected a rapid increase in unprocessed and retried messages, along with elevated lock contention in the sessions database. The root cause was traced to a customer implementation issue generating an extraordinarily high number of submissions. This drove excessive retries, magnifying actual traffic volume.  The use of time-ordered v7 UUIDs for session IDs, normally handled without issue, became problematic under this contention. Uniqueness checks on each session ID required more resources and triggered a succession of temporary deadlocks. These deadlocks would usually self-resolve, but the amplified traffic prevented recovery, turning a minor issue into a sustained queue blockage. ### Resolution Once the issue was identified, Learnosity moved the customer to a dedicated, isolated sync queue, preventing cross‑tenant impact while we investigated. We applied targeted rate limits for the isolated service to protect the database, and drained the backlog. Where safe, long‑running queries were terminated to free locks and allow forward progress.  To support faster diagnosis, Learnosity enabled detailed deadlock logging and expanded metrics around message retries, abandonment, and per‑session activity. Learnosity also worked with the customer to adjust implementation settings, reducing combined saves and submits by two orders of magnitude. Session IDs were also switched to v4 UUIDs which simplified uniqueness checks further preventing deadlocks. Immediately after these changes were put into use, queues began to rapidly recover, and normal processing resumed. Most sessions for the subset of affected customers saw short delays, while the most significantly delayed session took ~6 hours before final persistence. Throughout, we identified no data loss. ### Prevention To prevent recurrence, we are: * Implementing targeted load tests and contention simulations to replicate high-parallelism patterns. * Reviewing customer identifier schemes for session IDs and auditing usage across all customers \(initial checks confirm none of our Top 50 customers currently use v7 UUIDs\). * Analyzing adoption of per-tenant fair-use queues \(or equivalent fair-share policies\) to cap burst throughput from a single tenant and protect shared infrastructure.

    Possible Issue affecting manual updates to session data in US-East-1 (VA) region

    Beginn 8. August 2025 um 13:55 UTC · 24m

    Pending
    Betroffene Komponenten
    Updating session response scores
    1. 8. August 2025 um 13:55 UTCinvestigating

      As of 13:40 UTC, we began investigating a possible issue affecting manual update jobs to session response scores, sessions statuses, and session metadata in the US-East-1 (VA) region. We saw a brief buildup in the queue for these jobs, the queue processed as expected, and is now empty. However, we're investigating further, out of an abundance of caution, to see if there are any unexpected causes for the short queue buildup. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

    2. 8. August 2025 um 13:57 UTCinvestigating

      As of 13:55 UTC, all manual update jobs have been executed and the jobs queue is empty. Requests to manually update session response scores, session statuses, and session metadata should execute as expected. We are still investigating possible reasons for the blip. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    3. 8. August 2025 um 14:19 UTCresolved

      As of 14:10 UTC, we've found no negative behavior in the manual update processes and attribute the short burst of jobs to uncharacteristic testing of the new manual grading workflow. The jobs queue functioned as expected, ensuring no loss of requests, and the jobs were processed in a timely fashion. Learnosity will continue to monitor this activity for a further 60 minutes, but we are calling this resolved at this time. We apologize for the unwarranted alert. Please reach out if you have any questions or concerns.

    4. 8. August 2025 um 14:19 UTCpostmortem

      At 13:40 UTC on 8 August 2025, we noticed a brief backup of asynchronous jobs affecting manual session updates in the US-East-1 \(VA\) region. Possible jobs including manual updates to response scores, session statuses, and session metadata. The queue mechanism worked as expected, and all requests were processed in short order. Out of an abundance of caution, we raised an alert while systems remained operational, but found nothing of concern during our investigation. We hope the unwarranted alert was not a significant distraction.

    Issue affecting availability of session information in US-East-1

    Beginn 3. März 2025 um 13:48 UTC · 2h 6m

    OutageSchwerwiegender Vorfall
    Betroffene Komponenten
    Availability of session informationLoading and rendering of reports
    1. 3. März 2025 um 13:48 UTCinvestigating

      As of 13:45 UTC, we are experiencing delays in availability of session information in US-East-1 Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

    2. 3. März 2025 um 14:00 UTCinvestigating

      As of 14:00 UTC, we are still investigating an issue affecting the availability of session information in US-East-1. Both Data API and Reports API are experiencing extended delays and intermittent failures in returning session results. Authoring and Assessment APIs appear to be unaffected at this time and all session submissions are being successfully queued. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    3. 3. März 2025 um 15:06 UTCinvestigating

      As of 15:00 UTC, we are continuing to investigate the issue. Data and Reports API are still impacted. Assessment and Authoring APIs remain unaffected. Assessment submissions are being queued and will be processed as the system works through any backlog queue. Note that if the Data API or Reports API is used in a preliminary/synchronous assessment delivery workflow, this will likely have a knock-on affect when initializing assessments. Initializing any assessment API directly is unaffected at present. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    4. 3. März 2025 um 15:20 UTCmonitoring

      As of 15:15 UTC, all services have been restored. All queued messages have been processed and session information should now be available. Users of the Learnosity Firehose service will still see slight delays as messages are sent out for recently processed events. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    5. 3. März 2025 um 15:54 UTCresolved

      As of 15:45 UTC, after a further 30 minutes of monitoring, we are resolving this issue. All services remain fully operational. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalised any next steps or preventative measures required. Please reach out if you have any questions or concerns.

    6. 14. März 2025 um 16:31 UTCpostmortem

      ### Affected Systems and Regions On March 3, 2025, we experienced a temporary service slowdown impacting the availability of session information in the US-East-1 region. The issue began at 13:19 UTC and was resolved by 15:15 UTC. The Data API and Reports API experienced delays and intermittent failures when returning session results. Authoring and Assessment APIs were unaffected with no data loss. ### Investigation We identified a recently introduced inefficient database query was causing unnecessary strain when handling extremely large data sets. This led to delayed responses to the Data API sessions endpoints. This further led to delays or unavailability in select reports via the Reports API. Additionally, some timeouts and safeguards that should have triggered earlier did not activate as expected. ### Resolution To quickly restore performance: * We optimized the affected queries, improving efficiency and reducing database load. * We scaled up the impacted systems to immediately process the backlog. * We fine-tuned system timeouts and connection limits to prevent similar issues in the future. Following these fixes, all services resumed normal operations. ### Prevention To prevent future occurrences: * We have permanently updated our database handling methods to ensure more efficient query execution. * We are enhancing automated monitoring to detect and respond to database slowdowns before they impact customers. We appreciate your patience and remain committed to delivering a seamless experience.

    Analytics APIs Experiencing Degraded Performance in US-East-1

    Beginn 9. Januar 2025 um 13:33 UTC · 1h 17m

    IssuesGeringfügiger Vorfall
    Betroffene Komponenten
    Availability of session informationLoading and rendering of reports
    1. 9. Januar 2025 um 13:33 UTCinvestigating

      As of 13:30 UTC, We are currently investigating a possible performance degradation affecting Reports API and Data API results in US-East-1. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

    2. 9. Januar 2025 um 14:02 UTCinvestigating

      As of 14:00 UTC, Learnosity is working on identifying the cause of degraded performance in our Analytics APIs, Reports and Data. Authoring and Assessment stacks remain unaffected Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    3. 9. Januar 2025 um 14:13 UTCmonitoring

      As of 14:10 UTC, Learnosity has restored full service for all new requests to all Analytics APIs. We are monitoring stability and analyzing all queued requests. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    4. 9. Januar 2025 um 14:50 UTCresolved

      As of 14:50 UTC, after a further 60 minutes of issue-free monitoring, we are resolving this issue affecting availability of session information via our Analytics stack. Reports API and Data API are fully operational. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalized any next steps or preventative measures required. Please reach out if you have any questions or concerns.

    5. 14. Januar 2025 um 17:25 UTCpostmortem

      **Affected Systems and Regions** On 2025-01-09, Learnosity experienced a brief partial outage affecting our analytics stacks, specifically the Reports API and Data API in the AMER region. The issue began at 13:30 UTC and was resolved at 14:06 UTC, lasting 36 minutes. All other API's were unaffected and there was no loss of data. **Investigation** We discovered that a large number of atypical, inefficient Data API queries requested by customers were taking too long to complete. This prevented other queries from running in a timely manner, creating a backlog. It was determined that an additional database index would significantly improve the response times of these types of queries. **Resolution** Immediately upon discovering this issue, the impacted database instances were successfully scaled up to ease the backlog of requests. The additional index was implemented and all remaining queries were processed quickly. Affected APIs returned to normal operations, and further monitoring ensured the issue was fully resolved. **Prevention** Following further testing, the new index is working well and has become a permanent part of the system. We are also adding new automated monitoring and regression testing to ensure similar requests perform as expected.

    Investigating issue affecting live progress report / proctoring in VA

    Beginn 1. Oktober 2024 um 15:09 UTC · 59m

    IssuesGeringfügiger Vorfall
    Betroffene Komponenten
    AMER Analytics (Summary)Live Progress (Live Activity by User) report
    1. 1. Oktober 2024 um 15:09 UTCinvestigating

      As of 15:00 UTC, we are currently investigating an issue affecting the event delivery that powers the live progress report (live-activitystatus-by-user) in VA (us-east-1). Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

    2. 1. Oktober 2024 um 15:25 UTCidentified

      As of 15:20 UTC, the Systems Engineering team has identified the cause for the slower than normal delivery of events and is proceeding with mitigation steps. Learnosity Support and Systems Engineering teams are continuing to actively investigate the issue, and will follow on with an update and resolution as soon as possible.

    3. 1. Oktober 2024 um 15:40 UTCmonitoring

      As of 15:35 UTC, the Systems Engineering team has deployed a fix, the errors have stopped, and currently all systems are functional. We will continue to monitor for a time before resolving the incident but we are currently seeing no errors. We will follow on with an update and resolution as soon as possible.

    4. 1. Oktober 2024 um 16:07 UTCresolved

      As of 16:05 UTC, We are have resolved the issue affecting live progress reports in VA (us-east-1). As the recovery remains stable, we are now resolving the incident that was affecting live progress reports. Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalised any next steps or preventative measures required. Please reach out if you have any questions or concerns.

    Investigating potential issue affecting live proctoring in us-east-1 (VA)

    Beginn 13. September 2024 um 14:35 UTC · 1h 28m

    IssuesGeringfügiger Vorfall
    Betroffene Komponenten
    AMER Assessment (Summary)Sending and receiving Live Activity Status by User report events
    1. 13. September 2024 um 14:35 UTCinvestigating

      As of 14:30 UTC, We are investigating a potential issue affecting the delivery of some Events API events (used in the live proctoring / live-activitystatus-by-user report) in us-east-1 (VA). Assessments are otherwise proceeding normally and can be saved and submitted without any issue. This issue if confirmed would only affect the delivery of Events API events. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

    2. 13. September 2024 um 15:09 UTCidentified

      As of 15:00 UTC, the Learnosity Systems Engineering team has identified the issue preventing the assessment events from being delivered and are working on the implementation of remediation steps. Learnosity Support and Systems Engineering teams are continuing to work on the issue, and will follow on with an update and resolution as soon as possible.

    3. 13. September 2024 um 15:34 UTCmonitoring

      As of 15:30 UTC, events are flowing normally once again and all systems are operational. We will continue to monitor for some time before we resolve the issue fully to ensure we have a stable recovery.

    4. 13. September 2024 um 16:03 UTCresolved

      As of 16:00 UTC, after monitoring and confirming the stable state of the fixes put into place, we have resolved the Events API events delivery issue affecting proctored assessments in us-east-1 (VA). Learnosity Support and Systems Engineering teams will follow up with a post mortem once we have completed root cause analysis and finalised any next steps or preventative measures required. Please reach out if you have any questions or concerns.

    Possible issue affecting small number of adaptive sessions not triggering Firehose updates in US-East-1

    Beginn 22. Mai 2024 um 18:27 UTC · 1h 15m

    Pending
    Betroffene Komponenten
    Adaptive session information
    1. 22. Mai 2024 um 18:27 UTCinvestigating

      As of 18:10 UTC, we are trying to determine if an issue exists causing a very small number of adaptive sessions to not trigger a Firehose event after scoring in US-East-1. This is just an internal investigation at present, without confirmation, and we have no evidence that any impact outside the described implementation is affected. Learnosity Support and Systems Engineering teams are actively investigating the issue, and will follow on with an update and resolution as soon as possible.

    2. 22. Mai 2024 um 18:33 UTCinvestigating

      As of 18:30 UTC, Learnosity Support and Systems Engineering teams are continuing to investigate this unconfirmed issue and will follow on with an update as soon as possible.

    3. 22. Mai 2024 um 18:47 UTCinvestigating

      As of 18:45 UTC, Learnosity Support and Systems Engineering teams are continuing to investigate an unconfirmed issue, examining less than 500 potentially affected sessions out of 4.93 million submitted over the same period. We will follow on with an update as soon as possible.

    4. 22. Mai 2024 um 19:01 UTCmonitoring

      As of 19:00 UTC, we have been unable to find any systemic problems, while the number of potentially impacted sessions remains at one hundredth of one percent ( 0.01%) of total submissions over the period investigated, likely due to a content issue. Learnosity Support and Systems Engineering teams will continue to monitor for 30 minutes and expect to close this investigation at that time.

    5. 22. Mai 2024 um 19:42 UTCresolved

      As of 19:40 UTC, we believe we've identified content to be the root cause of this tiny number of sessions not passing through scoring and triggering a Firehose event. We've reached out to customer proactively and will help address this issue directly. We are now calling this issue resolved and apologize for any distractions. Please reach out if you have any questions or concerns.