Pricing
Status historyRSS updatesStatus API

Powered by Uptimus

    Viedoc driftsforstyrrelser

    50 Viedoc incidents · marts 2024 — official updates, affected components, duration and resolution details.

    Tilbage til aktuel status

    Oplysninger om PHP-sårbarhed (php2shell)

    Startede 21. oktober 2026 kl. 16.25 UTC · I gang

    Pending
    1. 21. oktober 2026 kl. 16.25 UTCmonitoring

      Status: Informativ, ingen indvirkning Vi er klar over den nyligt afslørede PHP sårbarhed kendt som "php2shell" (CVE- 2026- 109618 og CVE- 2026- 109619). Vi har vurderet sårbarheden og kan bekræfte, at den ikke påvirker Viedoc Suite. Der kræves ingen handling fra vores kunder, og der er ingen indvirkning på tilgængeligheden, sikkerheden eller integriteten af dine data. Vi fortsætter med at overvåge situationen som en del af vores standard sikkerhedsprocesser. Hvis noget ændres, vil vi opdatere denne meddelelse.

    Automatisk oversat fra den officielle hændelsesopdatering.

    Sporadisk e-mail fejl på EU 's produktionssted

    Startede 8. september 2026 kl. 14.07 UTC · 2d 18h

    IssuesMindre hændelse
    Berørte komponenter
    Background processes (import/export/revisions/alerts/archive/disposal)
    1. 8. september 2026 kl. 14.07 UTCinvestigating

      Email-advarsler udløst af EDC-handlinger i Viedoc har været svigtende på EU-produktionsinstansen i følgende tidsvinduer: Mandag den 31. august 11: 02-17: 25 Torsdag den 3. september, 13: 31 EST - fredag den 4. september, 22: 07 EST De fleste e-mails sendes som forventet, men en delmængde er forsinket eller i nogle tilfælde slet ikke leveret. Ingen andre tilfælde er berørt. Vi er aktivt at undersøge og vil sende opdateringer som mere information bliver tilgængelig.

    2. 10. september 2026 kl. 13.51 UTCmonitoring

      Vi har ikke observeret nye e-mail fejl siden vores tidligere rapport. Vi overvåger nu situationen og vil sende en ny opdatering, når vi kan bekræfte, at spørgsmålet er helt løst.

    3. 11. september 2026 kl. 08.25 UTCresolved

      Denne hændelse er blevet løst.

    4. 17. september 2026 kl. 08.12 UTCpostmortem

      * * Sporadisk e-mail fejl på EU 's produktionssted * * Name * * Beskrivelse * * Under de to vinduer nedenfor, email advarsler udløst af handlinger i Viedoc blev sendt periodisk. De fleste blev leveret som forventet. En delmængde blev forsinket, og nogle blev slet ikke leveret. * Mandag den 31. august 11: 02-17: 25 EST * Torsdag den 3. september 13: 31 EST - fredag den 4. september 22: 07 EST E-mail-advarsler i Viedoc ikke sendes af programmet i det øjeblik en bruger gemmer data. De er fremstillet af baggrundsbehandling, der kører umiddelbart bagefter og beslutter, baseret på din undersøgelse konfiguration, hvilke advarsler der skal sendes. Under disse to vinduer, at baggrundsbehandling ikke altid afsluttet, så de indberetninger, det ville have produceret var forsinket eller aldrig genereret. Name Data indtastet og gemt af brugerne i disse perioder blev gemt som forventet. Afbrydelsen påvirkede arbejdet, der blev håndteret af bestemte servere i stedet for platformen som helhed, hvilket er grunden til, at indberetninger blev intermitterende i stedet for at stoppe helt. Name * * Årsag * * Viedocs baggrundsbehandling kører på tværs af flere servere, der arbejder parallelt. Hver server har brug for en arbejdsforbindelse til en fælles intern tjeneste, som Viedoc bruger til at koordinere dette arbejde sikkert - for eksempel for at sikre, at to servere aldrig behandler det samme emne i samme øjeblik. Under disse vinduer mistede de enkelte servere forbindelsen og genetablerede den ikke. En berørt server fortsatte med at acceptere arbejde, men kunne ikke fuldføre det. Andre servere, der kører på samme tid var upåvirket, hvilket er grunden til virkningen var intermitterende, og hvorfor platformen fortsatte med at virke sund generelt. Name * * Korrigerende aktion * * * De berørte servere blev identificeret og erstattet. Normal behandling, herunder alarmgenerering, genoptages umiddelbart efter. * Vi bekræftede, at efterfølgende behandling afsluttet korrekt, og at ingen yderligere e-mail fejl opstod. Name * * Forebyggende handling * * Øget overvågning Vi har øget overvågningen og tilføjet advarsel, der registrerer denne specifikke fejl på en individuel server meget hurtigere, hvilket forhindrer dette i at mislykkes lydløst. Name Baggrund - igangværende undersøgelse Undersøgelsen af, hvorfor disse forbindelser mislykkedes forbliver åben, og vi behandler det som adskilt fra de lempelser ovenfor.

    Automatisk oversat fra den officielle hændelsesopdatering.

    Midlertidig forstyrrelse i Kina

    Startede 4. september 2026 kl. 15.00 UTC · 0m

    IssuesMindre hændelse
    1. 4. september 2026 kl. 16.15 UTCresolved

      Vi havde en midlertidig forstyrrelse, som førte til kunder ikke være i stand til at nå ud .cn portaler mellem omkring 23: 00 og 00: 00. Problemet var i vores infrastruktur, og vi undskylder for uforsonligheden. Systemerne er normale igen, og der er ingen problemer med fortrolighed eller integritet. Dette problem påvirkede kun adgangen til vores systemer.

    Automatisk oversat fra den officielle hændelsesopdatering.

    Central udbrud (Viedoc Europe)

    Startede 17. august 2026 kl. 09.54 UTC · 1h 27m

    OutageStørre hændelse
    Berørte komponenter
    Subject portal (ViedocMe)Document management (ViedocTMF)Application programming interface (API)Main portal (clinic/admin/designer)Reporting tool (ViedocReports)Background processes (import/export/revisions/alerts/archive/disposal)
    1. 17. august 2026 kl. 09.54 UTCmonitoring

      Vi oplever forespørgsler i en underliggende platform komponent fra Azure i Frankrig Central region, dette er at skabe nogle tab af trafik, som du kan opleve som tjenesten ikke er tilgængelig i korte perioder. Vi efterforsker og vil holde dig underrettet her.

    2. 17. august 2026 kl. 10.09 UTCmonitoring

      Vi fortsætter med at overvåge for yderligere spørgsmål.

    3. 17. august 2026 kl. 11.25 UTCresolved

      Incident er nu blevet løst af Microsoft, og vi kan bekræfte, at Viedoc suite er fuldt operationel igen. Vi undskylder ulejligheden.

    4. 17. august 2026 kl. 12.27 UTCpostmortem

      # # Summary Vi har oplevet problemer i en underliggende platform komponent fra Azure\ (Azure App services\) i Frankrig Central region, som skabt nogle tab af trafik, som kunderne oplevede som tjenesten ikke er tilgængelig. Vores system overvågning fangede problemerne med det samme og alarmerede vores platform team, som reagerede inden for 1 minut. Vi kunne inden for 15 minutter bekræfte, at problemet var med platformen udbyder. # Tilfælde ramt Alle produkter i Europa # # Varighed 2026- 08- 17 09: 32 UTC til 2026- 08- 17 11: 20 UTC. # Root Cause En infrastrukturkomponent i Microsoft Azure Cloud med ansvar for behandling af indgående anmodninger inden for Azure App Service-platformen i Frankrigs centrale region oversteg operationelle tærskler, hvilket reducerer tilgængeligheden af serviceressourcer, der understøtter kundearbejde. Som et resultat, Viedoc oplevet intermitterende tilgængelighed problemer og mislykkede anmodninger. # Impact Dette påvirkede Viedoc tilgængelighed uden at påvirke datafortrolighed, integritet eller privatlivets fred. Handlinger Follow up med platform udbyder om kapacitet.

    Automatisk oversat fra den officielle hændelsesopdatering.

    Spørgsmål i Kina-regionen

    Startede 29. juli 2026 kl. 12.07 UTC · 4h 59m

    OutageStørre hændelse
    Berørte komponenter
    Document management (ViedocTMF)Application programming interface (API)Background processes (import/export/revisions/alerts/archive/disposal)Subject portal (ViedocMe)Main portal (clinic/admin/designer)Reporting tool (ViedocReports)
    1. 29. juli 2026 kl. 13.35 UTCinvestigating

      Vi er i øjeblikket ved at undersøge dette problem, opdatering kommer snart.

    2. 29. juli 2026 kl. 14.11 UTCinvestigating

      Spørgsmålet hænger sammen med en nedlukning af et datacenter i Kina, og vi undersøger fortsat.

    3. 29. juli 2026 kl. 17.06 UTCresolved

      Hændelsen blev identificeret som et problem i Azure Cloud Kina, og vi har nu fået bekræftet, at problemet er blevet mildnet. Når vi har en root- årsag analyse vil vi opdatere med en hændelse rapport. Vi undskylder for ulejligheden.

    4. 31. juli 2026 kl. 13.23 UTCpostmortem

      # # Summary Vores Kina region lidt et udfald på grund af en konfiguration fejl i vores platform udbyder relateret til intern messaging mellem vores systemkomponenter, effektivt gør vores produkt suite ubrugelig i en periode på 4 timer og 40 minutter. Kundens indvirkning forventes at være lav på grund af dette er nat i Kina med normalt meget lav trafik volumen. Vores system overvågning fangede problemerne straks og alarmerede vores platform team, som reagerede inden for 1 minut, og vi kunne bekræfte inden for 1 time, at problemet var med platformen udbyder. En type A support billet blev logget med vores platform udbyder, der kunne bekræfte, at de faktisk havde et problem 2 timer og 32 minutter senere, og det blev løst 38 minutter efter. # Tilfælde ramt Alle produkter i Kina # # Varighed 2026- 07- 29 12: 07 UTC til 2026- 07- 29 16: 47 UTC. # Root Cause En konfigurationsfejl ved vores platform udbyder i Kina, 21ViaNet / Microsoft Azure Cloud Kina. # Impact Kun til rådighed. Dataintegritet og fortrolighed er upåvirket. Handlinger Opfølgning med platform udbyder om at forbedre deres operationelle procedurer, overvågning og hændelsesreaktion.

    Automatisk oversat fra den officielle hændelsesopdatering.

    Viedoc Mig Indlæg optaget i UTC i stedet for lokal tidszone

    Startede 9. juli 2026 kl. 21.40 UTC · 6d 12h

    Pending
    Berørte komponenter
    Subject portal (ViedocMe)Subject portal (ViedocMe)Subject portal (ViedocMe)Subject portal (ViedocMe)
    1. 9. juli 2026 kl. 21.40 UTCidentified

      Siden udgivelsen 4,95, Viedoc Mine indlæg er registreret i databasen ved hjælp af UTC tid snarere end stedet lokale tidszone. Hvis UTC-datoen og lokalitetens lokale dato afviger på indsendelsestidspunktet, vil hændelsesdatoen i Viedoc Clinic fejlagtigt afspejle UTC-datoen i stedet for den lokale dato. Afhængigt af de dataindgangsvinduer, der er defineret i undersøgelsens design, kan dette også resultere i forespørgsler uden for vinduet. Revisionssporet i Viedoc Clinic UI, der er tilgået via "Vis historie", viser den korrekte dato og tidsstempel i henhold til den lokale tidszone. Dette spørgsmål påvirker både de nuværende og ældre versioner af Viedoc Me ansøgning. Spørgsmålet er blevet identificeret, og et fix er ved at blive udarbejdet. Fixet vil sikre nye indsendelser er gemt ved hjælp af stedets lokale tidszone, men det vil ikke rette tidsstempler eller out- of- vindue forespørgsler, der allerede er forekommet. Disse vil blive behandlet i en fremtidig opdatering.

    2. 14. juli 2026 kl. 14.57 UTCidentified

      Vi arbejder aktivt på at løse dette problem så hurtigt som muligt. Et hotfix er i øjeblikket ved at blive færdiggjort og testet, og vi vurderer, at det vil blive frigivet til produktion i første halvdel af næste uge.

    3. 16. juli 2026 kl. 06.53 UTCidentified

      Vi vil begynde at frigive hotfix til produktionsmiljøer i løbet af dagen i dag. En opdatering vil blive bogført, når udgivelsen er færdig på alle tilfælde.

    4. 16. juli 2026 kl. 09.42 UTCresolved

      Hændelsen er blevet løst med frigivelse af hotfix 4.95.2 til alle uddannelse og produktion tilfælde. Denne udgivelse forhindrer problemet i at gentage fremad. Begivenhedsdatoer, der allerede er registreret i UTC, vil dog forblive uændrede for nu, og eventuelle out- of- window forespørgsler, der allerede er udløst af dette nummer, vil også forblive indtil rettet. Indlæg, der i øjeblikket opbevares i UTC, vil blive rettet på et senere tidspunkt. Tidspunktet for denne korrektion er endnu ikke blevet bekræftet.

    Automatisk oversat fra den officielle hændelsesopdatering.

    Batch upgrade after form revision clears data in repeating item groups on EU and US instances

    Startede 10. juni 2026 kl. 18.32 UTC · 6d 16h

    Pending
    Berørte komponenter
    Main portal (clinic/admin/designer)Main portal (clinic/admin/designer)
    1. 10. juni 2026 kl. 18.32 UTCinvestigating

      After the Viedoc 4.95 release, when a revision is applied to a form containing a repeating item group, and a user upgrades the form from the landing page, data previously entered in the repeating item group is cleared. The issue is occurring on the EU and US training and production instances. Investigation is ongoing. Until further information is known, we advise all customers to refrain from applying revisions requiring form upgrades on forms containing repeating item groups.

    2. 10. juni 2026 kl. 23.31 UTCidentified

      The issue has been identified and a fix is being implemented.

    3. 16. juni 2026 kl. 14.45 UTCidentified

      Hotfix 4.95.1 has been released on the US training and production instances. The incident is resolved on those instances. The hotfix will be deployed to EU training and production instances tomorrow.

    4. 17. juni 2026 kl. 10.40 UTCresolved

      Hotfix 4.95.1 has been released on the EU training and production instances. The incident is now resolved on all instances.

    Temporary operational disturbance observed on the Chinese instance

    Startede 3. juni 2026 kl. 12.12 UTC · 1h 11m

    IssuesMindre hændelse
    Berørte komponenter
    Document management (ViedocTMF)Application programming interface (API)Background processes (import/export/revisions/alerts/archive/disposal)Subject portal (ViedocMe)Main portal (clinic/admin/designer)Reporting tool (ViedocReports)
    1. 3. juni 2026 kl. 12.12 UTCmonitoring

      Temporary operational disturbance observed in the Viedoc suite on the Chinese instance (clinic.viedoc.cn).

    2. 3. juni 2026 kl. 13.23 UTCresolved

      This incident has been resolved.

    Operational disturbance observed on the Chinese instance

    Startede 28. maj 2026 kl. 05.30 UTC · 1h 10m

    OutageStørre hændelse
    Berørte komponenter
    Document management (ViedocTMF)Application programming interface (API)Background processes (import/export/revisions/alerts/archive/disposal)Subject portal (ViedocMe)Main portal (clinic/admin/designer)Reporting tool (ViedocReports)
    1. 28. maj 2026 kl. 07.35 UTCinvestigating

      Operational disturbance observed in Viedoc suite on the Chinese instance (clinic.viedoc.cn). Investigations are ongoing since 05:20 CMT

    2. 28. maj 2026 kl. 07.38 UTCresolved

      The incident has been resolved since 06:40 CMT

    Operational disturbance observed on the Chinese instance

    Startede 18. maj 2026 kl. 09.55 UTC · 37m

    Pending
    Berørte komponenter
    Document management (ViedocTMF)Application programming interface (API)Background processes (import/export/revisions/alerts/archive/disposal)Subject portal (ViedocMe)Main portal (clinic/admin/designer)Reporting tool (ViedocReports)
    1. 18. maj 2026 kl. 10.24 UTCinvestigating

      Operational disturbance observed in Training on the Chinese instance (clinic.viedoc.cn). Investigations are ongoing.

    2. 18. maj 2026 kl. 10.26 UTCinvestigating

      We are continuing to investigate this issue.

    3. 18. maj 2026 kl. 10.32 UTCresolved

      This incident has been resolved.

    Issues with randomization configuration approval and legacy medical coding tool on EU and US instances

    Startede 6. maj 2026 kl. 09.48 UTC · 5d 0h

    IssuesMindre hændelse
    Berørte komponenter
    Main portal (clinic/admin/designer)Main portal (clinic/admin/designer)
    1. 6. maj 2026 kl. 09.48 UTCidentified

      After the Viedoc 4.94 release, the following issues affect the EU and US training and production instances: - Users cannot approve randomization configurations in Viedoc Admin when kit allocation is not used. Randomization and kit allocation in Viedoc Clinic are not affected. - WHO Drug dictionary terms are not loading as expected in the legacy medical coding tool. The new medical coding tool is not affected. We are working on a fix for these issues.

    2. 7. maj 2026 kl. 06.34 UTCidentified

      After the Viedoc 4.94 release, the following issues affect the EU and US training and production instances: - Users cannot approve randomization configurations in Viedoc Admin when kit allocation is not used. Randomization and kit allocation in Viedoc Clinic are not affected. - WHO Drug dictionary terms are not loading as expected in the legacy medical coding tool. The new medical coding tool is not affected. - Intermittent failures in Designer, where users are unexpectedly logged out, are affecting certain users. We are working on a fix for these issues, and the Designer issue is currently under investigation.

    3. 8. maj 2026 kl. 08.22 UTCidentified

      A hotfix is currently being prepared and will be deployed to both our EU and US instances. Based on preliminary timelines, we expect this release to go out on Monday, May 11.

    4. 11. maj 2026 kl. 09.54 UTCresolved

      This incident was resolved this morning with the release of hotfix version 4.94.1 on both the EU and US instances.

    Operational disturbance observed on the European instance

    Startede 27. april 2026 kl. 11.00 UTC · 8h 0m

    Pending
    Berørte komponenter
    Subject portal (ViedocMe)Document management (ViedocTMF)Application programming interface (API)Main portal (clinic/admin/designer)Reporting tool (ViedocReports)Background processes (import/export/revisions/alerts/archive/disposal)
    1. 27. april 2026 kl. 13.04 UTCinvestigating

      Operational disturbance observed in Viedoc 4 on the European instance (v4.viedoc.net). Investigations are ongoing.

    2. 27. april 2026 kl. 17.40 UTCinvestigating

      Microsoft Azure experienced a power event at their France Central datacenter, caused by a loss of utility power from an external provider. Power has since been restored via backup generators and recovery is underway. As of now, there are no active disruptions to Viedoc services, and we continue to monitor the situation closely.

    3. 27. april 2026 kl. 17.42 UTCmonitoring

      Microsoft Azure experienced a power event at their France Central datacenter, caused by a loss of utility power from an external provider. Power has since been restored via backup generators and recovery is underway. As of now, there are no active disruptions to Viedoc services, and we continue to monitor the situation closely.

    4. 27. april 2026 kl. 19.00 UTCresolved

      This incident has been resolved.

    Sporadic failures with text message delivery in some European countries

    Startede 9. december 2025 kl. 09.37 UTC · 2h 43m

    Pending
    Berørte komponenter
    Subject portal (ViedocMe)Main portal (clinic/admin/designer)
    1. 9. december 2025 kl. 09.37 UTCinvestigating

      We have received reports of intermittent delivery failures affecting text messages sent to numbers with country codes +31 (Netherlands), +32 (Belgium), +33 (France), and +44 (United Kingdom). An investigation is currently underway together with the subproccessor. The default SMS sender is changed from a phone number to "Viedoc" to reduce the risk of texts being treated as spam.

    2. 9. december 2025 kl. 12.20 UTCresolved

      Since 10.30 CET today text messages are delivered as expected to numbers with country codes previously listed.

    Issues Generating the User Administration Log

    Startede 5. december 2025 kl. 12.25 UTC · 11d 2h

    Pending
    Berørte komponenter
    Main portal (clinic/admin/designer)Main portal (clinic/admin/designer)Main portal (clinic/admin/designer)Main portal (clinic/admin/designer)
    1. 5. december 2025 kl. 12.25 UTCidentified

      The User Administration Log (Excel) and Log of Users and Roles (PDF) cannot currently be generated for studies containing more than 100 users. This issue was introduced in the 4.90 release and affects all instances where version 4.90 has been deployed. The root cause has been identified, and a corrective fix is currently in preparation.

    2. 9. december 2025 kl. 08.30 UTCidentified

      A fix has been developed and is currently undergoing testing. The fix is planned to be deployed as a hotfix.

    3. 16. december 2025 kl. 15.18 UTCresolved

      This issue has been resolved with the release of Viedoc 4.90.1, which was deployed to all production instances on 2025-12-16. The User Administration Log (Excel) and Log of Users and Roles (PDF) can now be generated as expected, including for studies containing more than 100 users.

    Delay in email delivery on US and EU instances

    Startede 1. december 2025 kl. 13.10 UTC · 19h 11m

    IssuesMindre hændelse
    Berørte komponenter
    Background processes (import/export/revisions/alerts/archive/disposal)Background processes (import/export/revisions/alerts/archive/disposal)
    1. 1. december 2025 kl. 13.10 UTCidentified

      Since around 12.30 CET, 1st of December, Viedoc is not sending any emails from the EU and US training and production instances due to a service disruption. The issue has been identified and we are working with our subprocessor to resolve the issue. All emails triggered during the affected period have been queued and will be delivered. The incident is caused by a malfunction at one of our subprocessors.

    2. 1. december 2025 kl. 13.46 UTCmonitoring

      The email queue is currently being processed, and messages are being delivered, although with a temporary delay.

    3. 2. december 2025 kl. 08.21 UTCresolved

      The queue had been fully processed at around 9.20 PM CET on 2025-12-01. Since that time emails have been delivered without delay.

    Delay in email delivery on EU instances

    Startede 25. november 2025 kl. 13.31 UTC · 19h 36m

    IssuesMindre hændelse
    Berørte komponenter
    Background processes (import/export/revisions/alerts/archive/disposal)
    1. 25. november 2025 kl. 13.31 UTCmonitoring

      Between 13:22 CET and 14:12 CET on 25 November 2025, Viedoc did not send any emails from the EU training and production instance due to a service disruption. The issue was identified and fully resolved at 14:12 CET, after which email processing resumed as normal. All emails triggered during the affected period have been queued and will be delivered. The incident was caused by a malfunction at one of our subprocessors.

    2. 25. november 2025 kl. 14.16 UTCmonitoring

      The email queue is currently being processed, and messages are being delivered, although with a temporary delay.

    3. 26. november 2025 kl. 09.08 UTCresolved

      The queue had been fully processed at around 5.30 PM CET on 2025-11-25.

    Operational disturbance observed on the Chinese instance

    Startede 25. november 2025 kl. 01.00 UTC · 0m

    OutageStørre hændelse
    1. 25. november 2025 kl. 03.19 UTCresolved

      Operational disturbance observed on the Chinese instance (clinic.viedoc.cn). Resolved

    Cancellation of the 4.89 Release for CN Instances

    Startede 11. november 2025 kl. 11.55 UTC · 0m

    Pending
    Berørte komponenter
    Document management (ViedocTMF)Application programming interface (API)Background processes (import/export/revisions/alerts/archive/disposal)Subject portal (ViedocMe)Main portal (clinic/admin/designer)Reporting tool (ViedocReports)
    1. 11. november 2025 kl. 11.55 UTCresolved

      The planned release of version 4.89 to the CN training and production instances has been cancelled due to unforeseen technical issues specific to the Chinese server environment. As the 4.90 release is already scheduled for December, we will proceed directly from version 4.88 to 4.90 for the CN training and production instances. The impact to end users is minimal, as version 4.89 contains only non-critical bug fixes. No other instances or regions are affected by this change. We appreciate your understanding and apologize for any inconvenience this may cause.

    Login issues on EU production instance

    Startede 2. november 2025 kl. 22.30 UTC · 0m

    OutageStørre hændelse
    1. 3. november 2025 kl. 13.27 UTCresolved

      From approximately 23.44 UTC to 23.54 UTC, users faced issues logging in to the European production instance. Investigations to find the root cause are ongoing. No other instances were affected.

    2. 10. november 2025 kl. 10.45 UTCpostmortem

      Description From approximately 23.44 UTC to 23.54 UTC on 2025-11-02, users were unable to log in to the European production instance. It is possible that users faced sporadic issues logging in also between approximately 23.00 UTC and 23.44 UTC. No other instances were affected. ‌ Cause The cause of the issue was an unexpected error in the identity provider \(idp\), leading to an accumulating memory consumption during the evening \(UTC\) of 2025-11-02. From approximately 23.00 UTC users would face sporadic failures when trying to log in. From approximately 23.44 UTC users could not log in. At 23.54 UTC on 2025-11-02 the idp was restarted, and users could log in as expected again. ‌ Corrective Action Alerts were triggered when the idp did not respond and the idp was restarted again at 2354 UTC on 2025-11-02. After the restart users could log in as expected again. ‌ Preventive Action Preliminary investigation identified a few improvements that could be made to make the idp more resilient. Implementation of these improvements have been initiated while the investigation of the issue continues.

    Sporadic email failures on EU production server

    Startede 2. september 2025 kl. 13.56 UTC · 1d 0h

    IssuesMindre hændelse
    Berørte komponenter
    Main portal (clinic/admin/designer)
    1. 2. september 2025 kl. 13.56 UTCinvestigating

      Emails triggered from EDC actions in Viedoc has been failing sporadically on the EU production instance since approximately 9AM CEST 2025-09-02. No other instances are affected. Investigations are ongoing.

    2. 3. september 2025 kl. 08.49 UTCidentified

      The issue has been identified and a fix is being implemented.

    3. 3. september 2025 kl. 11.25 UTCmonitoring

      A fix has been implemented and emails are now being sent as expected. We will continue to monitor the situation.

    4. 3. september 2025 kl. 14.27 UTCresolved

      Since approximately noon CEST, emails triggered from the EDC has been delivered as expected.

    5. 9. september 2025 kl. 20.22 UTCpostmortem

      **Description** Between 9AM CEST on 2025-09-02 and noon CEST 2025-09-03 there were sporadic failures in email alerts expected to be triggered from Viedoc on the EU production instance. No other instances were affected. Customers were initially informed through the status page at 15.56 CEST on 2025-09-03. ‌ **Cause** On Tuesday at 9AM CEST on 2025-09-02, there was a very high load on the system. The high load on the system caused worker function apps, used to handle various processes  in the EDC, to scale out to the maximum use of 20 instances in parallel. This caused failures, and in combination with in-built retry policy of failing actions, there was an overload on the system. While form save and form view continued to work as expected throughout the duration of the issue, there were sporadic errors in specific system actions in the post-processing step. * EDC email alerts, configured in the CRF design, were sporadically not sent during this time window. It did not affect all emails. * Terms defined to be coded in the medical coding tool sporadically failed to sync to Viedoc Coder. It did not affect all terms. ‌ **Corrective action** Once the root cause of the exceptions was identified, the worker function app was gradually scaled down to run on 4 instances to avoid having too many actions ongoing at the same time. This work begun around 11AM CET on 2025-09-03 and an immediate improvement could be seen. Around noon 2025-09-03 the situation was completely resolved, and all emails could be sent as expected again. ‌ Email alerts not sent during this issue could not be re-sent and will also not show up in the communication log. ‌ All terms that had not synced properly with Viedoc Coder during the issue, were synced with Viedoc Coder on the afternoon of 2025-09-03 and 2025-09-04. ‌ **Preventive action** Scaling down the worker function app to run on 4 instances is not considered to bring any noticeable impact on performance, but will ensure the same overload issue will not occur again. An evaluation will be performed if it is necessary to scale up the worker function app again and if so, how this could be done without causing any similar issues.