48 Viedoc incidents · marts 2024 — official updates, affected components, duration and resolution details.
Vi havde en midlertidig forstyrrelse, som førte til kunder ikke være i stand til at nå ud .cn portaler mellem omkring 23: 00 og 00: 00. Problemet var i vores infrastruktur, og vi undskylder for uforsonligheden. Systemerne er normale igen, og der er ingen problemer med fortrolighed eller integritet. Dette problem påvirkede kun adgangen til vores systemer.
Automatisk oversat fra den officielle hændelsesopdatering.
Vi oplever forespørgsler i en underliggende platform komponent fra Azure i Frankrig Central region, dette er at skabe nogle tab af trafik, som du kan opleve som tjenesten ikke er tilgængelig i korte perioder. Vi efterforsker og vil holde dig underrettet her.
Vi fortsætter med at overvåge for yderligere spørgsmål.
Incident er nu blevet løst af Microsoft, og vi kan bekræfte, at Viedoc suite er fuldt operationel igen. Vi undskylder ulejligheden.
# # Summary Vi har oplevet problemer i en underliggende platform komponent fra Azure\ (Azure App services\) i Frankrig Central region, som skabt nogle tab af trafik, som kunderne oplevede som tjenesten ikke er tilgængelig. Vores system overvågning fangede problemerne med det samme og alarmerede vores platform team, som reagerede inden for 1 minut. Vi kunne inden for 15 minutter bekræfte, at problemet var med platformen udbyder. # Tilfælde ramt Alle produkter i Europa # # Varighed 2026- 08- 17 09: 32 UTC til 2026- 08- 17 11: 20 UTC. # Root Cause En infrastrukturkomponent i Microsoft Azure Cloud med ansvar for behandling af indgående anmodninger inden for Azure App Service-platformen i Frankrigs centrale region oversteg operationelle tærskler, hvilket reducerer tilgængeligheden af serviceressourcer, der understøtter kundearbejde. Som et resultat, Viedoc oplevet intermitterende tilgængelighed problemer og mislykkede anmodninger. # Impact Dette påvirkede Viedoc tilgængelighed uden at påvirke datafortrolighed, integritet eller privatlivets fred. Handlinger Follow up med platform udbyder om kapacitet.
Automatisk oversat fra den officielle hændelsesopdatering.
Vi er i øjeblikket ved at undersøge dette problem, opdatering kommer snart.
Spørgsmålet hænger sammen med en nedlukning af et datacenter i Kina, og vi undersøger fortsat.
Hændelsen blev identificeret som et problem i Azure Cloud Kina, og vi har nu fået bekræftet, at problemet er blevet mildnet. Når vi har en root- årsag analyse vil vi opdatere med en hændelse rapport. Vi undskylder for ulejligheden.
# # Summary Vores Kina region lidt et udfald på grund af en konfiguration fejl i vores platform udbyder relateret til intern messaging mellem vores systemkomponenter, effektivt gør vores produkt suite ubrugelig i en periode på 4 timer og 40 minutter. Kundens indvirkning forventes at være lav på grund af dette er nat i Kina med normalt meget lav trafik volumen. Vores system overvågning fangede problemerne straks og alarmerede vores platform team, som reagerede inden for 1 minut, og vi kunne bekræfte inden for 1 time, at problemet var med platformen udbyder. En type A support billet blev logget med vores platform udbyder, der kunne bekræfte, at de faktisk havde et problem 2 timer og 32 minutter senere, og det blev løst 38 minutter efter. # Tilfælde ramt Alle produkter i Kina # # Varighed 2026- 07- 29 12: 07 UTC til 2026- 07- 29 16: 47 UTC. # Root Cause En konfigurationsfejl ved vores platform udbyder i Kina, 21ViaNet / Microsoft Azure Cloud Kina. # Impact Kun til rådighed. Dataintegritet og fortrolighed er upåvirket. Handlinger Opfølgning med platform udbyder om at forbedre deres operationelle procedurer, overvågning og hændelsesreaktion.
Automatisk oversat fra den officielle hændelsesopdatering.
Siden udgivelsen 4,95, Viedoc Mine indlæg er registreret i databasen ved hjælp af UTC tid snarere end stedet lokale tidszone. Hvis UTC-datoen og lokalitetens lokale dato afviger på indsendelsestidspunktet, vil hændelsesdatoen i Viedoc Clinic fejlagtigt afspejle UTC-datoen i stedet for den lokale dato. Afhængigt af de dataindgangsvinduer, der er defineret i undersøgelsens design, kan dette også resultere i forespørgsler uden for vinduet. Revisionssporet i Viedoc Clinic UI, der er tilgået via "Vis historie", viser den korrekte dato og tidsstempel i henhold til den lokale tidszone. Dette spørgsmål påvirker både de nuværende og ældre versioner af Viedoc Me ansøgning. Spørgsmålet er blevet identificeret, og et fix er ved at blive udarbejdet. Fixet vil sikre nye indsendelser er gemt ved hjælp af stedets lokale tidszone, men det vil ikke rette tidsstempler eller out- of- vindue forespørgsler, der allerede er forekommet. Disse vil blive behandlet i en fremtidig opdatering.
Vi arbejder aktivt på at løse dette problem så hurtigt som muligt. Et hotfix er i øjeblikket ved at blive færdiggjort og testet, og vi vurderer, at det vil blive frigivet til produktion i første halvdel af næste uge.
Vi vil begynde at frigive hotfix til produktionsmiljøer i løbet af dagen i dag. En opdatering vil blive bogført, når udgivelsen er færdig på alle tilfælde.
Hændelsen er blevet løst med frigivelse af hotfix 4.95.2 til alle uddannelse og produktion tilfælde. Denne udgivelse forhindrer problemet i at gentage fremad. Begivenhedsdatoer, der allerede er registreret i UTC, vil dog forblive uændrede for nu, og eventuelle out- of- window forespørgsler, der allerede er udløst af dette nummer, vil også forblive indtil rettet. Indlæg, der i øjeblikket opbevares i UTC, vil blive rettet på et senere tidspunkt. Tidspunktet for denne korrektion er endnu ikke blevet bekræftet.
Automatisk oversat fra den officielle hændelsesopdatering.
After the Viedoc 4.95 release, when a revision is applied to a form containing a repeating item group, and a user upgrades the form from the landing page, data previously entered in the repeating item group is cleared. The issue is occurring on the EU and US training and production instances. Investigation is ongoing. Until further information is known, we advise all customers to refrain from applying revisions requiring form upgrades on forms containing repeating item groups.
The issue has been identified and a fix is being implemented.
Hotfix 4.95.1 has been released on the US training and production instances. The incident is resolved on those instances. The hotfix will be deployed to EU training and production instances tomorrow.
Hotfix 4.95.1 has been released on the EU training and production instances. The incident is now resolved on all instances.
Temporary operational disturbance observed in the Viedoc suite on the Chinese instance (clinic.viedoc.cn).
This incident has been resolved.
Operational disturbance observed in Viedoc suite on the Chinese instance (clinic.viedoc.cn). Investigations are ongoing since 05:20 CMT
The incident has been resolved since 06:40 CMT
Operational disturbance observed in Training on the Chinese instance (clinic.viedoc.cn). Investigations are ongoing.
We are continuing to investigate this issue.
This incident has been resolved.
After the Viedoc 4.94 release, the following issues affect the EU and US training and production instances: - Users cannot approve randomization configurations in Viedoc Admin when kit allocation is not used. Randomization and kit allocation in Viedoc Clinic are not affected. - WHO Drug dictionary terms are not loading as expected in the legacy medical coding tool. The new medical coding tool is not affected. We are working on a fix for these issues.
After the Viedoc 4.94 release, the following issues affect the EU and US training and production instances: - Users cannot approve randomization configurations in Viedoc Admin when kit allocation is not used. Randomization and kit allocation in Viedoc Clinic are not affected. - WHO Drug dictionary terms are not loading as expected in the legacy medical coding tool. The new medical coding tool is not affected. - Intermittent failures in Designer, where users are unexpectedly logged out, are affecting certain users. We are working on a fix for these issues, and the Designer issue is currently under investigation.
A hotfix is currently being prepared and will be deployed to both our EU and US instances. Based on preliminary timelines, we expect this release to go out on Monday, May 11.
This incident was resolved this morning with the release of hotfix version 4.94.1 on both the EU and US instances.
Operational disturbance observed in Viedoc 4 on the European instance (v4.viedoc.net). Investigations are ongoing.
Microsoft Azure experienced a power event at their France Central datacenter, caused by a loss of utility power from an external provider. Power has since been restored via backup generators and recovery is underway. As of now, there are no active disruptions to Viedoc services, and we continue to monitor the situation closely.
Microsoft Azure experienced a power event at their France Central datacenter, caused by a loss of utility power from an external provider. Power has since been restored via backup generators and recovery is underway. As of now, there are no active disruptions to Viedoc services, and we continue to monitor the situation closely.
This incident has been resolved.
We have received reports of intermittent delivery failures affecting text messages sent to numbers with country codes +31 (Netherlands), +32 (Belgium), +33 (France), and +44 (United Kingdom). An investigation is currently underway together with the subproccessor. The default SMS sender is changed from a phone number to "Viedoc" to reduce the risk of texts being treated as spam.
Since 10.30 CET today text messages are delivered as expected to numbers with country codes previously listed.
The User Administration Log (Excel) and Log of Users and Roles (PDF) cannot currently be generated for studies containing more than 100 users. This issue was introduced in the 4.90 release and affects all instances where version 4.90 has been deployed. The root cause has been identified, and a corrective fix is currently in preparation.
A fix has been developed and is currently undergoing testing. The fix is planned to be deployed as a hotfix.
This issue has been resolved with the release of Viedoc 4.90.1, which was deployed to all production instances on 2025-12-16. The User Administration Log (Excel) and Log of Users and Roles (PDF) can now be generated as expected, including for studies containing more than 100 users.
Since around 12.30 CET, 1st of December, Viedoc is not sending any emails from the EU and US training and production instances due to a service disruption. The issue has been identified and we are working with our subprocessor to resolve the issue. All emails triggered during the affected period have been queued and will be delivered. The incident is caused by a malfunction at one of our subprocessors.
The email queue is currently being processed, and messages are being delivered, although with a temporary delay.
The queue had been fully processed at around 9.20 PM CET on 2025-12-01. Since that time emails have been delivered without delay.
Between 13:22 CET and 14:12 CET on 25 November 2025, Viedoc did not send any emails from the EU training and production instance due to a service disruption. The issue was identified and fully resolved at 14:12 CET, after which email processing resumed as normal. All emails triggered during the affected period have been queued and will be delivered. The incident was caused by a malfunction at one of our subprocessors.
The email queue is currently being processed, and messages are being delivered, although with a temporary delay.
The queue had been fully processed at around 5.30 PM CET on 2025-11-25.
Operational disturbance observed on the Chinese instance (clinic.viedoc.cn). Resolved
The planned release of version 4.89 to the CN training and production instances has been cancelled due to unforeseen technical issues specific to the Chinese server environment. As the 4.90 release is already scheduled for December, we will proceed directly from version 4.88 to 4.90 for the CN training and production instances. The impact to end users is minimal, as version 4.89 contains only non-critical bug fixes. No other instances or regions are affected by this change. We appreciate your understanding and apologize for any inconvenience this may cause.
From approximately 23.44 UTC to 23.54 UTC, users faced issues logging in to the European production instance. Investigations to find the root cause are ongoing. No other instances were affected.
Description From approximately 23.44 UTC to 23.54 UTC on 2025-11-02, users were unable to log in to the European production instance. It is possible that users faced sporadic issues logging in also between approximately 23.00 UTC and 23.44 UTC. No other instances were affected. Cause The cause of the issue was an unexpected error in the identity provider \(idp\), leading to an accumulating memory consumption during the evening \(UTC\) of 2025-11-02. From approximately 23.00 UTC users would face sporadic failures when trying to log in. From approximately 23.44 UTC users could not log in. At 23.54 UTC on 2025-11-02 the idp was restarted, and users could log in as expected again. Corrective Action Alerts were triggered when the idp did not respond and the idp was restarted again at 2354 UTC on 2025-11-02. After the restart users could log in as expected again. Preventive Action Preliminary investigation identified a few improvements that could be made to make the idp more resilient. Implementation of these improvements have been initiated while the investigation of the issue continues.
Emails triggered from EDC actions in Viedoc has been failing sporadically on the EU production instance since approximately 9AM CEST 2025-09-02. No other instances are affected. Investigations are ongoing.
The issue has been identified and a fix is being implemented.
A fix has been implemented and emails are now being sent as expected. We will continue to monitor the situation.
Since approximately noon CEST, emails triggered from the EDC has been delivered as expected.
**Description** Between 9AM CEST on 2025-09-02 and noon CEST 2025-09-03 there were sporadic failures in email alerts expected to be triggered from Viedoc on the EU production instance. No other instances were affected. Customers were initially informed through the status page at 15.56 CEST on 2025-09-03. **Cause** On Tuesday at 9AM CEST on 2025-09-02, there was a very high load on the system. The high load on the system caused worker function apps, used to handle various processes in the EDC, to scale out to the maximum use of 20 instances in parallel. This caused failures, and in combination with in-built retry policy of failing actions, there was an overload on the system. While form save and form view continued to work as expected throughout the duration of the issue, there were sporadic errors in specific system actions in the post-processing step. * EDC email alerts, configured in the CRF design, were sporadically not sent during this time window. It did not affect all emails. * Terms defined to be coded in the medical coding tool sporadically failed to sync to Viedoc Coder. It did not affect all terms. **Corrective action** Once the root cause of the exceptions was identified, the worker function app was gradually scaled down to run on 4 instances to avoid having too many actions ongoing at the same time. This work begun around 11AM CET on 2025-09-03 and an immediate improvement could be seen. Around noon 2025-09-03 the situation was completely resolved, and all emails could be sent as expected again. Email alerts not sent during this issue could not be re-sent and will also not show up in the communication log. All terms that had not synced properly with Viedoc Coder during the issue, were synced with Viedoc Coder on the afternoon of 2025-09-03 and 2025-09-04. **Preventive action** Scaling down the worker function app to run on 4 instances is not considered to bring any noticeable impact on performance, but will ensure the same overload issue will not occur again. An evaluation will be performed if it is necessary to scale up the worker function app again and if so, how this could be done without causing any similar issues.
When exporting custom reports in xptv8 format through Viedoc Reports, and using an %xpt2loc macro to import the data to SAS, the macro can fail when using long labels for columns. Investigation is ongoing.
The issue has been resolved with a hotfix on Viedoc Reports. When exporting custom reports in xptv8 format from Viedoc Reports, using an %xpt2loc macro to import the data to SAS will now work as expected also when using long labels for columns.
Intermittent network disturbance observed on the Chinese instance (clinic.viedoc.cn). There are problems to reach the instance from time to time. Investigations are ongoing.
Access to Chinese instance is stable again. Continue monitoring status.
This incident has been resolved.