52 Sashido incidents · december 2023 — official updates, affected components, duration and resolution details.
Vores hold undersøger i øjeblikket årsagen til problemet.
Vores overvågningssystem rapporterer, at alle de svigtende ansøgninger er sunde. Vi vil fortsat overvåge proaktivt.
Gode nyheder! Tingene burde være normale igen. Lad os vide, om der er yderligere problemer. Mange tak for din tålmodighed, mens vi arbejdede på at løse dette.
Automatisk oversat fra den officielle hændelsesopdatering.
Vi oplever i øjeblikket et problem, der påvirker udgående transaktionsmails fra Parse Servers i EU-regionen. Vores e-mail levering udbyder, Amazon SES, har midlertidigt pause sende fra en af vores produktionsregioner, mens de gennemgår usædvanlige sende aktivitet. Som et resultat, nogle e-mails såsom kontomeddelelser, verifikation e-mails, adgangskode nulstiller, og andre transaktionsmeddelelser kan blive forsinket eller ikke leveret. Vores systemer forbliver operationelle, og der er i øjeblikket ingen tegn på, at kundedata er blevet påvirket. Vi undersøger aktivt problemet med AWS og gennemgår vores e-mail infrastruktur. Gendannelse af normal e-mail levering er vores højeste prioritet. Vi vil give en anden opdatering, så snart vi har flere oplysninger.
Vi fortsætter med aktivt at arbejde på at løse den transaktionsmæssige e-mail levering afbrydelse påvirker Parse Servers i EU-regionen. Kommissionen bemærker, at LuxOpCo ikke har fremlagt nogen dokumentation for, at LuxOpCo ikke er en kriseramt virksomhed. Vores nuværende fokus er ikke kun at genoprette normal service, men også at identificere den grundlæggende årsag og sikre, at problemet ikke kan gentage sig. Undersøgelsen tager længere tid, end vi oprindeligt havde forventet, og vi forstår, at transaktionsmæssig e-mail levering, herunder kontoverifikation, password nulstiller, meddelelser og andre applikationer e-mails, kan være afgørende for nogle af vores kunders virksomheder. Hvis denne hændelse har en betydelig indvirkning på din produktion ansøgning eller business operationer, bedes du kontakte vores kunde Success Team på support @ sashido.io Vi vil samarbejde med dig direkte for at evaluere og, hvor det er muligt, give en alternativ e-mail levering løsning, mens undersøgelsen med AWS fortsætter. Vores Parse Server infrastruktur og applikationer forbliver operationelle, og vi har i øjeblikket ingen tegn på, at kundedata er blevet påvirket. Vi undskylder for den udvidede forstyrrelse og værdsætter din tålmodighed, mens vi arbejder med AWS for fuldt ud at løse problemet. Vi vil give endnu en opdatering, så snart vi har meningsfulde fremskridt at dele.
Vi har identificeret kilden til den usædvanlige sende aktivitet, der forårsagede Amazon SES til pause udgående e-mail levering fra den berørte produktionsområde. Vores team er allerede begyndt at implementere de nødvendige rettelser og sikkerhedsforbedringer. Når disse ændringer er afsluttet, vil vi validere de berørte systemer for at sikre, at problemet er blevet behandlet fuldt ud, og at normal afsendelse kan genoptages sikkert. Efter denne validering, vil vi koordinere med Amazon AWS for at fuldføre deres gennemgang og gendanne udgående Transactional e-mail levering. Undersøgelsen og oprensningsprocessen har taget længere tid, end vi oprindeligt havde forventet, og vi forstår, at Transactional e-mails - herunder kontoverifikation, password nulstiller, meddelelser og andre applikationer e-mails, kan være afgørende for nogle kunders virksomheder. Hvis denne hændelse i væsentlig grad påvirker din produktion ansøgning eller business operationer, bedes du kontakte vores Support team. Vi vil arbejde med dig direkte for at evaluere og, hvor det er muligt, give en alternativ e-mail levering løsning, mens den endelige oprydning og AWS gennemgang er ved at blive afsluttet. Vores Parse Server infrastruktur og applikationer forbliver operationelle, og vi har i øjeblikket ingen tegn på, at kundedata er blevet påvirket. Vi sætter pris på din tålmodighed og vil give en anden opdatering, så snart oprydning og validering processen skrider videre.
Аmazon AWS har genoprettet sende adgang til de berørte SES-konto, og transaktionsmæssig e-mail levering er nu operationel igen. Vi har bekræftet, at e-mails bliver sendt med succes og vil fortsat nøje overvåge infrastrukturen og sende aktivitet for at sikre, at tjenesten forbliver stabil. På nuværende tidspunkt mener vi, at den umiddelbare afbrydelse af tjenesten er løst, mens overvågningen vil fortsætte som en ekstra forholdsregel. Tak for Deres tålmodighed og forståelse under hele denne hændelse.
Transaktionel e-mail levering forbliver operationel, og vi fortsætter med nøje at overvåge infrastrukturen efter den seneste oprydning arbejde. Som en del af de yderligere sikkerhedsforanstaltninger, vi har indført, ser vi i øjeblikket et lille antal falske positiver, der påvirker nogle legitime kundeapplikationsaktiviteter. Vi gennemgår disse sager løbende og implementerer justeringer så hurtigt som muligt for at minimere enhver virkning og sikre, at kundeprogrammer fortsætter med at fungere normalt. Vi vil gerne takke alle kunder, der har rapporteret uventet adfærd, dine rapporter er yderst nyttige i at give os mulighed for at identificere og behandle disse sager hurtigt. Vi værdsætter også Deres tålmodighed og forståelse, mens vi fortsætter med at styrke beskyttelsen af vores infrastruktur. I nogle situationer kan vores sikkerhedsforanstaltninger midlertidigt være mere restriktive end nødvendigt, men vores prioritet er at beskytte platformen og i sidste ende applikationer og virksomheder hos vores kunder, der kører på den. Vi vil fortsat overvåge situationen nøje og om nødvendigt foretage justeringer.
Denne hændelse er blevet løst.
Automatisk oversat fra den officielle hændelsesopdatering.
We're experiencing an increased rate of unhealthy applications. Currently we are looking into the issue.
We are currently experiencing connectivity issues affecting part of our infrastructure hosted in the FR-PAR datacenter region. Our datacenter provider, Scaleway, has confirmed an ongoing network incident in the same region. According to their latest update, the issue is related to a power problem inside the datacenter, and most services have already recovered while their teams continue working on the remaining affected systems. Some users may experience intermittent connectivity issues, increased latency, or temporary service interruptions. Our team is actively monitoring the situation and will continue to follow Scaleway’s recovery progress closely. We will provide another update as soon as we have more information or once the situation is fully stabilized. Provider incident: https://status.scaleway.com/incidents/qkj49g6g3ykp
A fix has been implemented and we are monitoring the results.
Good news! Things should be back to normal again across the board. Please let us know if there is any further trouble. Thank you so much for your patience while we worked to sort this out.
We're experiencing an increased rate of applications health errors currently looking into the issue.
Our monitoring system report all of the failing applications are healthy. We we will continue monitoring proactively.
The incident is resolved. Apologies for this connection blip. Thank you as always for sticking with us, we really appreciate it.
Our team is currently investigating the root cause of the issue.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
Issue is resolved and for more than 1 hour there are no reports for unhealthy applications.
We're experiencing an increased rate of applications health errors currently looking into the issue.
The issue has been identified and a fix is being implemented.
Our monitoring system report all of the failing applications are healthy. We we will continue monitoring proactively.
Issue is resolved and for more than 1 hour there are no reports for unhealthy applications.
We are currently experiencing an issue where builds and deployments are completing successfully at the infrastructure level, but are not being correctly reflected in the Dashboard for some applications. This appears to be a synchronization issue affecting the Dashboard visibility layer only. There is no impact on the actual build or deployment processes. Our team is actively investigating the root cause.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
Our team is currently investigating the root cause of the issue
We have identified and mitigated the immediate issue affecting multiple hypervisors. The platform is currently stable. We are continuing to investigate the underlying cause of the unusual hardware behavior to ensure long-term reliability. At this stage, there is no active impact, but we are monitoring closely and will share further updates as we learn more.
We continue to monitor the situation closely following the mitigation of the issue affecting multiple hypervisors. The platform remains stable following the applied mitigation. We are continuing to monitor the situation closely and investigate the root cause of the unusual hardware behavior. If you notice anything unexpected or experience any issues, please contact our support team immediately. Our team is actively investigating the underlying cause to ensure long-term reliability.
We continue to monitor the situation closely and investigate the underlying cause. As shared previously, we will provide the next update on this status page as soon as we have any significant new information.
After 10 days with no further anomalies observed, we are closing this incident. Over the course of the remediation, we fully replaced 100% of our database hardware infrastructure in Europe. This included a significant number of migrations, reconfigurations, and coordinated infrastructure changes performed across the platform. The work required a major effort from our engineering team and was executed with a strong focus on maintaining platform stability and minimizing customer-facing impact. Since completion, the platform has remained stable, and the affected services are now running on new and improved hardware. We will continue to monitor the environment closely, but at this stage the incident is considered resolved.
We're experiencing an increased rate of applications health errors currently looking into the issue.
We have implemented a fix for this issue and are currently monitoring the results. We will update once we have confirmed it is resolved.
Good news! Things should be back to normal again across the board. Please let us know if there is any further trouble. Thank you so much for your patience while we worked to sort this out.
We're experiencing an increased rate of applications health errors currently looking into the issue.
All applications have returned to healthy state. We will continue to monitor the situation.
Good news! Things should be back to normal again across the board. Please let us know if there is any further trouble. Thank you so much for your patience while we worked to sort this out.
Our team is currently investigating the root cause of the issue.
All applications have returned to healthy state. We will continue to monitor the situation.
We have some good news to share! Everything is now looking clear from our side and all users should be able to work with their applications without any interruptions. Please let us know at [email protected] if you are still experiencing problems. We apologize for disrupting your day and thank you for your patience.
We're experiencing an increased rate of applications health errors currently looking into the issue.
We have implemented a fix for this issue and are currently monitoring the results. We will update once we have confirmed it is resolved.
Good news! Things should be back to normal again across the board. Please let us know if there is any further trouble. Thank you so much for your patience while we worked to sort this out.
Our team is currently investigating the root cause of the issue.
We have implemented a fix for this issue and are currently monitoring the results. We will update once we have confirmed it is resolved.
Good news! Things should be back to normal again across the board. Please let us know if there is any further trouble. Thank you so much for your patience while we worked to sort this out.
We're experiencing an increased rate of applications health errors currently looking into the issue.
We’ve identified the cause of the increased application errors. The issue originates from our data center provider, who is currently experiencing an outage impacting network and compute stability. Our platform remains operational, but some requests may continue to fail intermittently until the provider resolves the incident. You can follow their official updates here: https://bare-metal-servers.status-ovhcloud.com/incidents/9y4sd9ly0lqb We’ll continue to monitor and update as the situation evolves.
Our monitoring system report all of the failing applications are healthy. We we will continue monitoring proactively.
Good news! Things should be back to normal again across the board. Please let us know if there is any further trouble. Thank you so much for your patience while we worked to sort this out.
We're experiencing an increased rate of applications health errors currently looking into the issue.
We have implemented a fix for this issue and are currently monitoring the results. We will update once we have confirmed it is resolved.
We have some good news to share! Everything is now looking clear from our side and all users should be able to work with their applications without any interruptions. Please let us know at [email protected] if you are still experiencing problems. We apologize for disrupting your day and thank you for your patience.
Our team is currently investigating the root cause of the issue.
A fix has been implemented and we are monitoring the results.
Issue is resolved and for more than 1 hour there are no reports for unhealthy applications.
We are currently investigating this issue.
We’ve identified that the ongoing connectivity issues
We’ve identified that the ongoing connectivity issues affecting some services in our North America datacenter are due to scheduled maintenance on the vRack network infrastructure by OVHcloud. - Maintenance window: Oct 28, 2025, 08:00–10:00 UTC - Impact: Intermittent or total loss of vRack connectivity during the maintenance period. - Current state: Maintenance is still in progress on OVH’s side. - Reference: https://network.status-ovhcloud.com/incidents/pc33ftfc2xxq We are actively monitoring the situation and will update once connectivity is fully restored. Thank you for your patience and understanding.
Our monitoring system report all of the failing applications are healthy. We we will continue monitoring proactively.
Good news! Things should be back to normal again across the board. Please let us know if there is any further trouble. Thank you so much for your patience while we worked to sort this out.
Our team is currently investigating the root cause of the issue.
All applications have returned to healthy state. We will continue to monitor the situation.
Good news! Things should be back to normal again across the board. Please let us know if there is any further trouble. Thank you so much for your patience while we worked to sort this out.
We're experiencing an increased rate of applications health errors currently looking into the issue.
Our monitoring system report all of the failing applications are healthy. We we will continue monitoring proactively.
Issue is resolved and for more than 1 hour there are no reports for unhealthy applications.
We're experiencing an increased rate of unhealthy applications. Currently we are looking into the issue.
All applications have returned to healthy state. We will continue to monitor the situation.
We have implemented a fix for this issue and are currently monitoring the results. We will update once we have confirmed it is resolved.
Good news! Things should be back to normal again across the board. Please let us know if there is any further trouble. Thank you so much for your patience while we worked to sort this out.
