Historique des pannes de Otava

Service Impacting: Yes Description: OTAVA engineers are investigating an issue impacting DRaaS powered by Veeam (OCC) services in the Mid-Michigan (MM1) data center. The affected servers are being rebooted as part of the recovery efforts. Additional updates will be provided as more information becomes available. If you have any questions or concerns regarding this maintenance, please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

OTAVA engineers are investigating an issue impacting DRaaS powered by Veeam (OCC) services in the Mid-Michigan (MM1) data center. Cloud Connect jobs are currently failing, and a Severity 1 case has been opened with the vendor for further investigation.

Start Time: 7/3/2026 - 6:15 PM ET End Time: No restoration estimate at this time. Type: Utility Disruption Service Impacting: No - Due to the redundant nature of the design, no impact is expected. Description: At 6:15 PM we experienced a utility power disruption at the MD1 data center. Power was successfully transferred to generator power. DTE does not have a utility restoration estimate at this time. We have staff monitoring the situation. We will provide updates once utility power has been restored. If you have any questions or concerns regarding this maintenance, please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

Utility power has been restored. All systems have returned to normal operation.

Service Impacting: Yes, jobs targeting IN1 GEN3 DPaaS. Description: We are currently investigating a management connectivity issue affecting IN1 GEN3 DPaaS, which has become unresponsive to management access. At this time, our team is assessing the cause of the issue and working to restore connectivity as quickly as possible. We will provide additional updates as more information becomes available. We apologize for any inconvenience and appreciate your patience while we work to resolve this issue. If you have any questions or concerns regarding this incident, please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

The incident has been resolved, and jobs are functioning as expected.

Start Time: 06/29/2026 at 8:30AM ET Type: Loss of Redundancy - Compute IOM Service Impacting: No, due to the redundant nature of the chassis connectivity, no service impact is expected. Description: OTAVA® engineers have identified the issue and are working to correct it a.s.a.p. If you have any questions or concerns regarding this incident, please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

This incident has been resolved.

Start Time: 06/24/2026 at 8:30AM ET Type: Loss of Redundancy Service Impacting: No, the circuit gracefully failed over to the redundant circuit, but we are currently operating on a single circuit. Description: OTAVA® Internet provider is in the process of fixing the disconnect between Ashburn (VA4) to Austin (AU1). If you have any questions or concerns regarding this incident, please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

We are continuing to monitor for any further issues.

The ISP has confirmed that there was a fiber cut. The circuit has been repaired and is back in service.

Ann Arbor (AA2) Internet connection loss of redundancy - 06/24/2026 Start Time: 06/24/2026 at 12:15 EST Type: Loss of redundancy [ISP Fiber Cut] Service Impacting: No, the circuit gracefully failed over to the redundant circuit, but we are currently operating on a single circuit. Description: OTAVA® Internet provider is in the process of fixing the fiber cut to the Internet circuit to Ann Arbor (AA2). If you have any questions or concerns regarding this maintenance, please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

A fix has been implemented and we are monitoring the results.

We are continuing to monitor for any further issues.

The circuit has been repaired and it is back in production.

Austin (AU1) to Las Vegas (NV1) Transport Circuit lost of redundancy - 06/10/2026 Start Time: 06/10/2026 - 11:22 AM ET Type: ISP Fiber Cut Service Impacting: No, the circuit gracefully failed over to the redundant circuit, but we are currently operating on a single circuit. Description: OTAVA® Internet provider is in the process of fixing the fiber cut to the transport circuit between Austin (AU1) and Las Vegas (NV1). If you have any questions or concerns regarding this maintenance, please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

The fiber cut has been repaired and the Austin to Las Vegas transport link is back up.

Denver (DEN2) OTAVA® DRaaS powered by Veeam (OCC) Outage - 6/10/2026 Service Impacting: Yes, jobs targeting Denver OCC may fail. OTAVA® engineers are investigating an issue with DEN2 Cloud Connect components. Client jobs targeting DEN2 Cloud Connect may fail. If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

A fix has been implemented and we are monitoring the results. Please retry any failed jobs and they should succeed. If you have any questions or concerns please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

This incident has been resolved.

A fix has been implemented and we are monitoring the results. OTAVA engineering is aware of the outages and will be providing updates as more information becomes available.

OTAVA engineers will be failing over to a secondary node at 5PM ET. Clients may experience downtime during the failover.

We are continuing to monitor for any further issues.

OTAVA is working with our vendor to investigate this issue. We have enabled additional logging to support our troubleshooting efforts and will provide an update as our investigation progresses.

We are continuing to work on a fix for this issue.

Engineers have implemented a fix that will prevent the issue from recurring.

This incident has been resolved.

We are currently investigating this issue.

Engineers are currently monitoring network traffic for stability.

This incident has been resolved.

Denver (DEN2) OTAVA® DRaaS powered by Veeam (OCC) Service Partial Outage - 5/27/2026 Service Impacting: Yes, jobs to Denver OCC may fail. OTAVA® engineers are investigating an issue with DEN2 Cloud Connect components.  Client jobs targeting DEN2 Cloud Connect may fail. If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

The issue has been identified. OTAVA® engineers are working with the vendor to apply the least-impactful fix.

We are continuing to work with the vendor on this issue. We are formalizing a procedure and will implement the fix following process approval. We will provide more details as they are available.

At this time, a fix has been implemented. A service provider rescan may be necessary from on-premise VBRs set up to send cloud connect copies to Denver. Some copy job tasks may still be failing, so we will provide updated guidance as we find the solution for those errors. At this time, most jobs should be working.

This incident has been resolved. If you have any questions or concerns regarding your cloud connect backups or replications please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

OTAVA Engineers are currently investigating this incident. OTAVA will provide updates as soon as they become available. Server performance in the AU-PUBVC2 environment may be degraded while engineers perform remediation tasks.

The OTAVA engineer identified the issue and is implementing the fix.

This incident has been resolved.

Service Impacting: Yes , Customers will not be able to access VSPC. OTAVA® engineers are investigating an issue with DEN2 VSPC (vac.newcloudnetworks.com).  Clients may not be able to access the VSPC to manage or review backup jobs or services.   However, jobs will continue to run normally, as they do not rely on the VSPC to run. If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

OTAVA® engineers have found a solution and are applying the fix. VSPC (vac.newcloudnetworks.com) will be accessible after resolution verification. As a reminder, jobs will continue to run normally, as they do not rely on the VSPC to run. If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

Otava Engineers are currently investigating the issue. Otava will provide updates as they become available. Impact: We are investigating reports of degraded network access for customers in the Mid-Michigan datacenter.

We are continuing to investigate this issue.

We are continuing to monitor for any further issues.

We are currently investigating reports of degraded network connectivity impacting customers hosted in the Mid-Michigan datacenter. Further updates will be provided as more information becomes available.

We are continuing to investigate this issue. Further updates will be provided as more information becomes available.

A fix has been implemented and we are monitoring the results.

We are currently investigating this issue.

OTAVA network engineers have identified the issue and are currently reviewing several implementations for remediation. Network connectivity may continue to be unstable as changes are applied and tested.

OTAVA engineers continue to investigate and mitigate the ongoing connectivity degradation. Customers may continue to experience intermittent connectivity issues while remediation efforts are underway. We will provide additional updates as more information becomes available.

OTAVA engineers are continuing to mitigate the ongoing connectivity degradation. Customers may continue to experience intermittent connectivity issues while remediation efforts are underway. We will provide additional updates as more information becomes available.

OTAVA engineers have identified the cause of the disruption and have implemented measures to remediate the issue. We are seeing connectivity return to normal at this time. We will continue to monitor for anomalies over the next 24 hours.

OTAVA engineers are investigating additional reports of latency. Customers may continue to experience intermittent connectivity issues while remediation efforts are underway. Additional details and information will be made available as quickly as possible.

OTAVA Engineers have and continue to take appropriate and proactive measures to address the recent internet connectivity degradation and are actively working with our upstream Internet Service Providers to further strengthen protections against potential denial-of-service (DoS) events. At this time, safeguards are being implemented to enhance service stability as we continue to troubleshoot and monitor the environment to ensure ongoing performance. Our teams remain fully engaged with the ISPs and will provide updates as needed. Please Note: Customers who are using Cloudflare proxy services still may be experiencing degradation of services. OTAVA engineers are working with the impacted ISP to resolve the issue.

OTAVA engineers are continuing to engage with upstream ISP engineers to resolve on-going issues. Additional details and information will be made available as quickly as possible.

OTAVA engineers are continuing to engage with upstream ISP engineers to resolve on-going issues and implement mitigation tools. Additional details and information will be made available as quickly as possible.

OTAVA engineers are continuing to adjust upstream ISP remediation methods to resolve on-going connectivity issues. Additional details and information will be made available as quickly as possible.

OTAVA and ISP engineers continue to monitor and validate remediation efforts related to the connectivity degradation.  Some customers may continue to experience connectivity issues and are encouraged to open a support ticket or contact OTAVA Support directly via the support line. Please include the affected destination IP address, source IP (if available), and protocol details so our teams can review and implement any necessary whitelist adjustments as quickly as possible with the ISPs.

ISP engineers are whitelisting IPs while OTAVA engineers validate connectivity.  Some customers may continue to experience connectivity issues and are encouraged to open a support ticket or contact OTAVA Support directly via the support line. Please include the affected destination IP address, source IP (if available), and protocol details so our teams can review and implement any necessary whitelist adjustments as quickly as possible with the ISPs.

ISP engineers are continuing whitelisting IPs while OTAVA engineers validate connectivity. Some customers may continue to experience connectivity issues and are encouraged to open a support ticket or contact OTAVA Support directly via the support line. Please include the affected destination IP address, source IP (if available), and protocol details so our teams can review and implement any necessary whitelist adjustments as quickly as possible with the ISPs.

ISP engineers have whitelisted IPs while OTAVA engineers validated connectivity. OTAVA customers who are still experiencing issues, please contact OTAVA support at your earliest convenience. Please include the affected destination IP address, source IP (if available), and protocol details so our teams can review and implement any necessary whitelist adjustments as quickly as possible with the ISPs.

ISP engineers have whitelisted IPs while OTAVA engineers validated connectivity. OTAVA customers who are still experiencing issues, please contact OTAVA support at your earliest convenience. Please include the affected destination IP address, source IP (if available), and protocol details so our teams can review and implement any necessary whitelist adjustments as quickly as possible with the ISPs.

DDoS attack activity has decreased, and ISP mitigation efforts remain active while connectivity validation continues. OTAVA and ISP engineers are continuing to review customer traffic and application functionality. Customers who are still experiencing issues are encouraged to contact OTAVA Support and provide affected destination IP addresses, source IPs (if available), protocol/port information, timestamps, and application details so our teams can investigate and coordinate any additional mitigation or whitelist adjustments with the ISP as quickly as possible.

The DoS activity has subsided, and network stability has largely returned to normal operation. OTAVA and ISP engineers continue to monitor the environment and validate connectivity across customer services. Customers who continue to experience connectivity or application issues are encouraged to contact OTAVA Support with affected destination IP addresses, source IPs (if available), protocol/port information, timestamps, and application details so our teams can review and coordinate any additional whitelist adjustments with the ISP if necessary.

ISP engineers are currently tuning down DoS mitigation measures to enhance network stability. Customers who continue to experience connectivity or application issues are encouraged to contact OTAVA Support with affected destination IP addresses, source IPs (if available), protocol/port information, timestamps, and application details so our teams can review and coordinate any additional adjustments as necessary.

This incident has been resolved.

Event: The Mid-Michigan \(MM\) data center experienced a Denial of Service \(DoS\) event impacting connectivity to the data center. Start Time: 5-13-2026, Wednesday 7:13 PM ET Resolved Time: 5-18-2025 , Monday 3:40 PM ET Cause: A Denial of Service \(DoS\) event impacted network connectivity at the Mid-Michigan site. Resolution: Internet Service Provider \(ISP\) and OTAVA engineers implemented routing and networking configuration changes such as geoblocking, Access Control Lists \(ACL\) changes, black hole routing and service provider AI-Powered DoS Protection. More information will be provided when the RCA is available.

Service Impacting: Yes – Encore services may be unavailable until further notice. Description: OTAVA engineers are currently investigating an unexpected outage impacting Encore (portal.otava.com). Related services may not be available at this time. If you have any questions or concerns regarding this incident, please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

Start Time: 4/15/2026 - 3:00 PM ET End Time: 4/15/2026 - 4:00 PM ET Type: Emergency Service Impacting: Yes - Managed Endpoint (Automate) will be unavailable during this period Description: OTAVA® engineers will be performing a reboot to restore MFA functionality. If you have any questions or concerns regarding this maintenance, please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3

This incident has been resolved.

Type: Service Outage Service Impacting: Yes – Microsoft 365 backups and restores may be unavailable at this time. Description: OTAVA® engineers are currently investigating an unexpected outage impacting Indianapolis (IN1) OTAVA Backup for Microsoft 365 (VBO). Backups and restores may be unavailable at this time. If you have any questions or concerns regarding this incident, please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

A fix has been implemented and we are monitoring the results.

The incident has been resolved. You may need to upgrade your Veeam Explorers to the latest version, if you have not. Please open an OTAVA ticket if you need any assistance.

OTAVA® engineers have identified an issue with a component of OT Monitor and are currently in the process of providing the solution. Service Impacting: Yes - Customers may not receive alerts from OT Monitor until the issue is resolved.

This incident has been resolved.

Denver OCC jobs are currently failing with the error: “The remote certificate is invalid according to the validation procedure.” The issue appears to be related to a recent update of the certificate. Our team is actively investigating the problem and working on a resolution. Impact: Jobs may not complete successfully until the issue is resolved. Next Update: We will provide an update as soon as more information is available.

A workaround has been applied, and Denver OCC jobs have resumed normal functionality. Our team is actively monitoring to ensure stability.

This incident has been resolved.

At approximately 12:38pm ET on 3/13/26 our MD1 - Metro Detroit data center encountered a utility power disruption. All systems successfully transferred to generator power and OTAVA engineers are onsite and will continue to monitor the situation.

The MD1 – Metro Detroit data center remains on generator power. The local utility provider is currently estimating restoration of commercial power by Sunday, March 15, 2026. The OTAVA Data Center Operations team remains onsite and continues to actively monitor facility operations.

Commercial utility power has been restored at the MD1 – Metro Detroit data center, and all systems have returned to normal operations. The facility is stable, and no further impact is expected.