Ionos Cloud outage history

We are rolling out a security update across our Compute infrastructure to address CVE-2026-99999. A runtime mitigation has already been applied to a large portion of our fleet, and we are now deploying the permanent kernel fix across all remaining hosts. The permanent fix is applied via live migration to avoid service disruption. In rare cases, virtual machines that cannot be live-migrated may experience a brief restart.

The security update for CVE-2026-99999 has been applied across our Compute infrastructure. All hosts are now protected. We will continue to monitor for any issues. If you experience any problems, please contact our support team.

We are currently experiencing intermittent issues with our phone availability, which may prevent our agents from connecting to incoming calls. We are actively working to resolve this as soon as possible. If you are unable to reach the team by phone, please open a ticket via email or the DCD Support form.

We are currently investigating an issue with our Nextcloud Workspace.

Due to a critical, industry-wide PHP security vulnerability, IONOS has executed an emergency temporary shutdown of all Nextcloud-based services. Services were shut down immediately to protect customer data and infrastructure, and will be restarted as soon as the official PHP patch is available, tested, and fully implemented. We estimate that the service will be available again in the next 2-3 hours.

Our Nextcloud-based services have been restored. As part of resolving the current PHP security vulnerability, we have preventively suspended all administrator accounts on your system that were changed and/or newly created within the last 24 hours. What does this mean for you? - Review required: Please check your system's user management for any unknown or suspicious accounts. - Reactivation: If you created a legitimate administrator account yourself in the last 24 hours that is now suspended, please briefly contact our support team so we can unlock it after verification.

We are currently investigating an issue potentially affecting kube-api availability as well as issues when trying to schedule stateful workflows. Customers may experience delays in NodePool scaling and maintenance operations. Our engineering team is actively working to resolve the issue as quickly as possible. We will provide regular updates as new information becomes available.

We are continuing to investigate this issue.

We have identified the source of the issue. Our Compute and MK8s Team have responded and are working on mitigation. We will keep this status page updated. We aim to publish the next update within 30 mins, or when we can share a significant update.

Mitigation rollout is completed and all systems are operational again. We are setting this Incident into monitoring state and continue to observe the situation.

System are back running & stable now, this incident is resolved.

We are currently experiencing intermittent issues with our phone system, which may prevent our agents from connecting to incoming calls. We are actively working with our vendor to resolve this as quickly as possible. If you are unable to reach the team by phone, please open a ticket via email or the DCD Support form.

The issue has been resolved and phone lines should operate normally again.

We are currently investigating an issue affecting persistent volume management (PVC) and node provisioning within Managed Kubernetes clusters. As a result, some workloads may experience prolonged timeouts during volume attachment (AttachVolume timeouts) or delays in NodePool scaling and maintenance operations. Our engineering team is actively working to resolve the issue as quickly as possible. We will provide regular updates as new information becomes available.

We have identified the source of the issue. Our Compute and MK8s Team have responded and are working on mitigation. We will keep this status page updated. We aim to publish the next update within 30 mins, or when we can share a significant update.

Mitigation is still progressing. We expect completion within next 30-45 minutes.

Mitigation rollout is almost completed on all hosts. We are setting this Incident into monitoring state and monitor the remaining rollout. Customer impact should be mitigated.

We are marking this incident as resolved and will publish an RCA once it is compiled.

We are writing to inform you that we have been experiencing sporadic connection issues and substantial delays on packet delivery. Network technicians have started working on the occurrence immediately after detection and will isolate the problem and solve the issue as quick as possible. However, it is possible that there will be a certain degradation in connection quality affecting individual virtual resources. We will inform you as soon as the functionality has been restored.

A fix has been implemented and we are monitoring the results.

We are marking this incident as resolved, as no further disruptions or degradations were detected after the mitigation was deployed. Our Network Team is currently working on establishing the Root Cause. The result of this investigation will be published here.

The DCD is currently unavailable. We are currently investigating this issue and will keep you updated.

This incident has been resolved.

There may be periods of increased waiting times when calling IONOS Cloud Support by phone. We ask Customers and Partners to contact Cloud Support via the DCD Form or Email, instead. Thank you.

We are marking this note as resolved.

The AI Model Hub API is currently unavailable. We are working on restoring the service.

This incident has been resolved.

The AI Model Hub inference API is currently unavailable. API requests are returning 503 errors across all model endpoints (chat, embedding, rerank, image). This is related to the ongoing Kubernetes infrastructure issue in Frankfurt. Our team is actively working on restoration. We will provide another update within 30 minutes.

This incident has been resolved.

Customers may experience connection problems to the control plane and degraded functionality of kubernetes

Our engineering team has identified the root cause of the Kubernetes service degradation and is actively working to restore full functionality. Customer workloads in affected locations may be experiencing disruptions, including unavailability of running services. Cluster management operations -- deploying, scaling, and accessing the Kubernetes API -- are also impacted. We will provide another update within 60 minutes.

Our engineering team has identified the root cause of the Kubernetes service degradation and is actively working to restore full functionality. Customer workloads in affected locations may be experiencing disruptions, including unavailability of running services. Cluster management operations -- deploying, scaling, and accessing the Kubernetes API -- are also impacted. We will provide another update within 30 minutes.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

The DCD is currently unavailable, displaying an endless "Loading DCD..." page. We are currently investigating this issue, and will keep you updated.

The DCD is now available again, and we are monitoring the situation.

We are continuing to monitor for any further issues.

After hours of monitoring incident the DCD is fully stable and operation

We are writing to inform you that we have been experiencing sporadic connection issues and substantial delays on packet delivery. Network technicians have started working on the occurrence immediately after detection and will isolate the problem and solve the issue as quick as possible. However, it is possible that there will be a certain degradation in connection quality affecting individual virtual resources. We will inform you as soon as the functionality has been restored.

On top of the network issue in TXL, there may be delays in provisioning.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We are currently experiencing control plane instability affecting our managed Kubernetes clusters. While existing application workloads continue to run normally, administrative and management capabilities are currently degraded. Our engineering team is actively working on a resolution, and we will provide updates as new information becomes available.

We have successfully identified and mitigated a high-latency condition in our core storage infrastructure that temporarily impacted the responsiveness of the Kubernetes control plane. Our engineering team is currently executing a phased reconciliation and rollout process to gracefully restart the management instances and restore full functionality. Existing application workloads continue to run normally and remain unaffected.

The incident has been mitigated successfully at approx. 19:00 UTC, 23.09.2026. The teams involved are currently investigating the root cause and we will publish a report here once it is compiled.

We are currently investigating network connectivity alerts related to our TXL datacenter. While connectivity appears to be restored already, we are actively monitoring the situation to ensure stability. We will provide updates as new information becomes available.

We see indications of a recurrence of the issue and are setting the status page to active. Our Network Team is investigating and we will share updates here as they become available. The issue is currently negatively affecting Compute, Provisioning and Network Components.

We are continuing to work on a fix for this issue.

Our Network Team has identified an issue related to a recent router replacement and applied a mitigation. Compute and Storage systems are showing signs of recovery. We are actively monitoring the environment until all services are fully restored

We are currently investigating potential residual impact on DBaaS Services. Compute and Block Storage have reported full recovery.

DBaaS service has recovered, as well. We are setting the incident into Monitoring status, while the teams are ensuring that all affected services operate normally.

We are marking this incident as resolved

While phone support will generally be available, our Support capacity is reduced due to multiple cases of sick leave. We want to inform you that our phone support will be limited during the following time slots, leading to increased response times on the phone channel. In these cases, please reach out per email. Thank you.

We have people available now, Support is reachable as usual.

While phone support will generally be available, our capacity is reduced due to multiple cases of sick leave. We want to inform you that our phone support will be limited during the following time slots, leading to increased response times on the phone channel. In these cases, please reach out per email. Thank you.

We still run short on coverage of service, and we are currently doing changes on our ticketing system. So we kindly ask for your understanding if Support responses are not as timely as you are accustomed to. Thank you.

We managed to fully re-establish our service, Support is back. thank you for your patience.

For our partners with subcontracts: We found that subcontracts cannot access the new Frankfurt location "de/fra/1". We are currently working on a fix to make this available, we will keep you informed when it's done.

Wir konnten das Problem beheben, sodass die neue Region nun für alle Verträge verfügbar ist. Wir monitoren, ob es dazu noch weitere Rückmeldungen gibt.

This incident has been resolved.

The DCD is currently unavailable, displaying an endless "Loading DCD..." page. We are currently investigating this issue, and will keep you updated.

The DCD is now available again, and we are monitoring the situation.

We are marking this incident as resolved. Our network team is investigating negative impact on the IAM Services caused by a change rollout. We will update the Status Page once the root cause has been established.

# **Preliminary Root Cause Analysis** This Root Cause Analysis is preliminary as research is still being conducted to determine the technical root cause of the incident. ## **What happened?** On August 31, 2026, between 15:00 UTC and 15:43 ITC, and again between 16:32 UTC and 16:35 UTC, customers were unable to reach IONOS Cloud's Identity and Access Management \(IAM\) service and the Data Center Designer \(DCD\), which is dependent on this service. The disruption affected direct logins, partner and reseller portal access, and associated management consoles. The total customer-facing impact lasted approximately 48 minutes across both intervals. Cloud APIs \([api.ionos.com](http://api.ionos.com)\) remained fully operational throughout the incident. The underlying application services were healthy at all times - the failure was confined to the network edge layer. ## **How was this possible? \(Root Cause\)** During a scheduled maintenance window on August 31, 2026, a network configuration update was applied to the edge network infrastructure with the intent of optimizing routing filters. The configuration was verified as correct prior to and during application. Following the rollout, a routing propagation anomaly emerged on one edge network switch, causing asymmetric routing behavior: incoming TCP connection requests from clients were silently dropped \(black-holed\) at the network edge before reaching the application cluster. Because the application services themselves remained up and healthy, this failure mode was not immediately visible through internal health checks - the services were isolated from receiving inbound public internet traffic rather than failing. The root cause of why this specific switch exhibited asymmetric routing behavior following an otherwise valid configuration change remains under active investigation. Whether this was triggered by a switch platform behavior or a software version-specific bug is being determined through staging environment reproduction. ## **What are we doing to prevent recurrence?** ### **Immediate Actions** * **Configuration Rollback:** Upon identifying the routing anomaly, a full rollback of the network configuration was executed across all affected edge switches. Routing announcements and TCP reachability of the public production IP addresses were verified following the rollback. All affected services - DCD, Partner Portal, Reseller Portal, and IAM - were confirmed fully operational by 16:35 UTC. \(DONE\) ### **Short-term** * **Staging Environment Replication:** Detailed test cases are being executed in the staging environment to reproduce the exact routing propagation behavior under the same switch configuration conditions. The goal is to determine whether the anomaly is attributable to a specific software version bug or a switch platform behavior, so that the precise failure condition can be isolated and addressed before any future rollout. ETA: Within two weeks ### **Mid-term** * **Revised Rollout Strategy:** Based on the findings from staging, a revised rollout approach will be designed to ensure that any future application of these routing filter optimizations can be performed with greater stability guarantees - including more granular validation checkpoints between switch-level changes. ETA: October 2026 ## **Closing remarks** We recognize that loss of access to IAM and DCD carries real operational impact. The fact that the underlying services were healthy throughout is not a mitigation of that impact. We are committed to ensuring that the root cause is fully understood before any re-attempt of the original change, and that the revised rollout strategy addresses the conditions that led to the asymmetric routing behavior. We thank you for your patience while we conclude the investigation.