已退化的性能升级 Hypernode 和创建分支
Issues轻微事件
受影响的组件
Hypernode ServersHypernode Toolset / API / Control Panel
- identified
这一问题已经确定,一个解决办法正在实施之中.
- monitoring
一项固定措施已经实施并正在得到监测
- resolved
这一事件已经得到解决.
自动翻译自官方事件更新。
53 Hypernode incidents · 2022年9月 — official updates, affected components, duration and resolution details.
这一问题已经确定,一个解决办法正在实施之中.
一项固定措施已经实施并正在得到监测
这一事件已经得到解决.
自动翻译自官方事件更新。
我们正经历一个DNS故障 由一次DDoS攻击 目标我们的DNS托管伙伴的基础设施。 他们正在积极诊断和减轻袭击。 我们和他们的团队保持密切联系 并监视局势 服务恢复后我们会尽快更新.
该团队已经针对DDOS袭击进行了缓解,并将密切监测局势.
这一事件已经得到解决.
自动翻译自官方事件更新。
目前我们在对Hypernodes(版本修改,白名单,SSH密钥)进行修改时正经历一些退化. 请联系急救人员支援。 这不影响您的服务器或应用程序的一般可用性 .
已推出一个补真,现在将处理遗漏的改动,请注意任何遗漏的动作的 " hypernode-log " .
所有系统正常运行。 我们自动化系统中的一个同步原始 被困在了死状态中 我们实施监控规则,以识别未来类似的问题,同时不断监控死锁系统.
这一事件已经得到解决.
自动翻译自官方事件更新。
We are currently experiencing intermittent downtime affecting services under the *.hypernode.io domain. Our DNS provider is undergoing a large-scale DDoS attack, which is causing instability in domain name resolution. As a result, some requests may fail to resolve or experience delays. We want to emphasize that our platform and infrastructure remain operational; however, access is impacted due to external DNS issues. Our team is actively monitoring the situation and working with the provider to mitigate the impact. We are also exploring mitigation strategies to reduce dependency on the affected service. We will continue to provide updates as more information becomes available.
We experienced a brief period of instability affecting DNS resolution for the *.hypernode.io domain, which may have caused temporary downtime or failed requests. Service has since recovered, and all systems are currently operating normally. Initial indications suggested a possible DDoS attack at the DNS provider level; however, this has not been confirmed. We are continuing to monitor the situation closely and are in contact with the provider to better understand the root cause.
The earlier DNS resolution instability affecting the *.hypernode.io domain has now fully recovered. Stability has been restored and we are no longer observing any degraded performance or resolution failures over an extended period of time.
Part of our datacenter is currently under a DDoS attack, resulting in degraded performance across multiple nodes. We are actively investigating short-term mitigation options.
We have implemented anti-DDoS measures and rebooted the surrounding hypervisors, which has restored normal operations. We will keep monitoring the situation.
Since implementing these measures and closely monitoring the situation, we have not observed any further downtime or service disruptions.
We are currently experiencing issues with creating branchers, performing upgrades, and provisioning ordered Hypernodes. Our team is actively working to investigate and resolve the problem.
We have identified the issue and are currently working on a fix.
A fix has been implemented and we are monitoring the results.
The issue has been resolved. A fix was successfully implemented, and we are continuing to monitor the results to ensure stability.
We are currently experiencing degraded performance in the control panel. This issue has been caused by a portion of our non-critical infrastructure reaching resource limits. While the Hypernodes remain operational, users may notice slower or no responses when using the control panel. Our engineering team is actively working to resolve the capacity constraints and restore full performance. We expect operations to return to normal shortly.
A fix has been implemented and the problems in the control panel have been resolved, we will further monitor the situation.
We are continuing to monitor for any further issues.
Issues has been fixed we don't expect this to occur again
We are experience with our Object Storage service provider. Investigation with provider is ongoing.
The issue has been identified and a fix is being implemented.
This incident has been resolved by our Object Storage service provider.
We experienced a brief network hiccup in one of our datacenters, which may have caused intermittent connectivity issues for approximately 15 minutes. The issue has been resolved, and network stability has been fully restored. Our team monitored the situation closely to ensure no further impact. No further action is required at this time. We will continue to monitor the platform to ensure ongoing stability.
We are currently aware of an issue affecting the control panel on my.hypernode.com. Some users may experience problems loading the “My Hypernodes” page. Our team is investigating the issue and working to determine the cause. At this time, no other services appear to be impacted.
The issue affecting the "My Hypernodes” page has been isolated and resolved. All services are now operating normally.
We are currently experiencing an issue with our control panel on my.hypernode.com. The “My Hypernodes” page is not functioning correctly, and the list of Hypernodes is not loading. Our team is aware of the issue and is actively investigating the cause. At this time, no other services are impacted. We will provide an update as soon as more information becomes available.
The issue affecting the control panel on my.hypernode.com, where the “My Hypernodes” page was not loading correctly, has been resolved. All functionality is operating normally again, and no other services were impacted. We will continue to monitor the platform to ensure stability.
We are currently experiencing issues with several services due to a broader Cloudflare disruption. This outage is affecting our Support ticketing system, and our phone support is currently unavailable. Additionally, some customers may experience 500 errors on their websites as a result of this Cloudflare issue. Cloudflare is currently investigating problems impacting the Cloudflare Dashboard and related APIs. Customers using the Dashboard or API functionality may encounter failed requests or error messages.
Cloudflare has resolved the underlying issue, and all our services are fully operational again. We have confirmed that both our Support systems and phone availability are functioning normally.
Cloudflare is currently experiencing issues routing requests to some Hypernodes. Please note that this is not a Hypernode outage, and the issue must be resolved on Cloudflare’s side. We recommend keeping an eye on Cloudflare’s status page for updates. If this significantly impacts you, a temporary workaround is to disable the Cloudflare Proxy in your Cloudflare DNS settings until the issue is resolved.
The incident has been marked as resolved by Cloudflare and we're expecting no more inconvenience.
We're currently seeing extremely degraded performance across our platform, this seems to be related to our DNS resolving to hypernode.io , we are actively investigating and will update when possible.
We have identified our DNS server is being flooded with packets. We are activating our counter-measures and continue monitoring the situation.
Sadly as we're trying to resolve this as quick we can, an alternative temporary solution is to use the direct IP if you're using an external DNS service, you can get this with using dig with our alternative DNS server, make sure to point your A-record to the resulting IP-address: ``` dig @80.84.227.137 abc.hypernode.io A ``` We hope this won't be needed for long and we can return operations as soon as possible.
As the attack seems to calm down, operations are returning on our nodes and internal infrastructure. We continue working on increasing our stability to mitigate future incidents.
Our DNS infrastructure has been running stable for the past 12 hours. We'll continue to monitor and improve the infrastructure.
We're currently seeing degraded performance for our Cloudflare users running on at least our German dedicated datacenter, we are monitoring the situation and are contact with the relevant third parties
We're seeing no current problems on the datacenter/Hypernode side, and it appears to be an instability with Cloudflare. Please be advised to consider disabling the proxy temporarily and monitoring the situation closely.
The Hypernode cloud platform for the R405 room is currently experiencing degraded performance and may cause incididental downtime, we are aware of this and are monitoring the situation. As soon as we know more we will update our statuspage.
We are continuing to investigate this issue.
Our provider has identified the issue and is working on a solution
Operational activities are being restored across the nodes.
All affected nodes have fully resumed operations. Earlier, our cloud provider in Room R405 experienced a temporary issue with their network-storage layer, which caused downtime on these nodes. The provider quickly diagnosed and resolved the problem. We confirm that no data loss occurred.
We are aware of issues with the Cloud Hypernodes and are in close contact with the datacenter to determine the cause.
We identified a brief downtime in one of the OpenStack availability zones, which triggered our monitoring systems to automatically reboot some nodes. Platform stability is now improving and returning to normal.
A brief network interruption in one of our OpenStack availability zones caused some instances to automatically reboot. The issue has been resolved and platform stability has been restored.
We are currently experiencing DNS disruptions caused by a DDoS attack. This has resulted in intermittent service issues for some users.
We've taken measures on our side to help mitigate the impact of the DDoS attack on our DNS infrastructure. At this time, full performance has been restored, and we are closely monitoring the situation.
The issue has now been fully resolved, and all services are operating normally.
The issue has been identified and a fix is being implemented.
This incident has been resolved.
We are currently experiencing issues with creating branchers, xgrading and providing ordered Hypernodes. Our team is actively investigating and resolving this issue.
The issue has been identified and we are working on a fix.
The issue has been resolved and had to do with resource-limitations with our service partners, we will be monitoring the situation over the coming week but do not expect any downtime at this current moment.