Lumaserv 장애 기록

1. 2026년 9월 27일 오후 11:59 UTCinvestigating

   우리는 현재 프랑크푸르트에 위치한 네트워크의 가용성에 영향을 미치는 네트워크 문제를 경험하고 있습니다. 우리는 가능한 한 빨리이 상태 페이지에 업데이트를 제공 할 것입니다.

2. 2026년 9월 28일 오전 12:23 UTCmonitoring

   00:05 UTC 이후, 우리의 프랑크푸르트 위치의 모든 서비스는 완전히 다시 도달 할 수 있습니다. 원인은 우리의 핵심 대패 중 하나에 있는 여정 엔진의 실패였습니다. 과다한 여정 엔진은 자동적으로 걸렸습니다. 스위치 오버 도중, 우리의 프랑크푸르트 네트워크의 범위는 약 사이에서 영향을 미쳤습니다. 23:52 과 00:05 UTC. 네트워크는 현재 중복 여정 엔진에서 안정적으로 실행됩니다. 우리는 실패한 성분을 회복하고 상황을 밀접하게 감시하기 위하여 계속됩니다. 우리는 한 번 전체 중복 복원 된 추가 업데이트를 제공합니다.

3. 2026년 9월 28일 오후 9:31 UTCresolved

   이 사건은 해결되었습니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 8월 11일 오후 6:07 UTCinvestigating

   Our upstream provider at our Zurich cloud environment, has confirmed network issues. They are currently investigating and we will provide updates as soon as available here. Link to our upstream‘s status page: https://status.nine.ch/

2. 2026년 8월 11일 오후 8:01 UTCresolved

   We have been informed that the incident is resolved since 9.06pm.

1. 2026년 7월 22일 오전 9:20 UTCinvestigating

   우리는 현재이 문제를 조사하고 있습니다.

2. 2026년 7월 22일 오전 10:00 UTCidentified

   문제는 식별되고 수정이 구현됩니다.

3. 2026년 7월 22일 오후 5:15 UTCresolved

   사건은 완전히 해결되었습니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 6월 24일 오전 9:01 UTCresolved

   During scheduled maintenance on our network infrastructure on Jun 23, 2026, one of our edge routers in Frankfurt (FRA) became temporarily unavailable by a software overload. This caused a network interruption of approximately one minute, until the second edge router fully took over the routing and connectivity was restored. Although the maintenance was announced, no service impact was expected. The disturbance was briefly wider than the announced scope, affecting connectivity for most of services in the Frankfurt location rather than only the one announced VLAN. We stopped the change immediately and have paused the remaining maintenance while we review the underlying causes, so the work can be completed reliably. We apologize for the any circumstances.

1. 2026년 6월 19일 오전 9:46 UTCidentified

   We have identified packet loss to one of our upstream providers, routes have been switched. We will be monitoring the situation.

2. 2026년 6월 19일 오후 12:36 UTCresolved

   The issue was caused by unstable firmware on the core router of one of our upstream providers. The firmware has now been updated, and the routes have been returned to their previous state.

1. 2026년 5월 5일 오후 9:41 UTCidentified

   The DENIC is currently experiencing a major issue causing all .de Domains to become unresolvable after dns caches run out. DENIC Statuspage: https://status.denic.de/pages/incident/592577eab611ce1e0d00046f/69fa60ef9d12f5057a974f38

2. 2026년 5월 6일 오전 7:35 UTCresolved

   This incident has been resolved.

1. 2026년 3월 12일 오후 5:21 UTCinvestigating

   During routine maintenance work, we encountered a service degration at our Zurich datacenter, which affects one hypervisor and the VM's running on it. We are currently investigating this issue and keep you updated here.

2. 2026년 3월 12일 오후 5:30 UTCresolved

   The issue has been identified and resolved.

1. 2026년 3월 10일 오후 12:23 UTCinvestigating

   At March 10, 2026, 11:13 AM, our monitoring alerted that the customer MySQL database was not available. The issue affects managed web hosting services which are operated within our Kubernetes clusters.

2. 2026년 3월 10일 오후 12:24 UTCinvestigating

   We are still investigating an issue with the VM's running the database.

3. 2026년 3월 10일 오후 12:54 UTCresolved

   The incident has been resolved and the environments are all available again. We will provide further information with our postmortem.

1. 2026년 3월 10일 오후 2:29 UTCresolved

   We encountered packet loss on virtual machines during a network reload in our virtualization environment when provisioning new SDN settings. The issue was resolved by March 8, 2026, 0:30. Moving forward, we will implement measures to ensure that network reloads do not cause packet loss. We will therefore announce the according maintenance frame here on the statuspage.

1. 2026년 2월 16일 오후 6:19 UTCinvestigating

   We are currently experiencing a network outage at our location in Frankfurt. We are currently investigating and provide updates on this page as soon as possible.

2. 2026년 2월 16일 오후 6:30 UTCidentified

   The network is available again since 19:24 CET. We continue investigating and analyzing the route cause.

3. 2026년 2월 17일 오전 9:15 UTCresolved

   The incident has been resolved and no further impact is expected. We will provide details today afternoon.

4. 2026년 2월 17일 오후 5:48 UTCpostmortem

   On February 16, 2026, an incident identified as INC-262 occurred, resulting in a critical network outage that affected all systems in our Frankfurt data centers for a duration of 8 minutes. The incident was triggered by a routine configuration change made to a static route on one of our routers. Prior to implementation, the configuration underwent validation via the operating system \(Juniper Junos OS\), which generally checks for errors. However, due to a bug in the firmware, the system incorrectly accepted a faulty configuration change. 4 minutes after the change was applied, both forwarding engines were reported unreachable by the routing engine. The issue was promptly detected and reported by our monitoring systems. An automatic rollback occurred one minute following the failure due to a lack of manual confirmation, which restored both routing engines and forwarding engines. Safety mechanisms have worked as expected and resulted in a quick recovery completed by 7:27 PM. To preemptively harden our infrastructure against similar issues in the future, we already planned to transition from a redundant routing chassis to completely isolated devices, ensuring full redundancy even in case of such failures. This transition is scheduled for April/May during a planned maintenance. We appreciate your understanding and patience as we work to enhance our systems' reliability.

1. 2026년 2월 11일 오후 1:51 UTCresolved

   At 1:28PM the connection between our locations in Frankfurt and Zurich become degraded since one gateway at our upstreams becomes unavailable. The issue was resolved by 1:49PM completely by changing the gateway's configuration and adapting to the new configuration. Moving forward, we will check for possible improvements to enhance network resilience.

1. 2026년 1월 23일 오후 12:00 UTCresolved

   Starting this morning, we have detected packet loss occurring a few times for a few seconds at carrier level. Our analysis revealed that this is related to incoming or outgoing traffic via Liberty Global (AS6830). As a mitigation measure, we have deactivated this carrier and are using alternative routes. We will monitor the situation closely and wait for a response before reactivating the carrier.

1. 2025년 11월 26일 오전 11:09 UTCresolved

   One of our upstreams was experiencing network issues for about 5 minutes. Redundancy was working as expected and there was no noticeable impact.

1. 2025년 11월 11일 오후 1:11 UTCresolved

   A physical network connection inside our network in our FRA-2 location failed causing packet loss and a disturbance of multiple services. All our links are setup redundantly and the redundancy automatically took over within a few seconds.

1. 2025년 10월 4일 오후 12:17 UTCinvestigating

   One of our uplink providers, IP-Projects (AS48314), is experiencing a network attack. This have lead to packet loss for max. five minutes, starting from 11.42. We have deactivated incoming and outgoing till the network is stable again there. Link to our uplink's status page entry: https://status.ip-projects.de/?page=Issue&id=551

2. 2025년 10월 5일 오후 10:16 UTCresolved

   This incident has been resolved.

1. 2025년 7월 23일 오후 2:13 UTCresolved

   From 22 July 2025 10pm to 23 July 2025 1.30pm, some customers using our DDoS Protected IP-Transit service experienced connectivity issues during active DDoS mitigation. This was due to an incorrect routing configuration by one of our upstream providers. The incident affected IP addresses with active DDoS mitigations during the mitigation period for the attack (typically 30 minutes). The majority of our services remained operational throughout this period. We resolved the issue by extending our routing policies and working with our provider to prevent this configuration problem from recurring. We apologize for any inconvenience this may have caused.

1. 2025년 7월 14일 오후 10:06 UTCinvestigating

   We are seeing issues regarding routes to Cloudflare, we're investigating and will update you as soon as we have further information.

2. 2025년 7월 14일 오후 10:11 UTCidentified

   We identified that Cloudflare DNS has major issues, causing some servers using Cloudflare DNS to have dns resolution issues. This is not an issue on our side but we'll keep monitoring it.

3. 2025년 7월 14일 오후 10:14 UTCidentified

   We identified that Cloudflare DNS has major issues, causing some servers using Cloudflare DNS to have dns resolution issues. This is not an issue on our side but we'll keep monitoring it. Cloudflare has posted the incident on their statuspage: https://www.cloudflarestatus.com/incidents/28r0vbbxsh8f

4. 2025년 7월 15일 오전 7:12 UTCidentified

   We continuing to monitor

5. 2025년 7월 15일 오전 7:13 UTCresolved

   The incident has been resolved by Cloudflare

1. 2025년 4월 2일 오후 1:11 UTCinvestigating

   We are currently experiencing network issues at our Frankfurt locations.

2. 2025년 4월 2일 오후 1:24 UTCmonitoring

   The network issue has been fixed and our network is stable again. We keep monitoring and will provide updates here.

3. 2025년 4월 2일 오후 9:27 UTCresolved

   This incident has been resolved.

4. 2025년 4월 2일 오후 9:27 UTCpostmortem

   At approximately 15:00 CET on April 2nd, 2025, we experienced a brief network disruption affecting our Frankfurt \(FRA\) infrastructure. The incident occurred during a scheduled maintenance window for one of our upstream providers. The network disruption lasted for approximately seven minutes, during which some of our services experienced reduced accessibility. Our network engineering team identified the issue immediately and implemented necessary measures to restore service stability. While we typically handle such maintenance windows without any customer impact, this particular incident resulted in a brief service interruption. We have conducted a thorough review of our maintenance procedures and will be implementing additional safeguards to prevent similar incidents in the future. We understand the impact this may have had on your operations and apologize for any inconvenience caused. Our team remains committed to providing reliable service and continuous improvement of our infrastructure management processes. If you have any questions or concerns, please don't hesitate to reach out to us.

1. 2024년 12월 5일 오후 1:54 UTCinvestigating

   We're currently experiencing a major outage on our compute infrastructure in our FRA-2 location. We're investigating the issue and will provide additional information soon

2. 2024년 12월 5일 오후 2:22 UTCidentified

   We've identified the issue and are working on recovering all services ASAP

3. 2024년 12월 5일 오후 2:27 UTCidentified

   We are currently booting all affected servers again. The services will be back online in the next minutes.

4. 2024년 12월 5일 오후 6:00 UTCmonitoring

   We brought all services back up and are closely monitoring them

5. 2024년 12월 5일 오후 10:14 UTCmonitoring

   All services are running as expected again. We will keep monitoring and let the incident open till we publish an official statement tomorrow.

6. 2024년 12월 6일 오전 9:40 UTCresolved

   All systems and services have been stable since the recovery and the normal operational state has been fully recovered. The root cause was a power outage in one of our locations. We don't have information about the exact cause yet but are in contact with the datacenter provider to get more information about the incident. To ensure that a similar incident can be preempted or mitigated, we will do an internal review and inform you about our steps as soon as this has been completed.

1. 2024년 10월 31일 오전 9:50 UTCinvestigating

   One of our hypervisors became unavailable, we're currently investigating the issue

2. 2024년 10월 31일 오전 10:31 UTCmonitoring

   The hypervisor has been taken out of service and all services have been distributed to different hypervisors. We're closely monitoring the situation and we'll post postmortem once we have all information on what happened.

3. 2024년 10월 31일 오후 12:56 UTCresolved

   This incident has been resolved.