54 Bigleaf incidents · luty 2022 — official updates, affected components, duration and resolution details.
A subset of customers served out of the NYC region may have experienced some packet loss and congestion 12:53PM EDT to 1:16PM EDT to due to an upstream network provider issue. At this time the issues have stabilized while our Network Operations group continues to monitor.
We are continuing to monitor for any further issues.
This incident has been resolved.
W dniu 21 października Bigleaf Networks i podzbiór stron / obwodów klientów Bigleaf zostały dotknięte przez problemy z połączeniami ze względu na niewłaściwą konfigurację z dostawcą podglądacza wyższego szczebla w naszym NYC PoP. Czas uderzenia: 10 / 21 - 12: 54 PM EDT do 1: 16 PM EDT Na dostawcę podglądaczy na wcześniejszych etapach, problemy zostały spowodowane przez błędną konfigurację w miejscu podglądacza i wpłynęły na dużą część połączeń z obiektem podglądacza NYIIX / Telehouse w Nowym Jorku, gdzie wiele sieci i połączenia w chmurze zbiegają się. Chociaż przypadek wpływu na tego operatora podglądającego jest znaczny, Bigleaf Networks był w stanie polegać na alternatywnych dostawcach i ścieżkach, które mogły poruszać się wokół tych kwestii po ich wykryciu. Na podzbiór klientów z Bigleaf wpływ miały te wydarzenia podczas tych zmian routingu i unikanie tej dotkniętej ścieżki sieciowej - co doprowadziło do utraty łączności w niektórych miejscach lub obwodach przez 2-5 minut w czasie dotkniętego uderzenia powyżej. W tym czasie nasz zespół potwierdził, że problemy zostały rozwiązane, ale nadal uważnie monitorujemy.
Automatyczne tłumaczenie oficjalnej aktualizacji incydentu.
19 października, 20 i 22 października, Bigleaf Networks oraz podzbiór sieci / obwodów klientów Bigleaf w naszym NYC PoP zostały dotknięte przez problemy z połączeniem z siecią wyższego szczebla i dostawcą szkieletu z powodu serii awarii zasilania w zakładzie wyższego szczebla w NYC. Czasy oddziaływania: 10 / 19 - 8: 32 PM EDT do 1: 14 AM EDT 10 / 20 - 4: 56 PM EDT do 1: 31 AM EDT 10 / 22 - 6: 43 EDT do 8: 01 EDT W zależności od dostawców, na których miało to wpływ, ich baza danych / sieć w NYC straciła swoją sieć energetyczną w dniu 10 / 19 i miała kilka innych problemów z generatorem zasilania / backup po początkowym niepowodzeniu. Chociaż przypadki przestoju z tym dostawcą są znaczące, Bigleaf Networks był w stanie polegać na alternatywnych dostawcach i ścieżkach, aby ominąć te kwestie po ich wykryciu. Na podzbiór klientów Bigleaf wpływ miały te wydarzenia podczas tych zmian routingu i unikanie tej dotkniętej ścieżką sieciową - co doprowadziło do utraty łączności niektórych stron lub obwodów przez 2- 5 minut w czasie tych dotkniętych czasy uderzenia powyżej. W tym czasie nasz zespół ściśle współpracuje z dostawcą wyższego szczebla w celu zapewnienia rozwiązania wszystkich powiązanych kwestii i nadal ściśle monitorujemy.
Automatyczne tłumaczenie oficjalnej aktualizacji incydentu.
Obecnie badamy łączność z tablicą rozdzielczą Bigleaf Cloud Connect. Łączność sieciowa nie powinna mieć wpływu.
Problem został zidentyfikowany i jest w trakcie wdrażania.
Wprowadzono rozwiązanie i monitorujemy wyniki.
Ten incydent został rozwiązany.
Automatyczne tłumaczenie oficjalnej aktualizacji incydentu.
We are currently investigating an issue with Bigleaf Cloud Connect Dashboard reporting on CPE and API functionality. Note: Network traffic and connectivity should not be impacted.
This incident has been resolved.
We are currently investigating network congestion issues that are affecting a subset of customers served out of our NYC gateway cluster. Customer sites may see performance impact over some ISP circuits, causing intermittent packet loss and latency.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating network congestion issues that are impacting a small subset of sites connected to our ATL gateway cluster. To verify if your site is impacted after logging into the Bigleaf Cloud Connect dashboard , follow this link to a filtered list of your accessible sites that are impacted by these issues.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating network congestion issues that are impacting a subset of sites connected to our LAX gateway cluster.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
A small subset of customers served out of our Los Angeles PoP may have experienced one minute of service interruption or missing performance metrics between 11:02am and 11:05am PST after an emergency change was made to avoid an upstream provider with congestion issues. We are continuing to monitor network performance after these changes, and all connectivity in this region should be performing as expected.
This incident has been resolved.
We are currently investigating an issue with our Bigleaf Cloud Connect Dashboard where it is slow or unresponsive.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We will be undergoing emergency maintenance in our Seattle Gateway Cluster. Expected Impact: No network connectivity impact is expected If you’re curious about the types of maintenance that we perform, feel free to check out the Maintenance section of our SLA here: http://www.bigleaf.net/sla . If you have any questions or concerns about this maintenance process, please let us know.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating an issue with our Bigleaf Cloud Connect Dashboard where it is slow or unresponsive. Customer and network traffic is unaffected.
We have taken steps to mitigate this issue and are seeing improved response times. We are continuing to monitor the situation.
This incident has been resolved.
We are currently investigating an issue with missing metrics and reporting in the Bigleaf Cloud Connect Dashboard.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We will be undergoing emergency maintenance in our Frankfurt, DE Gateway Cluster. Expected Impact: Customers may experience additional latency during this time, and up to three 10-second packet loss events. If you’re curious about the types of maintenance that we perform, feel free to check out the Maintenance section of our SLA here: http://www.bigleaf.net/sla . If you have any questions or concerns about this maintenance process, please let us know.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating slowness and intermittent responsiveness with our Bigleaf Cloud Connect Dashboard. Network connectivity is not affected.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
A small subset of customers served out of our Dallas PoP may experience up to two brief periods of packet loss as our team investigates and works to resolve an issue.
This incident has been resolved.
We are currently investigating an issue impacting a small subset of sites in Atlanta where site and circuit status shows incorrect status of 'offline'. Connectivity and traffic should be functioning as normal despite the alerts, but there may be some brief periods of packet loss while resolving the false alerting issues.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating an issue that is preventing any site configuration changes in the Bigleaf Cloud Connect dashboard from applying the updates to deployed Bigleaf Routers. There should be no impact to site connectivity or alerting.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating an issue as ALL Bigleaf alert emails are not being sent from the Bigleaf Cloud Connect dashboard. This includes StatusPage updates.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
We are continuing to monitor for further issues and implemented a temporary change while working towards a permanent fix. Current Impact: - To reduce immediate impact, only HIGH or SITE DOWN severity alert emails will be sent as expected. - Alert severity of MEDIUM or LOW will not receive an email alert at this time despite any user configurations. - Any and all alerts are being displayed correctly on the Site and Alerts sections of the Bigleaf Cloud Connect Dashboard.
This incident has been resolved.
