Historia awarii Crowdcast

1. 13 sierpnia 2026 01:04 UTCinvestigating

   Obecnie badamy problem, który uniemożliwia użytkownikom oglądanie wydarzeń z aplikacji mobilnej Crowdcast. Nasz zespół aktywnie pracuje nad rozwiązaniem tego problemu jak najszybciej. * * W międzyczasie możesz kontynuować oglądanie wydarzeń i odtwarzanie z przeglądarki mobilnej lub pulpitu. * * Podzielimy się kolejną aktualizacją, jak tylko zdobędziemy więcej informacji. Dziękuję za cierpliwość.

2. 13 sierpnia 2026 15:29 UTCresolved

   Zidentyfikowaliśmy problem i wprowadziliśmy rozwiązanie. Funkcjonalność zdarzeń w aplikacji mobilnej znów działa. Dziękuję za cierpliwość i zrozumienie.

Automatyczne tłumaczenie oficjalnej aktualizacji incydentu.

1. 3 sierpnia 2026 18:54 UTCresolved

   Crowdcast doświadczył problemu, w którym gospodarze pojawili się na żywo i nagrywali, ale uczestnicy nie mogli oglądać transmisji. Inżynierowie szybko zidentyfikowali przyczynę jako przeciążoną bazę danych. Aby przywrócić usługę, przełączyli konfiguracje bazy danych, co spowodowało krótką (~ 5 minut) przerwę w miejscu przed resetem zdarzenia i odzyskaniem usługi. Ulepszamy zaopatrzenie infrastruktury w duże wydarzenia, aby zapobiec podobnym problemom w przyszłości.

Automatyczne tłumaczenie oficjalnej aktualizacji incydentu.

1. 6 maja 2026 12:05 UTCinvestigating

   The team is actively investigating an issue that is affecting some hosts when broadcasting live. Attendees may have to refresh to view the live feed. We'll update you as soon as we can.

2. 6 maja 2026 13:40 UTCresolved

   We identified and resolved an issue that prevented events from loading correctly on the initial page load. As a result, some users needed to refresh the page before the livestream would appear. This has now been fixed, and events should load properly the first time.

1. 29 kwietnia 2026 16:17 UTCinvestigating

   The team is actively investigating an issue affecting our service that is causing increased buffering in live streams. This does not affect recordings, and we’ll provide an update as soon as possible.

2. 29 kwietnia 2026 17:14 UTCidentified

   We’ve deployed an update that mitigates the issue and are continuing to work on resolving the underlying cause of increased buffering in some sessions.

3. 29 kwietnia 2026 21:11 UTCmonitoring

   We’ve confirmed the issue as originating with our CDN provider, Cloudflare, and have implemented a temporary mitigation. We’re actively working with the provider to resolve the underlying cause and will share another update as soon as more information is available.

4. 30 kwietnia 2026 16:27 UTCresolved

   Our updates remain stable, and we are continuing to implement additional improvements to help ensure that issues at Cloudflare have a reduced impact on our services.

1. 23 kwietnia 2026 14:58 UTCresolved

   We experienced an issue that temporarily prevented some sessions from going live due to a brief compute disruption within our recording infrastructure. The problem was limited in duration, and we are actively implementing a patch to address the underlying cause and help ensure it does not happen again.

1. 17 marca 2026 17:17 UTCinvestigating

   The team is actively investigating an issue affecting our service. We'll update you as soon as we can.

2. 17 marca 2026 17:33 UTCidentified

   We’re currently experiencing a capacity shortage with our cloud provider (Hetzner). Hetzner has run out of available capacity in the region(s) we use, which is limiting our ability to provision additional resources and may impact scaling and recovery actions. • **Current status:** Investigating / Mitigating • **Impact:** Potential service degradation due to constrained scaling capacity (e.g., slower provisioning, delayed failover, or limited ability to add nodes). • **What we’re doing:** Actively working to secure additional capacity, including requesting more allocation from Hetzner and evaluating alternative capacity options. • **Next update:** We’ll share another update as soon as additional capacity is confirmed or our mitigation plan changes.

3. 17 marca 2026 17:51 UTCidentified

   We’ve identified an issue affecting our services where some hosts may experience difficulty going live. Our team is actively investigating, and we’ll share an update as soon as more information is available.

4. 17 marca 2026 17:52 UTCidentified

   ** Identified** We’re currently experiencing a capacity shortage with our cloud provider (Hetzner). Hetzner has run out of available capacity in the region(s) we use, which is limiting our ability to provision additional resources and may impact scaling and recovery actions. • **Current status:** Investigating / Mitigating • **Impact:** Potential service degradation due to constrained scaling capacity (e.g., slower provisioning, delayed failover, or limited ability to add nodes). • **What we’re doing:** Actively working to secure additional capacity, including requesting more allocation from Hetzner and evaluating alternative capacity options. • **Next update:** We’ll share another update as soon as additional capacity is confirmed or our mitigation plan changes.

5. 17 marca 2026 19:39 UTCmonitoring

   We’ve secured additional compute capacity, and the platform is currently stable. We’re continuing to monitor key signals (capacity headroom, provisioning queues, and error rates) to ensure stability remains sustained. • **Current status:** Monitoring • **Impact:** No current impact; capacity constraints have been mitigated for now. • **What we’re doing:** Ongoing monitoring and readiness to scale further if demand or capacity conditions change. • **Next update:** We’ll provide another update if conditions change or once we confirm sustained stability.

6. 17 marca 2026 19:55 UTCresolved

   We’ve confirmed the scaled-up resources are holding under current load, and service remains stable. The immediate capacity constraints are **resolved for now**. • **Current status:** Resolved • **Impact:** No current impact; services are operating normally. • **What we’re doing next:** We’re investigating and designing a multi-cloud capacity strategy to reduce dependence on a single provider’s regional availability and enable provisioning across multiple cloud providers concurrently.

1. 11 marca 2026 14:21 UTCresolved

   At **8:54 AM ET**, we experienced a **9-minute service disruption** in our livestreaming infrastructure that impacted **real-time audience playback**. During this window, viewers may have seen interruptions or inability to watch live content. • **Start time:** 8:54 AM ET • **Duration:** 9 minutes • **Impacted:** Live audience streaming sessions • **Not impacted:** Recording capture and stored playback assets The incident was caused by a previously unobserved failure mode. Mitigation has been applied, and we are implementing additional patches and safeguards to prevent recurrence.

1. 5 marca 2026 10:28 UTCinvestigating

   The team is actively investigating an issue affecting our service. We'll update you as soon as we can.

2. 5 marca 2026 12:27 UTCmonitoring

   We've scaled our systems to handle significantly higher loads from multiple simultaneous large-scale events. Our systems are currently stable, and we continue to monitor them closely.

3. 5 marca 2026 14:32 UTCresolved

   **All systems remain stable.** We have identified the root cause of the cascading issues that occurred under high load and are deploying an update today to prevent this from happening again. Thank you for your continued patience.

1. 13 lutego 2026 01:27 UTCinvestigating

   The team is actively investigating an issue affecting our service. We'll update you as soon as we can.

2. 13 lutego 2026 01:51 UTCidentified

   We’re hitting scale limits on our real-time chat and interaction services due to a large event. We’re increasing capacity now and will restore full service as quickly as possible.

3. 13 lutego 2026 02:53 UTCmonitoring

   Services are coming back online, and we’re monitoring performance.

4. 13 lutego 2026 03:10 UTCresolved

   **Impact:** Some real-time updates were delayed for a portion of users. No data was lost. **Timeline:** Started at 7pm, peaked at 7:10pm, and resolved by 9:30pm ET. **Cause:** A sudden traffic spike from a high-scale event pushed past our autoscaling limits. **What we did:** Added capacity and tuned autoscaling. **Now:** Everything is stable and performing normally. **What’s next:** Stronger scaling safeguards, smarter rate limiting, and improved monitoring.

1. 18 listopada 2025 11:47 UTCidentified

   We’re actively investigating a service disruption caused by our third‑party provider, Cloudflare. This issue is affecting us and many other services, including Zoom, Uber, Twitter, OpenAI, and AWS. We’ll share updates as soon as we have more information.

2. 18 listopada 2025 12:09 UTCmonitoring

   Cloudflare has acknowledged a global outage and you can follow updates [here](https://www.cloudflarestatus.com/incidents/8gmgl950y3h7). We'll continue to update as we have more information.

3. 18 listopada 2025 12:28 UTCmonitoring

   Cloudflare’s ongoing issue is still impacting Crowdcast. Access from some regions is improving, and we’re continuing to monitor and will share updates as we learn more.

4. 18 listopada 2025 13:45 UTCresolved

   Things look stable now—Crowdcast is back to normal. We’ll keep monitoring to make sure we’re fully in the clear.

1. 20 października 2025 08:54 UTCidentified

   A [major outage](https://health.aws.amazon.com/health/status) at Amazon Web Services (AWS) has taken down our [upstream hosting](https://www.vercel-status.com/) provider as well as large portions of the internet. We're monitoring the situation and will update you as soon as we can.

2. 20 października 2025 09:23 UTCmonitoring

   Our systems are back online. We're continuing to monitor the situation.

3. 20 października 2025 09:47 UTCresolved

   All systems are back to normal.

4. 20 października 2025 16:40 UTCidentified

   [AWS is having a massive outage](https://health.aws.amazon.com/health/status) again that's affecting us and [many other platforms](https://downdetector.com/). We'll update you as soon as we can.

5. 20 października 2025 19:53 UTCresolved

   Our systems are back to normal. We're continuing to monitor the situation at AWS for any other updates.

1. 29 września 2025 23:19 UTCresolved

   All systems are back to normal. During a busy period on Crowdcast, part of our system that checks if everything is healthy started timing out because of the extra traffic. Those timeouts made our infrastructure think something was wrong, so it restarted parts of the system. These restarts caused brief disruptions that affected access to event data and led to slower or interrupted service for some live events. We’ve adjusted how the health checks work so they’re more tolerant during heavy traffic and have added safeguards to keep the system stable.

2. 29 września 2025 23:19 UTCinvestigating

   The web API is timing out and the team is investigating.

1. 17 września 2025 17:18 UTCinvestigating

   The team is actively investigating an issue preventing new events from going live. We'll update you as soon as we can.

2. 17 września 2025 18:27 UTCresolved

   The issue has been resolved. The issue was caused by a change from one of our external providers, which unexpectedly disrupted part of our system. While the cause was outside of Crowdcast, our team resolved it quickly and has already put new safeguards in place to prevent this type of issue from happening again.

1. 21 sierpnia 2025 18:57 UTCinvestigating

   We’re experiencing an overall slowdown in the app’s performance. We suspect this may be related to an issue with [Cloudflare’s DNS services](https://www.cloudflarestatus.com/). Our team is actively investigating and will provide updates as soon as possible.

2. 21 sierpnia 2025 19:45 UTCresolved

   We've confirmed that the issues were related to an outage at Cloudflare which has now been resolved, and the app is now operating at normal levels.

1. 9 lipca 2025 14:05 UTCinvestigating

   The team is actively investigating an issue affecting our service. We'll update you as soon as we can.

2. 9 lipca 2025 14:57 UTCidentified

   We've identified that the issue resides with our cloud provider (Hetzner) not having enough compute resources. We're monitoring the situation closely and working on spinning up our servers on other cloud providers. You can monitor one of the issues here: https://status.hetzner.com/incident/8b1ceeb2-9daa-4971-a1a1-5346fb8fc45a

3. 9 lipca 2025 15:57 UTCmonitoring

   We were able to spin up additional resources so some sessions are starting to be able to go live. The core issue with Hetzner remains so we'll update you as soon as it's fully resolved. We recommend running a test event before your scheduled event.

4. 9 lipca 2025 22:51 UTCresolved

   Sessions are back online, and we’re actively adding support for more cloud providers to prevent this kind of issue in the future.

1. 6 lipca 2025 23:49 UTCinvestigating

   The team is actively investigating an issue affecting our service. We'll update you as soon as we can.

2. 6 lipca 2025 23:58 UTCresolved

   All systems are back to normal. We’re still digging into what happened to make sure it doesn’t happen again.

1. 12 czerwca 2025 13:24 UTCresolved

   Our backend services recently encountered rate limits due to an unexpected policy change by a vendor. The issue has been resolved, and we’ve implemented safeguards to prevent similar disruptions in the future. All systems are now fully operational.

1. 13 maja 2025 16:17 UTCinvestigating

   The team is actively investigating an issue affecting our service. We'll update you as soon as we can.

2. 13 maja 2025 16:31 UTCidentified

   We have identified that the issue is with our cloud provider, Vultr, specifically at one of their U.S. data centers that we utilize. We are collaborating with their team to resolve the problem and restore service. We will provide updates as soon as possible.

3. 13 maja 2025 16:58 UTCresolved

   The issue with our cloud provider, Vultr, specifically at one of their U.S. data centers that we utilize, has been resolved. We collaborated with their team to resolve the problem and restore service.

1. 12 maja 2025 23:13 UTCresolved

   Recording and streaming servers are back online. The issue was caused by an unexpected outage with our downstream cloud provider. We’re actively working on transitioning to a more reliable provider to prevent future disruptions.

2. 12 maja 2025 23:13 UTCmonitoring

   We're monitoring an issue with our recording and streaming servers.

1. 5 marca 2025 17:18 UTCidentified

   The team is actively investigating an issue affecting our service. We'll update you as soon as we can.

2. 5 marca 2025 17:25 UTCidentified

   We are currently experiencing an outage due to a third-party service issue. This is impacting the ability to go live in the browser and via RTMP. We recommend notifying attendees from within the event and considering rescheduling if needed. Our team is actively monitoring the situation. Please check the [status page](#) for updates. Thank you for your patience.

3. 5 marca 2025 18:15 UTCmonitoring

   We believe the incident is resolved and we are monitoring.

4. 5 marca 2025 18:18 UTCresolved

   The issue has been resolved. We will continue to monitor