Close Status

This incident has been resolved. The Close app and API are now functioning normally.

This incident has been resolved.

This incident has been resolved.

This incident has been resolved.

# Summary Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. The Close App and API were severely disrupted for all users for 18 minutes between 15:57 and 16:15 UTC on Monday March 16, 2026. Automated systems restored functionality by 16:15 UTC. # Incident Timeline | Time \(UTC\) | Event | | --- | --- | | 2026-03-16 | | | 15:57:00 | **Automated Database Maintenance begins** | | 15:58:42 | **Incident detected by monitoring** | | 15:58:49 | **Escalation acknowledged by Close Engineering On-Call** ​ | | 16:02:41 | **Incident accepted by Close Engineering** ​ | | 16:15:00 | **Affected database restored to normal operation** | | 17:09:22 | **Incident resolved** | # Root Cause and Resolution At ~15:57 UTC an automated database maintenance process began that starved our production database cluster of resources. This caused our primary instance and both replica instances of the production cluster to become unavailable for ~18 minutes. The system recovered and began performing normally at 16:15 UTC. To prevent this from happening in the future Close Engineering has provisioned additional resources to the affected database cluster to make it more resilient to sudden demand spikes. Close Engineering will also perform an audit of the maintenance process' configuration to identify opportunities to reduce its impact on system performance.

The issue has been resolved. Please reach out to support@close.com if you continue to have calling issues after refreshing.

This incident has been resolved.

Close sincerely apologizes for the interruption of our service. We take the stability of our platform very seriously. Below is an explanation of what happened and how we will prevent another such interruption from occurring. ### Impact Between Tuesday 10/21 14:46–17:24 UTC, Wednesday 10/22 7:50–17:00 UTC, and Thursday 10/23 8:00–15:00 UTC, customers relying on real-time updates in Close \(which is critical for proper & timely functioning of our calling features and the Dialer\) experienced delays, outdated data being displayed on their screens, Dialer disconnections, and other unexpected behavior. ### Root Cause and Resolution On Monday 10/20 at 20:09 UTC, a subtle bug was introduced, which resulted in UI clients slowly opening more and more WebSocket subscriptions for our Data Monitoring Service – a component which powers real-time updates for our Activity Reporting pages. Over the next few days, the number of subscriptions kept rising. The rise was slow enough where it didn’t trigger our alerting, but fast enough to eventually start putting a real strain on our infrastructure and saturating our processing capacity. Our first course of action was to scale the supply to meet the demand, but the demand continued to rise more quickly. Because of unrelated rate-limiting changes that looked more directly responsible for the limitation of the supply, our engineering efforts went in a few wrong directions until finally the root cause – the UI bug – was identified and fixed on Thu 10/23 at 5:19 UTC. However, because the fix required clients to reload their UI, intermittent problems continued until enough clients have reloaded, finally resolving the problem on Thu 10/23 at 15:00 UTC. To prevent this problem from ever happening again, we are improving our monitoring, alerting, testing, and in the longer-term the architecture of our WebSocket subscription service. ### Timeline * Mon 10/20 20:09 UTC: A subtle bug is introduced and over the next few days keeps slowly increasing the number of WebSocket-based Data Monitoring Service subscriptions. * Tue 10/21 14:46 UTC: Close receives first signals of Dialer problems where Users hear the callee many seconds before the corresponding Lead Page loads and immediately start looking into the problem. * Tue 10/21 5:24 UTC: First fixes aimed at increasing the allowed rate of processing of the system land in production. The situation appears resolved. * Wed 10/22 7:50 UTC: The problem comes back despite the fixes. Engineers increase the scale of the system to meet the increasing demand, but the demand keeps saturating the scale. Status page is opened at 10:35 UTC \([https://status.close.com/incidents/ydzwc4l6xmhc](https://status.close.com/incidents/ydzwc4l6xmhc)\). Problems continue on and off until they fully subside around 17:00 UTC * Thu 10/23 5:19 UTC: Engineers identify the root cause and fix it. * Thu 10/23 8:00 UTC: Concerning metrics continue to emerge despite the fix. Customer reports follow and Status page is opened at 11:40 UTC \([https://status.close.com/incidents/j49qgd3m049n](https://status.close.com/incidents/j49qgd3m049n)\). Engineers continue working on the impact mitigation and root cause diagnosis until it is found and fixed at 11:47 UTC \(excessive WebSocket Subscriptions for our real-time Data Monitoring Service, coming from UI clients who visited the Activity Reporting page\). However, the fix being picked up requires some idle UI clients to reload, hence the saturated demand continues until 15 UTC despite scaling attempts. * Thu 10/23 15:00 UTC: The problem is officially resolved.

This incident has been resolved.

This incident has been resolved.

The scheduled maintenance has been completed.

This maintenance has been canceled.

The scheduled maintenance has been completed.

The scheduled maintenance has been completed.

The scheduled maintenance has been completed.

The scheduled maintenance has been completed.

The scheduled maintenance has been completed.

The scheduled maintenance has been completed.

The scheduled maintenance has been completed.

The scheduled maintenance has been completed.