Historia awarii Callhub

CallHub services impacted across all geographies, due to issues with Cloudflare infrastructure. We're working with them on resolving these issues. We will keep you posted on the updates. If you have any queries please do send them to [email protected].

We believe the incident is now resolved. We are continuing to monitor for errors to ensure all services are back to normal.

CallHub services are currently operating normally. We are no longer observing any issues across the system.

We are currently investigating the matter with the concerned team and will keep you posted on it. Sorry for the inconvenience.

We have identified the problem and placed a fix to resolve it. Voice calls should work for all CallHub customers. We will share more insights later. We are still monitoring the voice services at our end.

This incident has been fixed and voice calls are working fine. It was a caching issue on one of our servers. We are considering it resolved now. Sorry for the inconvenience caused to you.

We regret to inform you that the CallHub services like calling & texting have been impacted at the moment since 04:50 PM PST. Our technical team is conducting a thorough investigation to identify the root cause of the interruption. We will provide regular updates on the progress. We apologize for any inconvenience this may cause and appreciate your understanding.

The problem has been pinpointed, and a solution is en route. Thank you for your patience as we work swiftly to implement the fix. Stay tuned for further updates. We appreciate your understanding.

The system is now operating, with full restoration expected imminently. Please anticipate slightly longer wait times momentarily, but these will soon decrease. Thank you for your continued patience as we work towards complete normalization.

We are continuing to monitor for any further issues.

We are considering this incident resolved. Root Cause: A sudden surge in database load and high data transfer overwhelmed the system. We've undertaken the necessary steps to address these situations.

On Feb 14, 2024, CallHub users experienced disruption on the CallHub dashboard, encountering HTTP error 403. This issue also affected the performance of the mobile app during the period from 5:16 PM to 5:50 PM PST. Upon conducting an initial investigation, we identified the root cause related to auto-scaling processes. The issue has been promptly addressed, and as of now, all services have been restored to normal functionality. Our team actively monitors the services to ensure stability. We apologize for any inconvenience caused during this period.

RCA - One of the database connection pooler system configuration settings was incorrectly set to lower limit when we upgraded the servers which resulted in failure of application to db server connections when the traffic was high. Impact - System was entirely down from 5:16pm PST to 5:35pm PST - 19min complete outage. System was available with degraded performance (slowness) from 5:35pm PST till about 5:50pm PST - 15min degraded performance.

We are working with the concerned team to get a fix or other feasible solution as soon as possible. Our engineering is also working on a fallback mechanism to avoid such instances. Stay tuned.

The concerned team has implemented a fix, and your services are back to normal. We are still monitoring the system. Sorry for the inconvenience caused to you.

This incident has been resolved.

We are working with the concerned team and will keep you posted here. A fix is being implemented. Stay tuned.

We are still working with the concerned teams on a fix. Please stay tuned.

The concerned team has implemented a fix and the calling service should be back to normal for you. This incident has been resolved.

We are working with the concerned team and will keep you posted here.

The concerned team has implemented a fix and is monitoring the impact. The calling service should be back to normal for you. Still, we are monitoring the system.

This incident has been resolved.

We are actively investigating the problem and collaborating with relevant teams to resolve it quickly.

A fix has been implemented, and the browser call is returning to normal. However, we are still monitoring the system.

The calling service has been fully restored and is now operational.

We briefly observed a substantial slowdown with DB queries between 23:15 & 23:25 and 23:46 & 23:57 UTC on October 18, 2022. After that, there have been no interruptions, and all services are functioning normally. Our team is thoroughly looking into the issue and will shortly present a root cause investigation.

Our engineering team has deployed the fix on servers to avoid such instances in the future. After monitoring the services for the last six hours, we consider it resolved.

Our teams are investigating the matter, and they have identified the issue. Please stay tuned.

We are continuing to work on the fix of this issue.

We have deployed the fix, and CallHub services are resumed now. However, we are still monitoring the situation.

This incident has been resolved.

We are currently investigating the matter and will keep you posted on it.

We identified the issue with one of our third-party services (Cloudflare’s network), and they have deployed the fix. As a result, all CallHub domains are accessible now, and we are still monitoring the situation.

This incident has been resolved and we are closing it for the time.

We are facing an infrastructure issue with one of our vendors (AWS) and working on this matter along with their team. Please stay tuned.

We are continuing to investigate this issue.

The issue has been identified and the concerned teams are working to deploy the fix. Please stay tuned. Sorry for the inconvenience.

A fix has been implemented and services recovery is there. CallHub application should be for you now. We are still monitoring the system for the time.

The issue has been resolved.

We are currently investigating an intermittent issue with latency and timeout errors—one of our Telecom Carriers Voice services got impacted. We are actively working on it. Please stay tuned.

A fix has been implemented and the services are recovering successfully. Voice calls should work for you. Still, we are monitoring for the time.

This incident has been resolved.

You may experience failed calls, latency/time outs, or delayed or failed messages intermittently. We are currently investigating the matter with our Telecom Carrier.

All services are restored and we are monitoring the situation. Please stay tuned.

This incident has been resolved.

With CallHub, a few customers are facing service interruption with voice broadcast and texting. We have identified the issue and are actively working on it. Please stay tuned for further updates.

CallHub Messaging service is functional now and we are still monitoring for the time. Voice Broadcast service is partially restored and it will be normal soon. Please stay tuned!

We have identified that the issue is still occurring, and both services calling and messaging are impacted. As a result, you may be experiencing failed calls, latency/time outs, and delayed or failed messages. The problem is happening at the Telecom Carrier end. We are in constant touch on this. Sorry for the inconvenience.

Both services (Voice Broadcast & Messaging) are back to normal at the Telecom Carrier end. Therefore, we are considering it as resolved for the time being.

We are currently investigating the issue and will keep you posted on it.

After identifying the issue, a fix has been implemented. As a result, the Calling functionality is back to normal. We are still monitoring the system.

The calling functionality is back to normal. We are considering it as resolved for the time being.

One of the web services got halted and we need to restart the service to fix it.

Our engineering team has been alerted and is actively working on it. We will update you posted on it.

SMS & MMS services are resumed as it is fixed by Telecom Carrier. We are still monitoring the situation.

SMS & MMS services are back to normal. We are considering it as resolved. Thanks!

Analytics services are intermittently failing across all geographies. We are actively working to resolve this.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

CallHub Voice & SMS services impacted across all geographies, due to issues with our telecom carriers. We're working with them on resolving these issues. We will keep you posted on the updates. If you have any queries please do send them to [email protected].

We are continuing to investigate this issue.

Cause of the Issue: It came to our attention that one of the accounts using our services was sending misleading and unsolicited texts. This is a violation of our terms of service and the offending account has been suspended. Unfortunately, this incident has impacted our partner carrier poorly. As a result, services have been disrupted for some of our customers. Please visit https://callhub.io/callhub-outage-aug2020/ for more details. Resolution: We are moving affected accounts to a different carrier on request. If you want us to move your account, please reach out to our support team at [email protected] and we will get started immediately. Below article contains the detailed instructions to resolve the issue https://support.callhub.io/hc/en-us/articles/900002213163-CallHub-Outage-steps-to-resolve

This issue is resolved now. Note: If your account was switched to a different carrier, please continue to work with support team in case of any trouble.