Groove 장애 기록

We're investigating an issue with an upstream SMTP provider causing issue with mail delivery

This incident has been resolved.

We are currently investigating this issue.

This incident has been resolved.

The issue has been identified.

We are currently investigating elevated error rates and intermittent timeouts affecting parts of the platform. The issue appears related to instability within our Kubernetes infrastructure, which caused a large number of application instances to temporarily fail health checks and be removed from service rotation. We have stabilized the affected services and are continuing to monitor the cluster closely. At this time, core services are recovering and request success rates are improving, though some users may still experience intermittent latency or 504 errors while systems fully rebalance. We will continue to provide updates as we learn more.

This incident has been resolved.

Some operations including logins, ticket creation, contact updates, and email ingestion experienced intermittent failures. We identified and resolved an issue with an internal database table that had reached a storage capacity limit, which caused these operations to fail. The issue has been mitigated and services are returning to normal. Incoming emails are continuing to catch back up as we monitor the recovery.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results - backlog is currently clearing over the next 30-60 minutes. Expected delays for incoming mails, outgoing mails being sent, rule processing, realtime, and other features during this time.

This incident has been resolved.

This incident has been resolved.

We are currently investigating this issue.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We are currently investigating this issue.

The issue has been identified and a fix is being implemented.

We've identified and corrected a rogue process putting elevated load on the search cluster.

We are currently investigating this issue.

We are continuing to investigate this issue.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We are currently investigating this issue.

We are continuing to monitor for any further issues.

This incident has been resolved.

We have implemented a fix to address the issue. Our team is actively monitoring the results to ensure system stability and performance. We will continue to observe the situation closely and provide further updates as needed. Thank you for your patience.

We are continuing to monitor for any further issues.

This incident has been resolved.

A fix for delayed messages from Live Chat was rolled out - we're seeing intermittent issues since deploying the fix. The teams are working on resolving the issues still being seen with delays for Live Chat messages. Note: no other areas of Groove are affected.

We have successfully deployed a fix for the issue causing delays in some live chats. We want to reassure you that all delayed chats have successfully reached Groove and no messages were lost. Thank you for your patience. If you experience any further issues, please contact our support team.

We have identified an issue where some live chats are delayed. We are in the process of deploying a fix. This has no impact on any other channels or features within Groove. We will update this status page when a fix has been deployed.

We have successfully deployed a fix for the issue causing delays in some live chats. We want to reassure you that all delayed chats have successfully reached Groove and no messages were lost. Thank you for your patience. If you experience any further issues, please contact our support team.

An issue with one of Groove's database hosting providers caused a portion of Groove users to be unable to access Groove from certain devices for approximately 17 minutes. No data was lost during this time, and service has been completely restored for all Groove accounts.

One of our upstream providers experienced an issue that corrupted some attachments before the mails reached Groove. This issue has been resolved. Some attachments can still be retrieved from your mail provider.

Our slack app is currently being reviewed for new scopes according to Slack's app policy. During this time, creating new Slack connections may be restricted.

This incident has been resolved. Slack has re-approved our integration.

Meta (Facebook Messenger and Instagram) currently have an outage affecting API endpoints from connecting. Users may have disconnections from their Social Chats while the outage is happening. https://metastatus.com/

Users may see degraded performance in some areas in Groove.

Full service has resumed.

Meta is currently experiencing a major outage of the Instagram Messenger API. Unfortunately, no additional information has been provided by Meta regarding the cause or expected resolution time. While not all Instagram channels on Groove are affected, some are experiencing disruptions. Our team is actively monitoring the situation.

We are continuing to investigate this issue.

Meta has resolved the incident on their end. We are continuing to monitor any impact to affected Instagram Messenger users in Groove.