52 Mitel incidents · luty 2022 — official updates, affected components, duration and resolution details.
Zdajemy sobie sprawę z problemu z jednym z naszych dostawców usług pochodnych, który ma wpływ na media w UE. Prowadzimy śledztwo i będziemy odpowiednio aktualizować.
Problem z dostawcą niższego szczebla jest teraz w pełni rozwiązany i możemy potwierdzić, że nie mamy już wpływu na żadne z naszych usług.
Automatyczne tłumaczenie oficjalnej aktualizacji incydentu.
Ten problem dotyczy każdego MSL host Daemon, który ma MBG jako część manifestu (samodzielny MBG, MiCollab, SMBC, serwer SVI). Wszelkie nowo utworzone połączenia na MSL 12.2 lub aktualizacje do 12.2 MSL otrzymają błąd "Nie można utworzyć raportu inwentarza". Naprawa jest testowana w naszej chmurze Dev. Zostanie to zaktualizowane, gdy fix zostanie przeniesiony do produkcji. Aktualizacja 8 / 24 - Mamy potwierdzone raporty, że problem ma miejsce na starszych wersjach MSL. Fix jest testowany i zostanie opublikowany jak najszybciej.
Naprawiono go jako część CloudLink Daemon w wersji 1.9.15. Ręcznie aktualizowane do tej wersji ze strony Daemon w menedżerze serwera lub czekanie na kolejne zaplanowane wydanie powinno naprawić problem.
Ten incydent został potwierdzony.
Automatyczne tłumaczenie oficjalnej aktualizacji incydentu.
We are currently experiencing an issue where the CloudLink Daemon service is down for MCX/MiCC. Our engineering team has been engaged and is actively investigating it.
Mitel Cloudlink Daemon has been rolled back to version 1.8.66+139. Please start Mitel Cloudlink Daemon service in the affected MiCC-B/MCX.
A fix has been implemented and we are monitoring the results.
We are continuing to monitor for any further issues.
Root cause: The update on March 23/24 wasn’t successful and left the Windows service “Mitel CloudLink Daemon” on MiCC-B systems in a stopped state.
We are aware of issues in the Tunnel service and are currently investigating.
The root cause has been identified, and our team is actively working on a resolution.
The service has been restored and we will continue to monitor.
We experienced a temporary capacity issue that affected service availability during periods of high demand. The issue has been resolved, and additional safeguards have been put in place to prevent recurrence.
We are aware of an elevated error rate on the Notification service in EU and are currently investigating. At this time it appears to be impacting Workflow and Microsoft Teams presence sync. As of 10:41 EST we are seeing the services recover. We are continuing our investigation to identify the root cause.
Service has been restored, we will continue to monitor
The issue was caused by an unexpected, rapid spike in traffic. This surge exceeded our service capacity, resulting in a temporary service disruption.
We are aware of an issue affecting Mitel One on Android (version 6.9.0) where the keypad may not open during an active call, preventing DTMF input. Our team is actively working on a fix and will share updates as they become available.
The root cause has been identified, and our team is actively working on a resolution.
The in‑call keypad issue on Android has been fixed. Users are advised to update to Mitel One v6.9.1, now available on the Google Play Store.
The issue was caused by a recent OS compatibility change affecting the call-screen display. It has been resolved in Mitel One v6.9.1 on Android, now available on the Google Play Store.
We are aware of an issue affecting CloudLink Gateway containers from starting correctly. We are are actively investigating and will update as the investigation progresses.
The root cause has been identified and we are actively working on fix.
A new build containing the fix has been created and we are working to make this available in the public cloud, once available we will provide an update.
CLGW version 2.4.18 now available from the download server, for any effected gateways please restart to trigger the upgrade process. If a restart is not initiated the update will occur inline with the CLGW update schedule.
This incident has been resolved. The issue was caused by an update to the timestamp validation. Changes where implemented and this was fixed in CloudLink Gateway 2.4.18.
Starting at approximately 3:00 AM EST our service provider had a major outage in the US East region where our services are hosted. This impacted operations for approximately 3 hours. As of right now service has been restored and we will continue to monitor services. For more details please see here: https://health.aws.amazon.com/health/status
There is still some degree of instability from our service provider which is resulting in some services intermittently being impacted. While the impact does not appear to last very long it continues to happen intermittently. We remain engaged with our provider and waiting for their resolution.
Our service provider has updated that the issue is fully resolved now and we can confirm we no longer show any impact to any of our services.
Currently there is an issue with the authentication in the iOS version of Mitel One version 6.8.1. Please DO NOT UPGRADE to this build, a new build will be released as soon as possible that will fix this issue. A further update to this will be posted once this new build is available.
We have identified the issue and are working on a fix.
The fix has been released in iOS version 6.8.2 of Mitel One.
The incorrect authentication URL was mistakenly used in version 6.8.1, this was corrected in version 6.8.2.
Starting on Saturday July 5 there was an issue with a firewall blocking traffic to the server used to deliver automatic updates to the CloudLink Gateway and CloudLink Daemon. This would not have impacted day-to-day operations and would only be seen if you were trying to apply updates or deploy a brand new system. We identified and disabled this firewall this morning at 9:00 UTC-0400 and observed all traffic was flowing correctly again.
This incident has been resolved.
We are aware of an issue with presence status displaying incorrectly. The issue is currently under investigation and we will give further updates when available.
A fix has been deployed with the CloudLink Gateway release 1.4.0.01-3826 or later.
This incident has been resolved. A change to the presence implementation inadvertently affected the Mitel One application and MiVO400 integration. This was fixed in the CloudLink Gateway release 1.4.0.01-3826.
We are aware of an issue with both CloudLink Gateway and CloudLink Daemon showing a pending update, but that update is not applied. We are currently working on a fix and will provide a further update when this is available. The pending update does not contain fixes for any customer reported issues, and your CloudLink connection will continue to function as normal.
This incident has been resolved.
We are currently investigating reports of Mitel One users not being able to log into the application. The underline issue appears to be related to the MiVO400 and is currently under investigation.
A fix has been deployed and we are monitoring. The fix may take some time to propagate but you can force a sync by logging into the Web Admin for your MiVO400. When successful you will see a full green dot at the top of the page.
This incident has been resolved.
We have identified and fixed an issue with the Twilio Social Media flow integration on MiContact Center Business where messages were very slow to be delivered. Please ensure that you do not configure the Fallback URL or Status URL in the Twilio console when configuring the Social Media flow integration. We will continue to monitor this issue and provide any updates as needed.
This incident has been resolved. Please ensure that you do not configure the Fallback URL or Status URL in the Twilio console when configuring the Social Media flow integration.
We are aware of an issue when customers send messages through the Twilio Integration configured via Mitel Administration Portal to MiContact Center Business. The messages will get routed to the agent however the agent is unable to send messages back to the customer. The agent inside of WebIgnite will see a message "Unable to send previous message." We are currently working on a fix and will provide an update once it has been deployed.
A fix has been deployed and the Twilio Integration should be working correctly again. We will continue to monitor for a couple days before closing this issue.
This issue has been resolved. A deployment on September 12th inadvertently impacted the Twilio integration which was subsequently fixed with a deployment on September 13th.
We are investigating outbound call issues from Mitel One Mobile clients with carrier minutes enabled. We will provide an update when we have more information.
We have identified the issue and are working on a fix. We will provide another update once the fix has been deployed.
A fix has been deployed, outbound calls should now be working properly again. We will continue to monitor and update.
This issue has been resolved. This affected a small number of customers, it was fixed with a deployment on September 17th.
We are currently investigating elevated error rates on the Presence service in EU region. At this time it does not appear to be affecting service but we will continue to investigate and update accordingly.
The Presence service in the EU region has returned to normal operation. We will continue to monitor over the next few days.
This incident has been resolved. The incident appears to be as a result of unexpected issues within the network. We are no longer seeing elevated error rates and service has been healthy for the past 5 days.
We have identified an issue where presence status from Microsoft Teams is not in sync with MiCollab. Users may notice their presence in MiCollab does not match Microsoft Teams. We are currently working on a fix and will provide an update once the fix has been deployed.
The presence status from Microsoft Teams is now correctly syncing with MiCollab again. This was caused by an issue with Microsoft Teams as acknowledged by Microsoft. It lasted from Aug 8, 2024, 8:00 p.m. EDT until Aug 15, 2024, 7:00 p.m. EDT. Please see Issue ID TM859169 in your Microsoft 365 Admin Center Service Health dashboard for more information.
We are receiving multiple reports of CloudLink Gateways losing connection in the AP region. Customers may receive an error message with "CLGW_ERROR_LITTLE_MOLE_FAILURE". We are currently investigating the cause.
We have identified the issue and are working on a fix.
A fix has been implemented and we are monitoring the results.
This incident has been resolved. An incorrect retention policy resulted in the container image being removed which took the service offline.
We are currently investigating reports of Mitel One Clients not connecting
We are continuing to investigate this issue.
We have identified the issue is affecting the Media services in AP and are currently working on a fix. We will update again when the fix has been deployed.
A fix has been implemented and we are monitoring the results.
This incident has been resolved. A certificate renewal batch process failed due to the depreciation of a compute environment by our cloud provider. The compute environment has been replaced and certificates are renewing correctly again.