50 recorded Ibb incidents since July 2020, with official updates, affected components, duration and resolution information.
The conversation history no longer appears in the livefeed panel. And an error banner appears when opening it. We are investigating. Thank you for your understanding.
The conversation history issue is now resolved. Thanks again for your patience.
From 11:05am CEST to 2:51pm CEST on the 22nd of August, you may have noticed that no chat notification was displayed on your website. This was caused by a deployement made by our technical team, which introduced a regression. Not all the clients were impacted. The issue was addressed as soon as we noticed it. We are sorry for any incovenience this may have caused.
**Incident:** On September 2nd, between 11:01 CEST and 13:04 CEST, we experienced an incident impacting the service in charge of the iAdvize engagement \(handling targeting\). During this timeframe, the display of notifications on our customers' websites and on mobile applications fluctuated between functioning randomly and not being displayed at all. As a result, starting a conversation from Chat / Call / Video / mobile application channels was degraded \(86 min\) or even completely cut off \(37 min\). Social channels were not impacted. This unavailability of our engagement service occurred because: * After a restart, our mirroring service moved to the same server instance as our engagement service * Due to an unexpected resource usage spike on the mirroring service, the engagement service was left with insufficient resources to scale and run properly **Resolution** To solve this issue, we manually isolated our mirroring service to different server instance, ensuring the engagement service had enough resources to run properly again. **Actions for the future** * \(Done\) Isolate our mirroring service away from other critical services * \(Done\) Analyze the causes of the resource increase on our mirroring service, and implement optimizations to reduce its resource usage * \(Done\) Improve alerting alerting in case of network resource issue on server instances
During a few minutes from 3:38pm to 3:48pm (CEST), you may have experienced some issues on the conversation panel: - Ongoing messages disappear - Ongoing messages do not send - Conversations cannot be closed Our technical team has identified the issue and fixed it. An actualization of the conversation panel will probably be necessary to restore it. Thanks for your understanding.
The conversation panel is currently not available. We are working on the problem to get it resolved as soon as possible.
The cause of this issue has been identfied, and all is now back to normal. An intervention has been performed and the conversation panel is now accessible, visitors can initiate conversations again. We will keep monitoring the situation for the coming hours.
We are experiencing long update delay on our statistics service. As a consequence your statistics data may not be up-to-date, our technical team is currently working on it
Following a correction by our technical team, all reports are now properly updated. Thank you.
We are currently experiencing service disruptions, so you may notice issues on several services: - It is impossible to start new conversations - Errors on the conversations panel on opened conversations Our technical team is currently working on this issue.
Following technical interventions we are noticing great improvements. Services have been restarted gradually, while monitoring the results. You should notice improvements on your side: New incoming conversations can arrive on the conversation panel We are monitoring the situation, we will communicate again when we notice the situation is fully back to normal.
The incident is over since 10:50am CET. The conversation processing flow is now operational. Thank you for your patience.
This incident has been resolved.
Since Sunday Feb 11th we are experiencing issues on our statistics service. As a consequence data is currently not up to date. Our technical team has performed an intervention this morning, data is now gradually updating again. However due to the large volume of data to process, it will take several hours before it is 100% up to date. We will publish another message when all is back to normal, thank you for your patience.
We can now confirm that the statistics are up-to-date. Thank you.
Since yesterday you may experience errors when trying to access your statistics. Our technical team is currently working on solving this issue.
The problem is now solved, you can access your statistics. Thank you for your patience.
We are currently experiencing a slowdown on our statistics service. As a consequence the reports are not up to date. Our technical team is currently working on solving this issue.
Following the intervention of our technical team, the delay in the statistics update has been resolved.
This incident has been resolved entirely.
We are currently experiencing a slowdown on our statistics service. It is impacting the update of all the reports, our technical team is currently working on solving this issue.
Following the intervention of our technical team, the delay in the reports update has been resolved since 9:50 (CEST). Statistics are now up to date, thank you for your patience. We are continuing to monitor the situation.
This incident has been resolved. Thank you for your patience.
An incident is currently ongoing on personal canned answers. As a result, you may notice that some personal canned answers normally available are currently missing. We are currently working on the restoration of the service. Thanks for your understanding.
Personal canned answers have been restored. You can now use them normally. Thanks again for your patience.
This incident has been resolved.
We're seeing a high number of errors in the visitor engagement service (triggering engagement rules and displaying notifications). We are working on resolving the problem.
We are continuing our actions and seeing improvements. Engagement notifications are appearing again, but we have not yet returned to pre-incident volumes.
We confirm the return to normal and monitor platform activity.
After this monitoring period, we confirm that the situation has returned to normal. Thank you for your patience and understanding.
An incident is currently ongoing on our statistics service. You won't be seeing updated statistics for now. We are currently working on the restoration of the service.
A fix has been implemented and we are monitoring the results.
Stats are back now. Please note that none of them are missed. We are continuing to monitor for any further issues.
You may see stats disappear again. We are working on the issue. Please be sure that: - Conversations will remain unaffected as production will continue to run smoothly. - The only service temporarily disrupted will be access to statistics. - There will be no risk of data loss; once our next fix is implemented, we will recover all data.
An initiative has just been launched to stabilize the statistics problem. We will continue to monitor for any further problems.
This incident has been resolved.
We are currently experiencing a significant delay in updating our statistics. As a result, you might observe that your conversation and sales data is not updating. Please rest assured that no data has been lost, there is just a long delay in the information update.
This incident has been resolved.
We're seeing instability when it comes to connecting to the desk. The problem is probably random and does not affect all users. We are currently investigating.
We have identified the source of the problem and a patch is currently being deployed.
A fix has just been implemented and we confirm that the problem has been resolved. We invite affected users to refresh their browser to reload the desk correctly. We are monitoring activity.
This incident has been resolved.
We are currently experiencing a delay in the indexing of statistics, which has an impact on the recording and updating of closed conversations. Our technical team is currently investigating the problem. Thank you for your patience.
Our technical team is still mobilized to solve this statistics issue, impacting the update of closed conversations in the reports.
Our technical team has identified the cause of the problem and has performed an intervention. All reports are now updated and no delay is noted on new statistics updates. We will keep monitoring the situation for the coming hours.
The processed conversation statistics are now well up to date and the problem has been solved. However, we have identified occasional slowdowns in the updating of transaction data over the past few days. We are investigating the cause of these slowdowns. No data is lost, from time to time the information takes longer to update. Thank you in advance for your patience.
We have taken actions to solve the latency problem and see a clear improvement since April 29th. We will monitoring the situation during the coming days and close the incident if everything is ok. Thank you again for your patience.
Today we have been experiencing a delay in the sales statistics update. Our technical team is working on it.
After a technical intervention, data is now gradually being updated. Due to the large amount of data, it may take some time before the reports are up to date.
Data update is still slowly catching up. It will still take several hours before reports are up to date.
Statistics information is up to date since 2am (UTC+2), all is back to normal. We will keep monitoring the evolution of the situation.
This incident has been resolved.
The access to the platform is currently down, we are currently looking into the issue.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
The service to access to the platform is currently down. Our technical team is mobilized.
The issue has been identified. Some actions are still ongoing. The return to normal is gradual.
The access to the platform is operational again. Our teams monitor the platform.
All our services are now operating normally. We appreciate your patience as we worked through this and apologize for the inconvenience.
We are currently investigating a random white page problem on the desktop application. Thank you for your patience
We continue our investigations and invite you to refresh your browser or switch to private browsing to connect if necessary.
We see a return to normal and monitor the activity of the platform.
We confirm the resolution of the problem. You may need to clear your browser's cache (or use a private browser) to access the login form. If you have any problems, please do not hesitate to contact our support team. Thank you again for your patience.
We are currently experiencing slowdowns on the platform with difficulties in displaying the chat console. Our technical team is mobilized.
The slowdowns concern the whole platform. We are restarting some services and increasing the capacity of the servers.
We have noticed an improvement in the last few minutes and are monitoring the activity of the platform.
After a monitoring period, we close this incident. Thank you for your patience and understanding.
