52 Channable incidents · Νοέμβριος 2023 — official updates, affected components, duration and resolution details.
Αυτή τη στιγμή ερευνούμε αυτό το ζήτημα.
Το θέμα έχει εντοπιστεί και υλοποιείται μια λύση.
Εφαρμόστηκε μια ρύθμιση και παρακολουθούμε τα αποτελέσματα.
Το περιστατικό λύθηκε.
Αυτόματη μετάφραση από την επίσημη ενημέρωση του συμβάντος.
Αυτή τη στιγμή βιώνουμε ένα αυξημένο ποσοστό λάθους για το προϊόν και προσφέρουμε υποβολές μέσω της ολοκλήρωσης Kaufland μας. Ερευνούμε ενεργά το ζήτημα και βρισκόμαστε σε άμεση επαφή με την ομάδα Kaufland για να βρούμε μια λύση. Σημείωση: Η επεξεργασία παραγγελίας παραμένει πλήρως ανεπηρέαστη. Θα παρέχουμε άλλη ενημέρωση μόλις έχουμε περισσότερες πληροφορίες.
Εντοπίσαμε την αιτία των σφαλμάτων και κάναμε αλλαγές για να μετριάσουμε το ζήτημα. Τα ποσοστά σφάλματος είναι ακόμα αυξημένα ενώ επικυρώνουμε την αποτελεσματικότητα αυτών των αλλαγών με το Kaufland. Η απόδοση θα παραμείνει υποβαθμισμένη εν τω μεταξύ. Συνεχίζουμε να παρακολουθούμε στενά την κατάσταση.
Εντοπίσαμε την αιτία των σφαλμάτων και η κατάσταση έχει βελτιωθεί σημαντικά. Ο συνολικός αντίκτυπος είναι επί του παρόντος περιορισμένος· ωστόσο, επειδή εξακολουθούμε να βλέπουμε περιστασιακά λάθη, οι προσπάθειές μας για μετριασμό θα παραμείνουν σε ισχύ ενώ συνεχίζουμε να παρακολουθούμε την κατάσταση. Εάν εξακολουθείτε να αντιμετωπίζετε προβλήματα, παρακαλούμε επικοινωνήστε με την ομάδα υποστήριξης μας ώστε να μπορέσουμε να σας βοηθήσουμε περαιτέρω.
Αυτόματη μετάφραση από την επίσημη ενημέρωση του συμβάντος.
Since 12:00 PM UTC today we have been experiencing a disruption with our Mirakl order connections. This is causing a delay in fetching orders from marketplaces into Channable. We are actively monitorring the issue and will provide an update as soon as we have more information.
All Mirakl orders now being retrieved and processed.
We are experiencing issues with Shopify imports and order processing. Shopify has acknowledged an incident on their end: https://www.shopifystatus.com/ We are investigating what the exact impact of this incident is on our integration and will update once we know more.
The issues we are seeing are related to the Shopify incident linked below. Shopify imports will not succeed and order synchronization can not succeed while this is ongoing. Notice: You may receive automated emails about failed orders or import updates from Channable features that rely on Shopify. No action is required on your side at this time. These notifications are a side effect of the incident.
Shopify has identified the issue and rolled out a mitigation. We are seeing recovery of order synchronization and processing of import jobs. We will keep monitoring until full recovery of processes.
The issue has improved and Shopify order sync and stock updates are largely processing again. No action required: Any orders that failed during the incident have been or are being automatically re-sent by us.
We are investigating a potential issue with Shopify authentication. Channable projects affected by this issue will show a red 'failed' marker for Shopify imports, and will not receive new orders, returns or stock updates. This incident will be updated once we know more.
The issue has been identified and a potential fix is being worked on. The potential fix will require manual action from an authorized Shopify user to re-link Channable to Shopify. Instructions to perform this re-linking will be shared once they are ready.
We have applied a fix which has allowed stock updates to work again. This means that Channable is again able to notify marketplaces of stock changes when orders are placed on Shopify, but we are not yet able to send new orders or returns from other marketplaces to Shopify.
We have identified a fix for the issue affecting Shopify connections. This fix requires manual intervention by an authorized Shopify user, which means Channable cannot apply this fix for you. To restore Shopify connectivity, please 'Renew' the connection as described in this page: https://helpcenter.channable.com/get-started/getting-started/connections-in-channable/how-to-renew-a-connection
Contrary to our previous update we have identified that stock updates are still affected by this issue. The previously-shared fix for order connections will also fix stock updates and imports from Shopify. To restore Shopify connectivity, please 'Renew' the connection as described in this page: https://helpcenter.channable.com/get-started/getting-started/connections-in-channable/how-to-renew-a-connection After renewing the connection any orders that failed while the connection was not working, will need to be re-sent by following the steps in this helpcenter article: https://helpcenter.channable.com/list-advertise/orders/troubleshooting-orders/how-to-fix-resend-a-failed-order#resend-all-orders
As part of Shopify's own security measures our Shopify app has been delisted. We are currently reaching out to Shopify to re-list our app as soon as possible. Until that time renewing connections as described in the previous updates is not possible. We will update when it is again possible to renew connection(s) and resend all failed orders.
We have identified a fix for renewing Shopify connections while our app remains delisted. We are currently working on implementing this and getting it in our tool. We expect to provide an update on this within 30 minutes.
Shopify connections can be renewed again in Channable. Companies with broken Shopify connections will see a notification on log in. To restore Shopify connectivity, please 'Renew' the connection as described in this page: https://helpcenter.channable.com/get-started/getting-started/connections-in-channable/how-to-renew-a-connection. After pressing 'Renew' and logging in to Shopify you will need to press 'Open app'. This triggers the reauthentication process between Channable and your Shopify store.
Additional instructions for renewing Shopify connections and manual fallback We have provided further details and a manual workaround for users experiencing difficulties renewing their Shopify connection. You can find these full instructions at: https://helpcenter.channable.com/get-started/getting-started/connections-in-channable/how-to-renew-a-connection. - Do NOT uninstall the current Channable app in Shopify during this recovery process. - If the 'Renew' button does not work, you can manually trigger the reauthentication flow by logging into admin.shopify.com, navigating to Installed apps, opening the Channable app, and clicking Open app. - Orders that failed while the connection was down will not automatically retry. Re-send them manually once your connection is active. Note that re-sent orders may take some time to appear in Shopify. If you are still unable to renew your connection after trying the manual fallback, please contact our support team at [email protected].
The Channable App is still unlisted in Shopify We are aware that the Channable app is no longer listed in Shopify’s app marketplace. We are working with Shopify to get the Channable app re-instated. In the meantime, DO NOT uninstall the Channable app in Shopify. Removing the app will prevent you from renewing your Shopify connection. Follow the renewal instructions here: https://helpcenter.channable.com/get-started/getting-started/connections-in-channable/how-to-renew-a-connection
Fix implemented for Shopify metafields error message We have released a fix for users encountering the error message "Channable cannot access your Shopify metafields" for their Shopify imports. You can resolve this either by switching to Shopify Markets (if you haven't already done so), OR by applying a workaround in your Channable import settings. We have published the instructions for both methods in our changelog: https://helpcenter.channable.com/changelog#error-message-channable-cannot-access-your-shopify-metafields
Re-installations and new Shopify connections are now possible While we continue to work with Shopify to get the Channable app re-listed in Shopify's App store, we have identified temporary workarounds as follows: If you uninstalled the Channable app: You can now reinstall it directly from your Shopify admin backend without needing the app marketplace. Follow the instructions here: https://helpcenter.channable.com/changelog#i-uninstalled-the-channable-app-from-shopify If you need to install Channable for the first time: We have identified a direct installation URL that allows you to bypass the marketplace and establish a new connection. Follow the instructions here: https://helpcenter.channable.com/import-data/import-how-to/import-your-product-data/import-via-a-webshop/shopify#install-the-channable-app-in-shopify In addition, all issues affecting order synchronization have been resolved as connection renewal allows processing of all your orders
We're closing this status page now, as the majority of technical issues have been resolved. We're still working with Shopify to get the Channable app up and running again on the Shopify App store. All workarounds, such as the temporary fix for metafields, still apply. You can stay up-to-date by following the changelog.
We have identified an issue with background processing that started around 2:15 EST. We are investigating the cause.
We are continuing to investigate this issue.
We have identified the cause and are working on mitigating the issue. Until the this mitigation is in place background jobs will continue processing with delays.
We have implemented a mitigation and are working through our backlog of background jobs. Imports, Channel exports and other jobs are being processed.
This issue has been resolved. We are working through our backlog of background jobs at a normal pace again.
We have identified an issue which is causing some import jobs to fail to run. Affected jobs will show a red 'Failed' marker in the Channable tool. We have identified a likely cause of the issue and are working on a fix. Any affected import or export jobs will be rerun automatically once we have determined the issue is fixed.
A fix has been implemented and we are monitoring the results.
The issue that was causing the import jobs has been determined to be fixed. We have been made aware of an additional issue which is preventing projects using the image editor from working correctly. Affected projects will also show a red 'Failed' marker in the Channable tool. We have identified the cause of this issue as well and are working on deploying a fix. Any affected export jobs will be rerun automatically once we have determined the issue is fixed.
The issue affecting the exports using the image editor has been confirmed to be fixed. All affected import and export jobs have been resubmitted and will be retried by our system shortly.
This incident has been resolved.
Issues loading and processing items causing the tool to sometimes not show pages correctly. We are investigating the scope of the problem at the moment. We can see that exports, rules and creatives are mainly affected.
Restarted affected services and we will monitor. Missed orders or offer updates will be picked up, although with a slight delay. Any delayed exports or imports will continue to run again.
The incident has been resolved. No systems are currently affected anymore and processes are being caught up on. Do note that there still might be a small delay while we work through the backlog of tasks that got delayed.
Since January 20th, we have observed a reduction in performance for our Shopify order connections, occasionally resulting in the error message 'Unable to claim inventory' during order creation. Shopify is aware of this issue and is actively working on a solution. We have implemented measures to addressing this issue by increasing retries on order creation, to mitigate the impact. For any failed orders, the current workaround is to manually resend the Shopify order within our platform. We are monitoring the situation closely and will provide updates on this page as the situation evolves. Please note: This incident is isolated and does not affect any of our other integrations.
This incident has been resolved. After an extended period of monitoring, we have confirmed that Shopify orders are once again being created successfully and error rates have returned to normal. All systems are fully operational. Thank you for your patience.
Our phone provider is experiencing issues with their authentication system. This means our support department is not reachable by telephone. We are still reachable through all other means
We are continuing to work on a fix for this issue.
This incident has been resolved.
We are currently investigating a report about Returns not working as expected for different Mirakl platforms (i.e. Carrefour, Fonq and Kruidvat among others). We are determining the impact as we speak. Exports to these channels or regular Order handling remain unaffected.
We are continuing to investigate this issue.
We have determined what caused the problem and changed our internal logic for handling Returns for the different Mirakl platforms. The system is now quicker and more robust. Handling of returns will continue and works as expected. Other features were fully unaffected.
We have identified that we are affected by an ongoing outage in the global Cloudflare network. While this incident is ongoing images using background removal or the image editor may not work and fail to load with a 500 server error. This means that platforms configured to load feeds with images processed through Channable will not be able to load them. Exported feeds themselves are unaffected at this time and are being updated normally. Cloudflare is providing updates for this outage at the following URL: https://www.cloudflarestatus.com/incidents/8gmgl950y3h7
Cloudflare has reported that a fix has been implemented for the outage, and we can confirm that the performance of the affected infrastructure has significantly improved. We are still monitoring the situation to conclude whether the effect on Channable's infrastructure has fully been resolved.
This incident has been resolved.
We are currently investigating a problem with our rule processing engine which leads to degraded performance of item processing in Channable. This means that any page where items are shown in Channable will not work as expected and might show signs of slowness.
After some further complications, Channable is reported to be unreachable for certain users. We are working to add processing capacity in order to restore operations.
The issue was identified and our expected solution is having a positive effect on the recovery of the system. We are actively monitoring while the system recovers.
After monitoring we concluded that our systems are operational again and back to their original state of working.
We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently investigating this issue.
We are seeing an high backlog of failed imports that originate from the 'docs.google.com' domain. Because of this, other systems got backed up too. We are actively working on unclogging the backlog and getting systems operational again. The systems surrounding orders are unaffected.
While we work to resolve the blockade, we were forced to pause our jobs system. So no importers or exporters are running at the moment, everything remains queued for now.
The jobs system is unpaused. All projects not in scope of the problem are unblocked and are running again. All projects relying on a Google Spreadsheet importer are still being blocked.
Google is rolling out a fix, projects relying on the Google Spreadsheet importer are no longer blocked. We will continue to monitor the status.
After monitoring for a while we concluded that the problems have been resolved and systems are back to normal operation.
The image downloader is experiencing heavy load. We have applied performance improvements but it will take more time for us to fully catch up with the backlog. In the meantime, this might temporarily cause a `NOT DOWNLOADED YET` error to be shown for new images that we have not downloaded before.
We are continuing to monitor for any further issues.
This incident has been resolved.
We are seeing an increase in failed exports because part of the creatives suite is not working as expected. We are actively working to resolve this problem, however for certain exports that use the Image Editor it could result in failures. The hosted feeds are unaffected, but won't be updated when this problem happens. Other parts of the tool, including the downloading of images, are not affected.
For a small number of projects, a small number of product image files have been causing problems. A temporary fix has been implemented, causing us to mark images that are affected by the problem to appear as `NOT AVAILABLE`. The items causing problems can be omitted by editing your Image Generation Safety settings in the feed, or you can choose to wait for the problem to be fully resolved.
This incident has been resolved.
We are seeing delays in fetching images for our creatives suite. New images may be delayed by several hours. Images that have already been downloaded into our systems will function as normal. The problem has been identified and we are working on a solution.
A fix has been deployed and we are making progress towards our backlog. We expect to have caught up in a few more hours.
Our backlog has been cleared and everything is operational.
We have identified an issue with Beslist stock updates. We have paused them for now and new updates will not be visible in the app. We will unpause them as soon as things are running smoothly again.
Beslist stock updates are being processed again, we will continue to monitor the situation.
This incident has been resolved.
Cdiscount has reported a widespread outage on their systems: https://developer.octopia-io.net/seller-monitoring/ Channable is currently unable to sync offer listings or process orders for the Cdiscount marketplace as a result of this outage. Orders and offer synchronization will resume once the outage is resolved on Cdiscount's side. Real-time stock updates received during the outage will not be resubmitted after the outage is resolved. This incident does not affect any other marketplace.
Cdiscount has partially restored their services at June 18th 20:00 UTC: https://developer.octopia-io.net/seller-monitoring/ Channable's API exporters are operational again, but order connections and real-time stock updates are still affected by the outage. We will continue to monitor the restoration of operations.
Cdiscount reported the problem to be resolved on their end as of 14:00 UTC. Channable's services are all operational again for Cdiscount.