Pianoio outage history

We are currently experiencing a production incident on Piano Analytics products. Please excuse us for any inconvenience caused.

The incident has been resolved and is no longer affecting Piano Analytics. We apologise for the inconvenience caused.

**Incident explanation:** Snowflake outage affecting multiple core Snowflake services and features. (AWS - Europe (Ireland)) Many of Snowflake's services were unavailable, which affected part of the data processing chain, certain types of exports, and the analytics API. Incident Report : https://status.snowflake.com/incidents/bfh045zclk5k **Actions performed:** Monitoring the situation while awaiting the fix from Snowflake, along with several actions on our end aimed at resolving issues caused by this incident (fixing data processing delays on our end, fixing delays in certain exports, etc.).

Between approximately 06:54 and 15:51 UTC on October 16, posts sent to LinkedIn through SocialFlow failed to publish, and collection of LinkedIn post statistics was also affected. The issue was caused by a change on LinkedIn's side and has been resolved. Publishing and statistics collection have worked normally since 15:51 UTC. Posts that failed during this window were not retried automatically and will need to be rescheduled. No reauthentication of LinkedIn pages is required.

We are currently investigating an issue affecting automated publishing in SocialFlow. Clients may see new content from their feeds not appearing in publishing queues, and automated posts not being published to social networks, including X. The issue has been occurring since approximately October 10. Manual posting continues to work. Our engineering team is actively working to identify the root cause. We will share an update as soon as we have more information.

This incident has been resolved. Automated publishing in SocialFlow is working normally again, and new content from feeds is appearing in publishing queues and being posted to connected social networks. We apologize for any inconvenience caused.

**Root cause:** An internal component responsible for retrieving content from client feeds stopped processing on October 10 and was not automatically restarted. This reduced the amount of content reaching the publishing system, and automated posts to connected social networks were delayed or not published. Posts scheduled directly by users were not affected. **Resolution:** The component was restarted on October 12 at 07:20 UTC, and publishing returned to normal. Content that had been queued was published shortly afterwards. **Next steps:** We are improving our monitoring and automated recovery so that this type of issue is detected and resolved faster. We apologize for the disruption.

Between 14:53 and 16:26 UTC on September 21, Piano experienced an issue affecting the publishing of posts to X. During this period, affected posts failed to send and, after a limited number of automatic retries, were reported back as failed rather than published, with error notifications sent to the affected accounts. Posts that failed during this window were not sent retroactively once service was restored and need to be published again. The cause was an internal configuration change made during planned monitoring work, which has been reverted. Publishing to X has been fully restored.

We are currently experiencing a production incident on Piano Analytics products. Please excuse us for any inconvenience caused.

An API ingress configuration issue has been detected. Rollback to previous version has been applied to solve the problem.

We are currently experiencing a production incident on Piano Analytics products. Please excuse us for any inconvenience caused.

Some problematic behavior relative to authorization was active on production when it should have been disabled from the start. Problematic network annotation have been desactivated for production environment.

Piano has identified an issue affecting access checks in Composer for users in the US and EU regions, which began at approximately 06:49 UTC. The underlying cause has been addressed and access checks have been fully restored. Our team will continue to monitor the situation closely and will provide a further update should any issues arise.

This incident has been resolved. Access checks in Composer were affected between 06:45 and 07:39 UTC in the US region, and between 07:10 and 07:34 UTC in the EU region. Alongside the access check errors, a proportion of requests to the affected services also returned errors during the same periods. All affected functionality has been fully restored and no further errors have been observed.

This incident has been resolved.

Between 23:12 UTC on September 8 and 10:03 UTC on September 9, Piano experienced an issue affecting automated publishing from content feeds. During this period, configured RSS feeds could not be fetched, and feed-based posts were not published to Facebook or X. Articles released while the issue was active may not have been posted automatically to the affected social accounts. The underlying issue has been addressed and feed collection and publishing have been fully restored. Our team continues to work on measures to prevent a recurrence.

We are currently investigating an issue affecting personalized content and audience segmentation for users on our US environment. Some users may see missing or outdated personalization data, including issues with audience segments and user profile data used for targeting. The issue began around 05:00 UTC. Our engineering team is actively working to identify the root cause. We will share an update as soon as we have more information.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We are currently investigating an issue affecting personalized content and audience segmentation for users on our US environment. Some users may see missing or outdated personalization data, including issues with audience segments and user profile data used for targeting. Our engineering team is actively working to identify the root cause. We will share an update as soon as we have more information.

The API availability is improving and is expected to fully recover soon. We will update the status page once it is restored.

This incident has been resolved.

We are currently investigating an issue affecting the delivery of emails and push notifications for ESP users. Our engineering team is actively working to identify the root cause. We will share an update as soon as we have more information.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

This incident has been resolved. Between 05:02 and 07:30 UTC, a network fault affecting a single server at our hosting provider's data centre disrupted the infrastructure responsible for email rendering, sending, feed imports, and scheduled tasks. Emails scheduled during this window were held and sent once service was restored, and content used in campaigns sent shortly after recovery may have reflected slightly older feed data than usual.

Piano identified an issue affecting access to Piano Insight for some users. The certificate causing the errors has been renewed, and the fix has been verified internally. We are monitoring to confirm the issue is fully resolved and will share a further update shortly.

We are continuing to monitor for any further issues.

This incident has been resolved.

We are experiencing an issue with report generation. Our team is aware and actively working to resolve it.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

Piano is experiencing an issue affecting login and registration for users in the EU region. The login window may fail to open, or may be unresponsive. We have identified the cause and our team is deploying a fix. We will share an update as soon as further information is available.

Piano has identified that this issue affects login and registration across all regions, not only the EU as initially reported. Users in all regions may experience the same symptoms — the login window failing to open, or becoming unresponsive. Our team continues to deploy the fix. We will share an update as soon as further information is available. Affected components: Registration + Login - US - id.piano.io Registration + Login - Australia - id-au.piano.io Registration + Login - Asia - id-ap.piano.io Registration + Login - EU - id.piano.io

We are continuing to work on a fix for this issue.

Piano can confirm this issue in fact only affects the US and EU regions, as well as Sandbox environment. AP and AU are not affected. We apologize for any confusion caused by the earlier update. Sandbox does not have a dedicated status page component since it's not part of production infrastructure, this update is provided for visibility. Our team continues to deploy the fix. We will share an update as soon as further information is available.

Piano has implemented a fix across all affected regions and environments, including Sandbox. Login and registration functionality has been restored and is operating normally. We will continue to monitor the situation and will provide a further update should any issues arise.

This incident has been resolved.

RCA for the incident can be found at [**this link**](https://rca.piano.io/wp-content/uploads/2026/07/2026-07-27-RCA-Identity-Management-login-issues.pdf).

Following recent scheduled database maintenance in the AP region, we have identified a data gap affecting the Subscription Log and Subscription Insights sections of the Publisher Dashboard. Some clients may notice that recent purchases are not yet reflected in the Subscription Log report, and some measurements in Subscription Insights may appear temporarily inconsistent. Our team is regenerating the affected data and expects this to be resolved within the next few days. We will share an update as soon as we have more information.

A fix has been implemented and we are monitoring the results.

We have restored the affected data for Subscription Insights, and measurements there should now be displaying correctly. The Subscription Log report has not yet been fully recovered, and some recent purchases may still be missing from this view. Our team continues to regenerate the remaining data. We will share an update as soon as we have more information.

The data gap affecting the Subscription Log has now been fully resolved, and all recent purchases in the AP region are correctly reflected in the report. Combined with the earlier fix to Subscription Insights, both affected areas of the Publisher Dashboard have been restored to normal. This incident is now resolved. Thank you for your patience while we completed the data regeneration.

We are currently investigating an issue affecting the delivery of paywalls and experiences for publishers in Asia and Australia. Some requests are failing to complete successfully. Our engineering team is actively working to identify the root cause and restore normal service. We will share an update as soon as we have more information.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

An internal infrastructure issue affecting a backend system dependency in our Tokyo data center caused authentication failures that cascaded into request failures for the Audience API. Because Composer traffic for Asia (AP) and Australia (AU) routes through this instance, both regions saw failures of Composer requests. This incident has been resolved. Our engineering team identified the cause and restored the affected system; service has returned to normal in all regions. We apologize for any inconvenience this may have caused.

We are observing data processing delays on the User profile backend caused by data improvement efforts for managing the cx prefix and the composer application ID. If you are using the cxense's external profile update API (https://api.cxense.com/profile/user/external/update) you will observe 1-2 hours of delays when reading the same profile using (https://api.cxense.com/profile/user/external/update). Our team is now reconciling the data back into a single, correct view, and this reconciliation is the source of the ongoing delay. No data has been lost, all events are being processed, currently with an approximate delay of 1–2 hours. We apologize for the inconvenience and will share an update as the backlog clear

Data processing delays continue on the User Profiles backend, with current delays of 1–2 hours. We have identified that a recent data change led to a significant increase in data volume for a subset of customers, which is the primary driver of the backlog. Our team is carefully increasing processing throughput while closely monitoring overall system stability to avoid introducing further issues. We are optimistic about stabilizing the situation soon and will share an update as soon as we have more information.

Data processing on the Piano Audience User Profiles backend has returned to near real-time and the backlog is cleared. We are setting this incident to Monitoring while our team completes residual cleanup from the earlier data change. We will share a further update once the incident is fully resolved.

The incident has been resolved after monitoring for more than 24 hours. We are seeing the processing of the data in real-time

We are observing data processing delays on the User profile backen caused by data improvement efforts for managing the cx prefix and the composer application ID. If you are using the cxense's external profile update API (https://api.cxense.com/profile/user/external/update) you will observe 1-2 hours of delays when reading the same profile using (https://api.cxense.com/profile/user/external/update). Our team is now reconciling the data back into a single, correct view, and this reconciliation is the source of the ongoing delay. No data has been lost, all events are being processed, currently with an approximate delay of 1–2 hours. We apologize for the inconvenience and will share an update as the backlog clear

We are continuing to work on a fix for this issue.

We are currently experiencing a minor service disruption affecting user logins to the Piano dashboards, caused by an ongoing outage on Cloudflare's side. Our team is actively monitoring the situation. We apologize for any inconvenience this may cause and appreciate your patience. Further updates will be provided as soon as more information becomes available. For reference: https://www.cloudflarestatus.com/incidents/v3yl7jqmqj51

The incident has been mitigated, along with our provider's resolution. We will continue to monitor the situation.

This incident has been resolved.

Since approximately 11:00 PM (UTC) on June 8, 2026, there have been intermittent delays loading files into Snowflake. The average load duration is currently about 10 minutes, compared to the usual under 2 minutes. The issue has been identified and a ticket has been opened with Snowflake.

The incident has been resolved and is no longer affecting Piano Analytics. We apologise for the inconvenience caused.

**Incident explanation:** The latency was caused by an issue on our cloud data platform provider's side. An internal infrastructure configuration change introduced resource contention that affected workloads with high ingestion concurrency, leading to elevated response times and intermittent timeouts when loading files. The change has been reverted by the provider, and ingestion performance has fully returned to its normal baseline. **Actions performed:** Opened and tracked an incident with our data platform provider and provided detailed monitoring evidence to support their investigation. Deployed mitigations on our streaming ingestion stack (increased service scaling and improved handling of stalled ingestion calls) to limit customer impact during the incident. Temporarily tuned ingestion timeout and retry settings to keep file loading as stable as possible while the root cause was being addressed. Continuously monitored ingestion latency and error rates until performance returned to normal. Confirmed full recovery, then reverted the temporary configuration to restore standard behaviour and avoid masking any future issues. **Actions to come:** Continued close monitoring of ingestion latency over the coming days to confirm lasting stability. Keep the temporary mitigation patterns configurable in the code so they can be re-applied quickly should a similar provider-side degradation occur.