Sirv Status

Control panel at my.sirv.com went down

Control panel at my.sirv.com went down

Control panel at my.sirv.com went down

The issue has been resolved. One of the servers serving Germany was overloaded and responding slower than usual. The server is now healthy again. We are reviewing what actions to take to reduce the chance of this happening again.

Response time for some requests in Germany are slower than normal and a small number are timing out. The issue is being investigated.

The issue has been resolved. The issue was caused by packet loss affecting our new Los Angeles datacenter. The datacenter traced the issue to their upstream provider, Arelion, which was experiencing network instability. The issue was resolved when Arelion was temporarily disabled as an upstream provider and traffic was rerouted through other providers. We require that our datacenters identify and reroute traffic automatically and quickly in such scenarios. However, the resolution of this issue was not fast enough and given that it occurred soon after adopting this new datacenter, we have decided to cease using it. All traffic is now being routed from our original Los Angeles datacenter (as of 7 April 2025).

Some requests from California are slow and a small number (under 2%) are failing. This is being investigated and likely related to new CDN servers that were deployed today in Los Angeles.

The issue has been resolved. The issue was caused by two servers in one cluster failing simultaneously. This impacted the uploading of new files to about 20% of Sirv accounts. Uploads are designed to continue as normal when one server is down but when two servers are down, some uploads can fail, which is what happened. They failed due to an exporter service that stopped running on one server, then the other server. Errors prevented the services from restarting. Then a second issue caused the resolution to take much longer than expected because the servers couldn't be rebooted. This was due to an outdated BIOS. Once the BIOS had been updated, the servers were rebooted and the cause of the underlying issue was resolved. To prevent this from happening again, we have implemented a new BIOS management process with our datacenter. We are also shortening our hardware refresh cycle, for more frequent hardware upgrades. To reduce the chance of the exporter service failure from recurring, we have disabled a process and are monitoring the server metrics.

This issue remains in progress. The cause of the issue has been identified and we are working to resolve it as soon as possible.

Some file uploads are not completing due to a server cluster issue. The cause is being investigated. The issue is affecting a small number of accounts. If your file upload fails, please wait until this issue has been resolved.

Maintenance completed

Scheduled maintenance on part of Sirv's network will take place during a 2-hour window between 0330 and 0530 UTC on 6 December 2024. The actual duration of impact is expected to be less than 60 minutes. The purpose is to upgrade infrastructure at Sirv's primary datacenter. During the scheduled period: 1. Some accounts won't be able to upload files. 2. Some accounts may not see some files in my.sirv.com. 3. Some accounts won't be able to generate new images. 4. Some API requests may fail. All existing files will be delivered as normal from the CDN. We recommend not attempting to upload files during the period or you may receive an error.

The maintenance has been completed. All core services continued operating as normal during this period. In case you were not seeing files in your my.sirv.com control panel, you will see them now.

Scheduled maintenance on part of Sirv's network will take place during a 2-hour window between 0330 and 0530 UTC on 5 December 2024. The actual duration of impact is expected to be less than 60 minutes. The maintenance is to datacentre infrastructure, not to Sirv servers. All core Sirv services are expected to continue as normal - file serving, file processing, file uploads, CDN delivery. During the scheduled period, some customers may not see their files listed in my.sirv.com but they will exist and be processed and served as normal.

The maintenance has been completed.

Scheduled maintenance on part of Sirv's network will take place during a 2-hour window between 0330 and 0530 UTC on 4 December 2024. The actual duration of impact is expected to be minimal. The purpose is to upgrade infrastructure at Sirv's primary datacenter. Primary cluster and Riak down: no UI, no uploads, no FTP, REST API, etc, +logs down. Some accounts will switch to failover. During the scheduled period: 1. The control panel at my.sirv.com may be unavailable. 2. Some API requests may fail. 3. File uploads may fail. 4. Some accounts won't be able to generate new images. All files served from the CDN will be delivered as normal. We recommend not uploading files during the period or you may receive an error.

Sirv's primary image processing datacentre in Germany experienced sporadic degraded performance due to backbone maintenance. Approximately 1% of requests for unprocessed images were slow to load, taking a few seconds to be returned. To minimise the number of slow requests, a higher than usual proportion of requests were routed to Sirv's backup image processing datacentre. Requests fulfilled by the CDN were unaffected, typically returning requests at nominal speeds of 20-100ms. Classified as severity level 4: Minimal impact.