30 Frontify incidents · luty 2019 — official updates, affected components, duration and resolution details.
Obecnie badamy kwestie wydajności, które mogą prowadzić do wolniejszego niż zwykle ładowania aktywów lub ładowania strony biblioteki. Nasz zespół pracuje nad jak najszybszym przywróceniem normalnej wydajności.
Wprowadzono rozwiązanie i monitorujemy wyniki.
Ten incydent został rozwiązany.
Automatyczne tłumaczenie oficjalnej aktualizacji incydentu.
We are currently experiencing degraded performance on the Self-Service environment. Our team is actively investigating the root cause and working to restore stability as quickly as possible. At this stage, the impact is limited to the Self-Service environment; Enterprise customers remain operational.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
We are currently experiencing degraded performance across the application and are investigating the issue.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
Due to an Amazon Web Services outage, we are temporarily routing the delivery and processing of assets in the Americas via Europe. If you have any concerns, please contact [email protected]
After AWS reported that all issues had been resolved, we were able to migrate the delivery and processing of assets back to the US for US hosted customers this morning.
Our SelfService application is still affected by the ongoing AWS issue in the US region, preventing us from launching new servers. This may cause slower processing or temporary outages, mainly in image processing and search. Newly uploaded assets may not appear. AWS is working to resolve the issue.
The service is running and catching up with the queue. The search should start showing new content.
All uploaded assets are processed and should show and search results are up-to-date as well.
We are continuing to monitor for any further issues.
This incident has been resolved.
We recently experienced a partial outage caused by a global Amazon Web Services (AWS) issue. This led to temporary disruption of file processing and degraded performance across some other functionalities. AWS has since mitigated the issue, and all Frontify services are now fully operational. If you continue to experience any issues, please contact our support team at [email protected].
We are continuously working on enhancing our platform. Those ongoing improvements are implemented in small, iterative steps, and rigorously tested. During a database migration, a deadlock situation occurred for yet unknown reasons. This affected all our EU customers and caused a downtime of approximately 10 minutes. At no time did this compromise the security of our platform. Upon detecting the issue, we immediately took corrective actions and fixed the issue promptly. Once the immediate cause was identified and the deadlock resolved, all services were operational again. We are currently conducting a root cause analysis to prevent future occurrences and to extract valuable insights about what happened. Thank you for your understanding as we continue to refine our processes and enhance the reliability of our product.
We encountered higher load on our database cluster leading to slower response times.
Currently freshly uploaded assets aren't visible within the libraries.
We are currently working on a fix.
We are recreating the index for the overview of assets within the libraries. First customers should start to see assets they uploaded today.
We are still recreating the index while more and more libraries should be up to date.
We are working on the last big libraries
Still working on the last big libraries.
The issue has been addressed. We are monitoring the additional reindexing today to ensure no additional issues come up.
All remaining index jobs finished successfully. If you still experience issues with asset display, please contact Frontify support.
On Monday, on 12:40pm UTC a partial service outage occurred on all application clusters. Service restoration started at 13:05pm UTC with most of the accounts up and running in the following minutes. The service is fully restored. The issue was caused by database maintenance where legacy elements were renamed. We apologise for any inconvenience caused by this outage
This incident has been resolved.
A short 24-minute outage for non-Enterprise users occurred today around 13:21 pm UTC resulting in a 504 timeout error page. This was caused by the failure of one of the servers due to power issues at our hosting provider AWS. The application is running again, but there might be several residual issues that affect some functionalities. We are monitoring them as we expect an update on the issue from AWS. A further statement will provide more details once the hosting provider issue is completely addressed on their end.
The affected service has been restarted and all systems are back to normal.
On Monday morning, between 5:19 and 5:27 am UTC, one of our application clusters had a service interruption causing a short downtime. The following coincidence caused this: At midnight AWS took one of our application servers offline to mitigate a hypervisor issue. Shortly after, at 5:19 am UTC, our scheduled security update took the second application server out of service, causing the short outage. At 5:27 am UTC, the security updates finished successfully, and the server was retaken into service, ending the downtime. After investigating the issue, we put the first application server into service, so we are fully redundant again. To avoid this in the future, we enhanced monitoring and created automatic alerts notifying when there is only one application server in service. Additionally, any future scheduled security updates will first check for enough healthy resources before starting. We apologize for any inconvenience caused by this outage.
We can see slower response times for specific requests. These issues are related to the issues our infrastructure and services provider AWS is facing.
As AWS services are starting to resume the Frontify performance is increasing.
Frontify page speed is back to normal.
On September 29th a bug was reported that umlauts are not considered in group mapping, during the SSO login process. One of our developers fixed this bug, and, in order to verify the fix, he used some dummy credentials from his favourite show, SpongeBob. Unfortunately, the dummy credentials were shipped to production and affected the SSO login process. As a result, the wrong profile information was shown to already-registered users and the access to certain guidelines or projects was declined. Most importantly, no security breach happened and no external users had access to your account or your data. We rolled back the code and provided a fix shortly after that. We apologize for the circumstances. We learned our lesson from this and will analyze our deployment and peer programming process to avoid such mistakes in the future.
We are currently investigating a network issue within the data center
Our monitoring falsely reported an outage while there was none. We are sorry for the inconvenience this might have caused to you.
Our Provider Metanet has identified a network problem on their server clusters. They are working on a solution to fix the issue. Affected Datacenter: CH
The incident has been resolved. All systems are back online.
Partial Failure for some of our customers located in the EU Datacenter (Frankfurt) - we are investigating what happend and collecting all information. All customers are back online. Downtime was < 15min
We are investigating some issues with images not properly shown in preview.
We found some broken preview images in our CDN (cache) and are about to invalidate them.
Most preview images should look good, again while we are still invalidating some cache content.
All affected preview images on all edge locations of our CDN (cache) have been invalidated.
Due to the failure of one server users may have seen very long response times or even a 504 timeout error page.
The affected service has been restarted and all systems are back to normal. We'll be investigating why a single failure has led to this outage and take actions accordingly.
Due to a partition size change a server mount failed. For this reason, the application became unavailable in our Frankfurt Datacenter. Some servers needed a restart to fix the issue. The Infrastructure team is now analyzing why the fallback was not working properly.