55 Globalsign incidents · 2023年12月 — official updates, affected components, duration and resolution details.
在今天早些时候的Atlas证书签发(HVCA API)负载平衡器维护之后,一些客户报告了处理HVCA API答复的问题. 现已确定,这是由于对HTTP的一个响应头的意外修改所致。 我们正在努力尽快恢复这一变化。 请注意,证书发放不受影响.
响应头已更新, 以与维护前返回的头条的确切格式相匹配 .
没有报告其他问题,因此现在认为这个问题已经解决了.
自动翻译自官方事件更新。
我们正在调查一个影响Atlas合格TSA服务的问题,它可能造成间歇性错误。 一旦获得更多信息,我们将发布更多信息.
我们发现了一个TSA服务器的问题 服务器已经从游泳池中移除了 所以不应该引起更多的问题 我们的一位工程师正在努力恢复它.
TSA服务器故障的问题已经解决,我们正在继续密切监测情况,以确保不再出错。 这一问题导致HTTP 503响应率激增,在所有TSA服务中。 这导致安保部也退回少量上游错误.
没有发现其他错误.
自动翻译自官方事件更新。
我们在阿特拉斯门户网站上遇到间歇性超时和出错。 这涉及到我们的一个SaaS供应商的一个上游问题。 我们正在监测局势,并将随时提供最新情况。 后端API(证书发放,DSS,TSA等)不受影响.
我们的提供者向我们通报,他们目前正在测试解决这个问题的办法,并且正在监测。 一旦证实决议成功,我们将予以更新.
供应商已经确定了一个固定点,正在将其推出,我们正在密切监测,并将在我们看到完全恢复稳定时予以确认.
供应商的固定方案正在按区域推出。 过去40分钟里,我们没有看到门户受到任何影响,但继续密切监测,直到完全清楚为止.
提供方报告说这个问题已经解决,我们没有看到监测工作有任何进一步的中断,因此正在考虑解决这一事件.
自动翻译自官方事件更新。
我们正在调查与Atlas API连接的可能中断。 我们一旦确认范围,将公布进一步的最新情况
我们确认,所有服务都已投入运作。 我们的外部监测没有发现任何停电,然而,在一些地理图中可能存在一些短暂的丢失.
自动翻译自官方事件更新。
We’re currently investigating reports of a service interruption with GCC Certificate Issuance. The root cause is currently unknown. We understand the issues this may be causing and are working hard to resolve the problem as soon as possible. Occurred at: Aug 13, 08:21 UTC ■Scope of impact - GCC User Portal - Certificate Issuance - SSL API Certificate Issuance - SSL (Access to Domain Validation URL) - ePKI API Certificate Issuance - EPKI (Access to certificate acquisition URL) - JCAN User Portal - JCAN Certificate Issuance - JCAN API Certificate Issuance - JCAN (Access to certificate acquisition URL) [Note] There is no impact on the use of issued certificates or revocation confirmation.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
One of our upstream ISPs experienced an issue in their network serving our London data centre at approximately 19:34 UTC today which caused a brief interruption to connectivity to the Atlas APIs (DSS, Certificate Issuance, TSA). Some customers may have noticed timeouts or connection errors for a period of 1-2 minutes. This issue has been resolved.
We are experiencing problems with the logging cluster that is used for usage reporting in Atlas - directly via the APIs, and in the portal for DSS users. Users may receive errors when checking the APIs or may receive responses with missing usage. The team are working to resolve the issue ASAP.
The core issue has been resolved however we are continuing to closely monitor the situation and ensure the usage reporting cluster remains stable.
This incident has been resolved.
We are aware of an issue starting at around 5:17 UTC on Saturday 7th June which caused timeouts and errors for connections to the Atlas HVCA and DSS APIs with total disruption lasting around 12 minutes. This appears to have been related to upstream connectivity problems, however the Infrastructure team are investigating and will be planning appropriate resilience improvements.
We are aware of some further issues relating to the display of usage in the Atlas portal, affecting a small number of customers. The development team are actively working on a fix for affected customers. Certificate issuance and use of the APIs themselves is not affected.
Our development team are currently testing a fix for the issue. We will provide further updates as this is confirmed ready for production release.
The team have successfully deployed the fix and we are closely monitoring to ensure that the issue is resolved for all affected customers.
Following an extended period of close monitoring, the team have confirmed that the issue is resolved.
We just experienced a high spike in errors on the Atlas APIs (HVCA, DSS, TSA). The issue lasted for a few minutes between roughly 12:25 and 12:35 UTC. The issue is no longer occurring but the Infrastructure team are investigating the root cause.
The Infrastructure team have identified the root cause and are monitoring the situation.
The issue has been confirmed resolved
We are aware of incident which caused some intermittent periods of errors and timeouts between 04:00 and 04:45 UTC this morning. The Infrastructure team have identified and resolved the issue.
We are aware that following a portal update, some customers may see incorrect usage data displayed. Our development team are currently validating the fix and we expect this to be deployed on Monday. Certificate issuance and use of the APIs themselves is not affected.
The development team are deploying a fix which should restore proper visualisation of usage for some sets of customers. A fix for remaining affected customers is expected to be deployed tomorrow.
The development team are still working on resolving the issue. We will continue to post updates as they become available
The issue affecting usage display in the Atlas portal has been resolved. All fixes have been successfully deployed, and the service is operating normally.
We have seen a brief issue with issuance on GCC causing some certificate requests to fail or be delayed over the last 15-20 minutes. The team are working to ensure the problem is resolved.
The issue has been confirmed as resolved.
We identified an issue that affected the processing of some OCSP requests. Mitigation steps have been implemented, and the service is currently operating normally. We are continuing to monitor the situation to ensure ongoing stability.
We are continuing to work with our CDN providers to troubleshoot the issue before restoring full CDN redundancy. To provide some more details - the first reports of the issue were received on 4th March, with further reports on 9th-12th March. We were able to recreate and identify the issue on 16th March, with a mitigation put in place around 12:25 UTC on 16th March.
Our third party CDN provider has confirmed that a fix has been deployed and we are in the process of testing this before planning to restore full CDN redundancy tomorrow at around 10:00 UTC. Our team will be monitoring closely throughout this period.
Full CDN redundancy has been restored as planned, and behaviour is under close monitoring. If no issues are encountered, we will close this issue.
No further issues have been encountered
The team have seen an increase in error rate on the Atlas (HVCA and DSS) APIs. This seems to be related to an internal failover event but showed a brief failure rate of around 4% on /login calls. The team are confirming the root cause and are monitoring the situation, although it appears to be stable.
The issue has been confirmed related to a failover event on one of our database clusters and has been confirmed as resolved.
We are aware of an issue which is causing sporadic failures when calling the /stats endpoint on the Atlas HVCA API. The team is working on this and will restore full functionality as soon as we can. Certificate issuance and other features are unaffected.
The issue was resolved last night UK time, and full functionality has now been confirmed restored.
We are experiencing an issue with Japan certified timestamps. This relates to a problem at our upstream provider. We are monitoring the situation and will provide updates as we get them. This also affects customers obtaining Japan certified timestamps through the Atlas TSA and DSS services
The issue looks to have been resolved, however we are waiting for an update from our provider to confirm that everything is stable.
Our supplier has confirmed the issue is resolved.
We are seeing a higher than normal failure rate of calls to the /stats endpoints on the HVCA (Atlas Certificate Management API). This relates to problems in the logging cluster which the team are currently working to resolve. Issuance and other core services are not affected.
The team are continuing to work on resolving the issues with the logging cluster and hope to have the problem resolved soon.
The issue has been resolved and we've not seen further errors.
We are aware of an issue affecting TLS certificates issued on 1st December which can cause them to not be trusted by some browsers. This is caused by the trust status of some of the 2027 CT Logs. We are urgently removing affected logs from our configuration. Re-issuing affected certificates will resolve the problem.
The issue affecting TLS certificates issued on or after December 1 has been resolved. All affected CT logs have been removed from our configuration, and reissuing certificates restores trust in all major browsers. If you have not yet reissued your certificate, please do so to ensure proper functionality. Services are operating normally.
We have detected an issue with the Atlas portal which may cause errors when trying to access some sections. This is caused by the browser caching stale code - if you experience this issue, please do a hard reload ("Ctrl-F5" or "Ctrl-Shift-R") or clear your browser cache, which should resolve the problem.
We are no longer seeing any failed requests in our logs relating to this issue. If you do experience any problems with the portal, please try the actions mentioned above which should resolve it.
