Inline Manual outage history

We are aware of the recently disclosed vulnerability CVE-2026-66066 and are treating it seriously. Our production systems have already been patched. We are now conducting a thorough forensic review of relevant logs to confirm whether there was any attempt to exploit the vulnerability before the patch was applied. At this stage, we have found no evidence that Inline Manual was affected. We are continuing the investigation as a precaution and will share another update once the review is complete. [CVE-2026-66066] Possible arbitrary file read and remote code execution in Active Storage variant processing - https://discuss.rubyonrails.org/t/cve-2026-66066-possible-arbitrary-file-read-and-remote-code-execution-in-active-storage-variant-processing/91432

We are continuing to review our logs for any signs that CVE-2026-66066 was exploited before our systems were patched. So far, we have found no evidence of malicious activity related to this vulnerability. Our rapid response appears to have prevented any exploitation, and we have also rotated all affected secret keys as an additional precaution. Importantly, even if an attacker had attempted to exploit this vulnerability, it would not have given them access to customer data. We have confirmed this based on how the affected component is isolated within our infrastructure. We will continue reviewing the logs and monitoring for any suspicious activity over the weekend. We will publish another update sooner if we identify anything relevant, or by Monday, August 3 at the latest if there are no new findings.

Resolved - no evidence of exploitation We have completed our investigation into the potential impact of CVE-2026-66066. We took a deliberately thorough approach and went beyond our standard review process to ensure we had examined every relevant source of information. This included: - Patching the affected systems immediately - Reviewing application, infrastructure, and access logs covering the relevant period - Searching for known indicators and patterns associated with exploitation - Investigating unusual requests and activity in greater detail - Rotating all potentially affected secret keys as a precaution - Continuing enhanced monitoring after the initial review was completed - Verifying that the affected component could not provide access to customer data After completing this work, we have found zero evidence that the vulnerability was exploited or that Inline Manual was affected. We are therefore marking this incident as resolved. We will continue to follow our normal security monitoring processes and will take further action if any new information becomes available. Thank you, Marek and the team

We are currently experiencing technical difficulties with our main website, inlinemanual.com. Our team is actively working to resolve the issue as quickly as possible, and we will provide updates as we have more information. In the meantime, we want to reassure you that our platform, app.inlinemanual.com, is fully operational and unaffected by this outage. You can continue to access all your sites, content, and features without any interruption. We apologize for any inconvenience this may cause and appreciate your patience and understanding as we work to restore our main website. If you have any questions or need assistance, please feel free to contact our support team at [email protected].

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We're following the evolution of the situation around CVE-2024-3094 - https://nvd.nist.gov/vuln/detail/CVE-2024-3094 Our Engineering and Security teams are currently investigating if the CVE-2024-3094 incident applies to any of our service resources.

This incident has been resolved. After a thorough investigation we conclude that our systems were not affected by the CVE-2024-3094 and the specified library versions.

Authoring Tool is currently unavailable. We are actively investigating the issue and working to restore access as soon as possible. Thank you for your understanding and patience.

We have identified the issue affecting the Authoring Tool and are diligently working on a fix. We expect to restore full functionality soon.

The issue with the Authoring Tool has been resolved and we are now closely monitoring the system for stability. Thank you for your patience, and we apologize for any inconvenience caused. Please feel free to resume using the tool.

This incident has been resolved.

We are experiencing a degraded performance on one of our parts of the Inline Manual Portal - app.inlinemanual.com. Affected areas: - People Tab - latest profiles and updates are not showing. - Various operations on People profiles are not working through the Portal. This does not affect your users' experience, as their progress is still being appropriately stored and processed without any delay.

We have identified the issue and are working on a fix.

Issue has been fixed. Sync jobs across all analytics servers are running as expected. The deletion and other Profile actions are working. The Profiles are being re-synchronised. We are monitoring the re-sync process as it might take several hours to update affected profiles fully and for any recurrence of issues. We have confirmed that this did not impact the end-users' experience and only impacted the Inline Manual's Portal UI, where we display Profiles.

We have been following the data consistency thoroughly throughout the past few weeks. This incident has been resolved.

Incident ongoing since December 11th. We're following the evolution of the situation around CVE-2021-45046. Our Engineering and Security teams started to work on log4j incident on December 10th in the morning EU time and updated, mitigated or removed the vulnerable log4j packages from our environment. No misuse has been detected and active threat hunting continues. Our mitigation actions: - Updated log4j where possible - Scanned logs retrospectively one month back for various scenarios - Our security team is still actively hunting for signs of exploitation The service affected is not directly accessible through internet. It sits within our private network. The request can be made through an API which goes through application first, not passing any headers and it correctly sanitizes all user entered content. This has been verified and confirmed. The service itself logs only minimum information, none of the user entered content makes it through to the logs. The attacker would need to be logged in to Inline Manual Portal or have API Access Tokens to be able to communicate indirectly through the API with the service - we have not detected any attempts to do so.

We are closely following Log4j vulnerability evolution and the latest release of Log4j 2.17.0. As we have fully removed Log4j packages from our infrastructure, we are not affected by newly found issues within Log4j packages.

This incident has been resolved. Our systems were not compromised nor affected by the Log4j vulnerability.

We are experiencing downtime on our main app and analytics due to worldwide Google Cloud downtime. We are investigating.

We are continuing to investigate this issue.

Incident identified and confirmed to relate to Google Cloud platform global network issues. https://status.cloud.google.com/incidents/6PM5mNd43NbMqjCZ5REh

All systems are back to normal and we are monitoring the Google Cloud's status.

The issue with Google Cloud Networking has been resolved for all affected projects as of Tuesday, 2021-11-16 11:28 US/Pacific. https://status.cloud.google.com/incidents/6PM5mNd43NbMqjCZ5REh

We are experiencing degraded performance on Inline Manual Portal. This affects only users within the Inline Manual Portal at app.inlinemanual.com making some pages unavailable. The end-users consuming the player are not affected.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We are currently investigating this issue.

We have identified the problem and fixed the primary cause. We continue monitoring the situation.

This incident has been resolved.