Lattice outage history

Lattice is currently experiencing an incident on the admin Compensation Cycle Recommendations page. The Recommendations table is showing a "Something Went Wrong" error.

Our team has identified the underlying root cause and and is implementing a fix to resolve the error admins see on the Compensation Cycle Recommendations page.

A fix has been deployed. Admins viewing Compensation Cycle Recommendations should refresh the page and see data successfully. We will continue monitoring for the time being.

This incident has been resolved.

Lattice experienced an issue with Lattice AI Agent and Evidence-Based Reviews generation between 12:30pm Pacific and 1:15pm Pacific. AI agent and Evidence-Based Reviews generation has been restored.

For approximately 30 minutes, email and Slack messages failed to send. A fix has been deployed, and new messages are now sending as expected. Messages sent during the affected window may be delayed. We are continuing to monitor the service.

This incident has been resolved.

From 10:52–11:36 AM PDT, some users may have experienced issues logging in, making changes in Lattice, or using API requests to create or update data. This issue has now been resolved.

Lattice is experiencing a disruption in automated review cycle schedules. All review cycles on an automated timeline have not moved their state forward and automated reminders have not sent since Thursday. Our team is actively working to identify the underlying root cause and implement a solution.

Our team has identified the underlying root cause and are implementing a fix. All missed review cycle state changes will be made and associated notifications will be delivered.

Lattice is currently processing all missed review cycle state updates and notifications. Review cycles will be moved to the proper intended status and reminder notifications will be sent for all scheduled events between Thursday, August 6th and today.

All delayed review cycle state updates and notifications have now been sent. Automated review schedules are operating on time.

Some users who previously had Slack notifications active now have Slack notifications deactivated. These can be reactivated in Profile -> Manage Settings -> Notifications. Our engineering team is actively investigating.

Slack notification settings have now been restored.

After editing Default roles, the displayed counts for viewable and editable field-level permissions may be incorrect until the page is refreshed. As a workaround, refresh the page after saving. Our engineering team is actively investigating.

Our team has identified the underlying root cause and will be implementing a fix shortly.

This issue has been resolved.

Cloudflare, an upstream provider, is currently experiencing an issue. This temporarily affected Lattice site performance and caused connection timeout errors for some users. Lattice performance has returned to normal. Our Engineering Team is continuing to monitor the situation.

We are aware of an issue present over the weekend that meant that an error message appeared when attempting to end goals as complete or incomplete. Our Engineering Team have released a fix for this issue, and this has now been resolved.

Our Engineering Team is aware of degraded performance and long loading times for the login page, with some users experiencing long page load times for different areas in Lattice. We are investigating the issue.

Our Engineering Team has investigated this issue, and we are seeing a restoration of performance across Lattice. We will be continuing to investigate the root cause behind the slow down.

Our upstream provider, Intercom, is currently experiencing an issue that prevents our customers from reaching Customer Care via live chat. If you need immediate assistance, please email us at [email protected].

The issue has been identified, users can chat in to Customer Care and they will go directly to a live agent for support.

This incident has been resolved, users can now chat into Lattice and the FIN Bot will escalate to a live agent if needed.

The issue has been identified and a fix is being implemented.

This incident has been resolved.

Users may see a Something Went Wrong error when accessing Goals in Lattice. The issue has been identified, and a fix is being implemented.

This issue has been resolved.

When viewing shared review packets, if a score has been calibrated, the manager rating reflects the calibrated rating. The issue has been identified and a fix is being implemented.

This incident has been resolved.

We are experiencing an intermittent issue where the OpenAI model is not responding to requests, which is impacting the Lattice Agent. As a result, users may notice that the Agent occasionally does not respond to requests. We are currently investigating this issue.

A solution has been identified and a fix is being implemented.

This incident has been resolved.

Review packets are showing pre-calibrated scores, even when calibrated ratings are present. The issue has been identified and a fix is being implemented.

This incident has been resolved.

Users may have seen slowness or pages failing to load across the Lattice application between 8:00am and 8:05am ET. This incident has been mitigated and Lattice performance is back to normal. Our Engineering Team is investigating the root cause behind the slowdown.

An issue with an upstream provider, AWS, caused an outage of the Lattice platform. We have confirmed the issue has been resolved from our upstream provider, and we are actively monitoring.

We are currently investigating an issue that prevents admins from exporting PowerPoint slides from Engagement survey results.

The issue has been identified and a fix is being implemented.

This incident has been resolved.

Our upstream provider, Courier, is currently experiencing an issue which is affecting our ability to successfully redirect email links. This has also caused a delay to all Lattice product notifications sent via email, MS Teams, or Slack. Our Engineering Team are aware of this issue, and are actively investigating the effects of this downtime on our services.

Courier is reporting that a fix has been implemented and the issue has been resolved, and we are seeing the Lattice application redirect emails successfully. Our Engineering Team will continue to monitor the effects of this downtime on our services. All Lattice product notifications that were delayed as a result of this incident have now been delivered. Some notifications may have been delayed by up to one day.