Historia awarii Pulsara

We are currently investigating this issue.

This incident has been resolved. Customers may have experienced delayed push notifications from 9:54 AM MT for approx 45 minutes.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

At 11:30 MT, we observed an issue affecting the viewing of images uploaded by our imaging partners. We have identified the root cause and are currently preparing a resolution. We anticipate deploying the fix later this evening. Once deployed, all previously uploaded images will be viewable again.

We are continuing to work on a fix for this issue.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

The Pulsara Calling feature is temporarily unavailable because our third-party partner is down. Pulsara is investigating this issue and will update the Pulsara Status Page when more information is available.

Pulsara's third-party video platform has restored all services, and Pulsara Calls are working. Pulsara will continue to monitor this issue.

This incident has been resolved.

The Pulsara platform was temporarily unavailable from approximately 15:44 to 15:46 US Mountain time today. A crash of a cloud vendor component resulted in errors for all application functionality. Automatic monitoring and recovery systems were initiated. All systems are back online, and normal operations have resumed.

We have identified and are monitoring an issue with Google Firebase Cloud Messaging (FCM) affecting a small number of notifications sent to Android devices. FCM is aware of this issue and is providing status information on their Firebase Status Dashboard (https://status.firebase.google.com/incidents/PTWWAYPjkjPapM4qkGY2) We will post updates here as they become available from Firebase. Please Note: this does not affect any Apple devices.

We are continuing to monitor for any further issues.

Google FCM incident status page has announced that they have mitigated this issue and are currently monitoring it. As Pulsara opened this incident as a helpful way of providing transparency to this event, we will close out this incident now. For further information, please visit https://status.firebase.google.com/incidents/PTWWAYPjkjPapM4qkGY2. As always, Pulsara will continue monitoring all third-party services to ensure our product is running.

We are currently investigating this issue.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

This incident has been resolved. Pulsara will provide RCA information on this incident when it becomes available.

**Incident Summary:** On 08/27/2024, our web application experienced an error that prevented some users from accessing their account information. This issue occurred due to a mismatch between the API responses from our older web application and the newer server. **Impact:** Some users could not log in to access their account information or perform actions that required user authentication. **Resolution:** The issue was resolved after completing the web upgrade, including a parsing error fix. We apologize for any inconvenience this incident may have caused. We have already taken steps and remain committed to improving our systems and processes to prevent similar issues in the future.

The Pulsara platform was temporarily unavailable from approximately 18:04 to 18:07 US Mountain time today. A crash of a cloud vendor component resulted in errors for all application functionality. Automatic monitoring and recovery systems were initiated. All systems are back online and normal operations have resumed.

The Pulsara platform experienced a service outage due to a database failure from 13:41 MDT until 13:48 MDT. The Pulsara team was notified by internal monitoring systems about the issue and resolved the issue. All systems and services have recovered and are operating normally. The team continues to monitor all potentially impacted services.

An issue with the On Call Callendar integration has been identified and is being investigated.

Our development team is investigating an issue where our calendar integration is not appropriately applying the calendar filters to the on-call status. This may result in team members that use the calendar integration to be placed on call when they should not be.

The root cause of this issue with the calendar integration has been identified. We are actively working on a resolution and will provide a status update within the next 1-2 hours. Please monitor the call status of your team members who are using the calendar integration during this time.

We are currently working on restoring the calendar filters on a customer-by-customer basis.

All calendar filters have now been restored. At this time, all calendar integrations should be functioning normally. We are continuing to monitor this issue.

This incident has been resolved.

Beginning 07:30 US Mountain time on 10/25/2023, we began seeing intermittent failures with Edge push notifications. There appears to be a global issue with Microsoft Windows Notification Service (WNS) related to DNS entries for a portion of their infrastructure. This is not a problem specific to Pulsara or a problem that is in Pulsara's control to fix. We recommend using Google Chrome if possible during this degraded performance of Microsoft Edge.

Microsoft has resolved its DNS issue.

Issue description: We have identified a Microsoft issue that started at approximately 1100 hrs CDT on 08/09/23. Browser Push notifications sent to customers using Microsoft Edge are not delivered approximately 50% of the time. Impact: Customers using Microsoft Edge EDs and other teams may not receive notifications when logged into Pulsara HQ using the Microsoft Edge browser for inbound patients, consult and transfer requests, etc. Note: New patients in Pulsara HQ will still flash red to notify you of new patients. What do I need to do about it? We strongly recommend that customers relying on Pulsara browser push notifications (audible alerts) for patient awareness switch to Google Chrome until this Microsoft issue is resolved.

Microsoft recently released an update that disabled some previous settings for notification services in the Edge Browser. This issue has been identified and tied to this recent update from Microsoft Edge. To resolve this issue, please adjust your Edge browser settings to “enable” the Enable system notification setting. You can find this setting by navigating to edge://flags/#enable-system-notifications in your Edge browser. When you navigate this page, you will see the setting “Enable system notifications.” Please make sure that this setting is enabled. After you adjust this setting, we recommend sending yourself a test alert to ensure that alerts are working correctly. Please work with your IT staff if you have problems adjusting this setting.

We are monitoring this issue to ensure that the issue is resolved.

According to our monitoring, Microsoft resolved an issue on Friday, August 11th at approximately 0500 MST. We will continue to monitor the situation. At this time, all Microsoft Edge browser push notification functionality appears to be functioning normally.

We are currently experiencing a service disruption with our browser application. This only affects customers that are using our Federated Authentication integration. Our Development team is working to identify the root cause and implement a solution. If you are a customer utilizing Federated Authentication and see a black screen when logged in, the immediate workaround to get logged into Pulsara is to clear the browser data and then log back into Pulsara.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We are investigating an issue with our third-party service that delivers SMS Pulsara Patient invites. Currently, SMS invites are being delayed. Our engineering team is working on a resolution, and we will provide another update in 2 hours or as soon as more information becomes available.

We are continuing to investigate this issue. While the root cause is still being determined, the Pulsara Patient app can still be downloaded directly from the Apple App Store or the Google PlayStore.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We have identified an intermittent Pulsara MANAGER (Web application) issue that is impacting the ability of some Hospital Admin users to be able to manage the call status of other users. The Mobile application is not impacted and users are able to manage their own call status from within the HQ page or Android / iOS Pulsara MOBILE applications. We are currently working to resolve this issue and will update this page with additional information as it becomes available.

We are continuing to investigate this issue.

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We have identified an issue that has been reported by a few customers. These customers have reported that they have not been able to toggle on call from Pulsara MOBILE. Pulsara Administrators, should any of your users report this issue, our recommendation at this time is that you manage the affected user’s call status via Pulsara MANAGER. Pulsara is currently working to resolve this issue and will follow up with you when the resolution is deployed.

The root cause of the issue has been identified. A fix is in progress, will be tested, and then deployed to production as soon as testing is complete.

We are continuing to work on a fix for this issue.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We are currently experiencing a service disruption. Our Development team is working to identify the root cause and implement a solution.

As a result of issues with our cloud vendor, we're seeing intermittent issues with requests to our API endpoints.

We are continuing to monitor for any further issues.

This incident has been resolved.

Pulsara customers are having intermittent issues connecting to the Pulsara platform due to a partial outage of our upstream DNS provider. We are actively investigating and have started the process to switch to our backup DNS provider.

A fix has been implemented and we are monitoring the results.

This incident has been resolved. We will update this incident with the full root cause along with resolution shortly.

Delayed alerting for US customers from 12:27 MT to 14:12 MT

We are actively investigating delays in push notification delivery.

We have identified an issue with one of our third party vendors. We are actively working on a fix for this. Currently, no push notification are being sent.

Push notifications have now been restored. We are currently monitoring this event and will have more details to follow.

This incident has been resolved.

Today, at roughly 14:35 MST our internal monitoring systems alerted us of push notification failures. At 14:38 MST our engineering team discovered that the root cause of this incident was a failure with one of our third party vendors. During this time, no push notifications on either Android or iOS were sent. Pulsara video calls and all other data components continued to work during this time. Our development team continued to monitor this event and prepared backup alternatives to deploy to our production environment if needed. At 14:55 MST our third party vendor resumed working status and push notifications began to work again. At this time, all new push notifications were sent as expected. As of 14:55 MST, Pulsara was completely back up and running at 100%.