Blindside Networks outage history

We've identified an issue in the recently released Chrome 143 that prevents sharing a browser tab in screen sharing from working in BigBlueButton. We have a fix in place and are rolling it out now. The fix should be available to all users within the next 24 - 48 hours. As a workaround, we recommend any user wanting to share a browser tab in Chrome 143 to share application or full screen instead.

Updates rolled out over the weekend. Sharing of browser tab in Chrome 143 now works again. As an added bonus, Chrome on Mac OS X now supports sharing of system audio when sharing desktop!

We have identified an issue where uploaded slides in BigBlueButton will appear cutoff on the right. We have a fix in place and are rolling it out today. The problem should be resolved by start of classes tomorrow. For now, we recommend any presenters that are affected to use screen sharing to share their slides.

We've rolled out an update to our servers to resolve this issue. Uploaded presentations should now be displaying normally again.

Some users may notice longer than normal processing time due to an isolated issue with a third-party infrastructure provider. Only a limited number of recordings are affected, and all should complete within 24 hours.

Recordings are being processed and all are expected to complete within 12 hours.

All recordings previously delayed in queue have now been processed. Recording process times are back to normal.

We have identified an issue that sometimes prevents the educator from stopping screen sharing. We've already got a fix and are deploying it soon. In the meantime, here's a short video that shows how to workaround the issue and force screen sharing to stop. https://youtu.be/lcIP90hWU08 We'll update this incident when the fix is deployed to production.

We've rolled out a fix for this in our North America (USA) servers. We are rolling out a fix to other regions. We'll update this ticket when the updates have completed rolling out world-wide.

We are continuing to monitor for any further issues.

This incident has been resolved.

It looks like there was an incident 20 minutes ago with one of our hosting providers in their internet backbone connectivity. Their team is working on it and the connectivity appears to be stable now. You may have had a disruption in your BigBlueButton classes, or dropped classes. Going forward, it should be working as normal. We're monitoring and will update this status page accordingly.

Our hosting provider has indicated the issue was momentarily and now resolved. Smooth sailing ahead for your virtual classes.

Much of West and Central Africa is experiencing internet outages as undersea cables have failed. You may experience internet disruptions on any traffic routed through this area.

This incident has been resolved.

We are currently seeing networking issues in our Canadian region that is affecting a small number of meetings. We are working to circumvent the networking issue.

We see the networking issues have cleared. We will continue to monitor the servers.

Recording processing is experiencing delays in the RNA1 region. Recordings from Saturday at 3 PM EST onward are currently queued up to be processed. We are continuing to monitor this issue.

All pending recordings have been processed.

Recording processing may be slower than usual in RNA1 and REU1 regions. This delay could be up to 12 hours. We are working to restore normal processing speed.

Recording processing is starting to catch up and the backlog is being cleared.

This incident has now been resolved.

Hi everyone, On Monday, September 25, we detected an regression introduced in an upgrade to our infrastructure over the weekend. The regression would have manifested as, for some BigBlueButton sessions, the instructor could start and enter the sessions, but students could not join. Under the hood, there was a race condition in the state of the live session where our infrastructure did not return the sessions as 'active', which is the criteria for front-ends - such as Moodle, Canvas, and Schoology - to show the Join button for students to join and enter. We began receiving reports of issues early Monday morning and, after determining the underlying cause, we reverted the upgrade at around 11:00 AM ET. At that point, all subsequent sessions should have started/joined normally. We've added additional checks to ensure this issue doesn't occur again, and we apologize for any disruptions this may have caused in sessions. Regards,... Blindside Networks Team

Office 365 authenticated accounts are experiencing issues when logging into Personal Rooms. We are actively investigating.

Personal Rooms are functional. We are continuing to monitor.

We are continuing to monitor for any further issues.

This incident has been resolved.

This incident affected regions reu1 and roc2.

The Blindside Networks front-end GreenLight/Personal Rooms is experiencing issues with logging in. Users currently logged in are not affected.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We are tracking issues in a few sessions related to duplicate messages in the chat, whiteboard annotations disappearing, and connectivity issues. We are working on a fix for these issues and will have it deployed shortly.

Issues have been identified and patches are being deployed to address them.

We have deployed an update on May 9th and observe that it resolved issues with duplicate messages in the chat and whiteboard annotations disappearing.

We've identified an issue with provisioning of servers in our North America region. We are increasing our capacity and should have it resolves very soon.

This incident has been resolved. The increase in capacity has decreased the load on the existing servers.

We are seeing errors in the access for personal rooms in Europe and are investigating.

We identified a problem with delayed responses across Google Cloud regions. We made some updates to the Application and the Deployment to improve performance so the service could be recovered. The service was fully restored at 16:27 EST on Feb 21, once the changes where rolled out.

We detected an configuration issue last night (Feb 9th) that prevented launching of new meetings for customers in North America from 8:00 to 8:30 PM ET.

We apologize to those schools trying to launch classes at that time. Our engineering team had resolved the issue by 8:30 PM ET, Feb 9th. We're making changes in our processes to ensure the issue does not happen again. Starting and running of meetings in North America are proceeding normally.

Our hosting provider in North America is experiencing network connectivity issues.

Our service provider identified that between 16:24 - 17:01 UTC, there were networking issues. Network stability has been observed after the initial outage. We are monitoring the situation and will post an update once we confirm the issue is fully resolved.

Our service provider has confirmed a full resolution of the issue.

We have detected an intermittent issue with launching of new sessions in our North America region of servers. Engineering is working on a resolution.

Connections to the servers in the USA region have been stable for approximately one hour now. We believe this issue is resolved. We are continuing to monitor the situation.

Starting and running of meetings in North America have been stable for the last two hours.

We are seeing an issue with our cluster of LTI brokers regarding accepting incoming connections. We are working on resolving the underlying issue to restore service to schools that integrate with BigBlueButton via LTI.

A fix has been implemented and we are monitoring the results.

This incident has been resolved. The LTI brokers have been working as expected for 20+ hours.

As of Wednesday, Sept. 14th, we are seeing new meetings from Canvas with the following lock settings enabled by default: Share webcam See other viewers' webcams Share microphone Send Public chat messages Send Private chat messages We are working with Instructure to further investigate the cause of the lock settings being enabled by default. The workaround, for now, is that moderators can manually unlock these settings once the meeting has begun. For more information on lock, settings see: https://support.blindsidenetworks.com/hc/en-us/articles/360024409172-Lock-viewer-s-ability-to-use-their-mic-webcam-chat-shared-notes

Canvas/Instructor has released an update to revert the locking settings back to how they were handled previously. We have confirmed that the lock settings are back to normal now.