Ncedcloud outage history

May 23, 2024 9:35AM EDT Identified - NCDPI and Identity Automation have identified a data issue delaying the processing of Student and Staff data this morning. The issue has been identified and we are currently working towards a resolution. May 23, 2024 10:26AM EDT Identified - NCDPI and Identity Automation currently expect data to be processed successfully tomorrow morning, 5/24 during the next file run. May 24, 2024 9:16AM EDT Identified - The Student and Staff data processing delay persists as of 5/24. NCDPI and Identity Automation continue to work towards a resolution and will continue to communicate status until the issue is fully resolved. May 30, 2024 10:14AM EDT Identified - Student and Staff data is accurate up to 5/28. While the underlying cause of the file issues is being investigated, NCDPI and Identity Automation have a plan to manually fix and process the files on Mondays, Wednesdays, and Fridays until the issue is resolved. The next Student and Staff files will be processed on 5/31. May 31, 2024 10:15AM EDT Resolved - Student and Employee files were processed as expected on 5/31.

March 27, 2024 4:55PM EDT Identified - NCEdCloud Staff data updates are currently delayed due to an issue where a user field in source data is too large for the data staging system. NCDPI and Identity Automation are currently investigating and have come up with a proposed resolution. Another status update will be posted on the morning of 3/28. Staff data should be accurate up to 3/22 at this time. Note that employee adds are not affected, only updates of existing employees. March 28, 2024 9:58AM EDT Monitoring - NCEdCloud Staff Data should be current as of 3/28. NCPDI and Identity Automation are working to confirm that the issue will not reoccur in the 3/29 staff file. March 29, 2024 8:49AM EDT Resolved - Issue has been resolved as staff data process successfully as expected on 3/29.

February 20, 2024 8:33AM EST Investigating - NCEdCloud Source Data processing is currently on hold due to file delivery issues. NCDPI and Identity Automation are actively investigating. NCEdCloud data should be current up to 2/18. February 20, 2024 3:53PM EST Identified - NCDPI and Identity Automation expect that the file processing issue will be resolved on 2/21. February 21, 2024 8:45AM EST Resolved - Data in NCEdCloud is current as of 2/21.

January 30, 2024 10:02AM EST Investigating - NCEdCloud source data processing is currently delayed due to an unexpected number of emails missing on user accounts. The issue is being investigated by IDAuto and NCDPI. No NCEdCloud accounts have been affected, but processing will be delayed until new files are processed. January 31, 2024 9:57AM EST Resolved - Source data for 1/31 has been successfully processed.

September 15, 2022 10:28AM EDT Investigating - We have identified there is an issues and latency is expected at this time. We are investigating the core issue and will update site as details are available. September 15, 2022 2:16PM EDT Monitoring - At this time we are monitoring the site in order to ensure there are no additional issues. Upgraded to Operational. September 16, 2022 8:24AM EDT Investigating - It appears that we are experiencing latency within NCEdCloud 9-16-22. Currently amending the situation within AWS and will be monitor situation soon. September 21, 2022 12:44PM EDT Monitoring - Currently still monitoring issues that were happening with NCEdCloud last week. RCA occurring. September 29, 2022 2:03PM EDT Resolved - Issue is resolved. This is a 24 hours a day 7 days a week monitored event with alerts attached. RCA being determined.

February 28, 2022 1:37PM EST Monitoring - Amazon SNS service enables Rapid Identity to send SMS text messages directly to your employees. This service is currently having issues with specifically T-Mobile carrier. There is no indication that there are issues with any other carriers at this time. February 28, 2022 4:23PM EST Monitoring - The issue is not causing all carriers to be affected only T-Mobile and Sprint. March 1, 2022 9:42AM EST Resolved - On February 26, 2022, it appears that T-mobile's SMS servers started blocking the 10 origin numbers associated with our Amazon SNS account. We've reached out to the carrier to get reason behind the blockage, but at this time have still not heard back from them. To mitigate this issue we generated a new bank of origin phone numbers to send from. This change to origin numbers appears to have worked as we are now seeing successful delivery rate creep back up to normal. SMS service should now be restored for your T-mobile users. From what we can tell, this issue did not affect any other phone carrier.

February 10, 2022 11:11AM EST Investigating - We are currently experiencing a service disruption for NCDPI/NCEdCloud. We have identified the issue and are currently working to repair. No estimate at this time for repair. The ticket number for reference is 00062626. February 10, 2022 11:31AM EST Monitoring - Issue is being monitored and root cause analysis is being performed. All users should be able to access NCEdCloud at this time. February 10, 2022 1:00PM EST Resolved - This morning we experienced a 20 minute downtime that caused intermittent problems logging into NCEdCloud. The interruption has been identified and resolved with AWS and Identity Automation. Users should not be experiencing any further issues logging into NCEdCloud.

January 28, 2022 9:14AM EST Identified - There was an issue with a sync in PowerSchool that has been identified. There is a file that has not been able to process since Wednesday, January 26, 2022 identified as the Student file. We are currently working on a resolution for this issue. There will be no interruption or degradation of performance. Identity Automation appreciates your patience while we rectify the matter. January 30, 2022 10:25PM EST Resolved - All student file have been processed at this time.

January 9, 2022 9:50PM EST Identified - All internal Identity Automation emails are back up at this time. We do not expect any further downtime. January 9, 2022 9:50PM EST Resolved - All internal Identity Automation emails are back up at this time. We do not expect any further downtime.