Neon Fundraise 장애 기록

We are experiencing additional issues with Admin users being able to login to the system. Users may experience intermittent delays when attempting to access Neon apps. This does not affect individual Fundraiser access to their pages. The team is actively working on a resolution and will update here with more information as we work to resolve the issue.

All systems are currently operating as expected and Admin users should no longer be experiencing delays when logging in. We will continue to monitor the situation.

This incident has been marked as resolved following extended monitoring of the solution put in place.

We are currently investigating an issue where Neon Portal is loading slowly or timing out for many users. We are typically seeing logins resolve after a delay, but you may experience delayed login times to Neon One apps.

Loading performance has improved, but we're continuing to monitor closely to ensure that the issue is fully resolved.

Loading performance has maintained stable, but we'll continue to monitor closely to ensure that the issue does not recur.

We are currently investigating an issue causing charges to intermittently fail on Neon Pay.

We have isolated the issue and mitigated the affect to Payments. All charges are currently processing as expected and we are continuing to monitor as we work towards full resolution.

All charges have continued to process as expected - we will continue to monitor the situation before marking this as fully resolved.

No further issues have occurred since the fix was implemented and monitoring began, the issue has been marked as fully resolved.

We have identified an issue with high cpu usage. A fix has been implemented and we are monitoring to confirm that the issue does not recur.

This incident has been marked as resolved. The fix implemented earlier has addressed the CPU usage, and monitoring has confirmed that levels have remained normal since the issue was addressed.

AWS (our hosting provider) experienced a degradation of services last night and continues to experience issues in rectifying services. Overall Fundraise systems are operational as of now, but we are continuing to investigate to determine any impact. Information on the outage from AWS can be found here: https://health.aws.amazon.com/health/status

AWS (our hosting provider) experienced a degradation of services last night and continues to experience issues in rectifying services. Overall Fundraise systems are operational as of now. At this point we have not discovered any affected services, but we are continuing to investigate to determine any impact. Information on the outage from AWS can be found here: https://health.aws.amazon.com/health/status

AWS has confirmed that their fix is working as expected. At this point, we have concluded our investigation and have not found any additional services affected within Neon Fundraise. Please reach out to support with any additional questions or concerns. AWS Statuspage: https://health.aws.amazon.com/health/status

We are currently investigating an issue preventing users from logging into Neon Fundraise Admin. This issue is not affecting front-end forms, events, or payments.

We have identified the root cause of the issue and are working to release a fix.

A fix has been implemented for this issue and we are monitoring the situation.

This incident has been resolved.

Due to an update merged in last night's release, team captains cannot be created, neither through the front end registration form, nor through the Admin. This also means users cannot create teams through the registration form. The team has identified the issue and is preparing a fix to release tonight.

This incident has been resolved.

Our team discovered that recurring plan charges have not run since April 3rd due to a thread stoppage. The thread will be unblocked during tonight’s scheduled release, after which charges will resume from where they left off, so any plan that missed a charge during this period will be charged soon after the release. The team is also taking steps to prevent this from happening again in the future.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We are seeing a system slowdown. We’re actively working speed things up.

We are currently investigating this issue.

We have restarted the system, after which sites are now fully operational.

This incident has been resolved.

Rallybound will perform server/infrastructure maintenance this evening. The system may be briefly unavailable (0-5 minutes) as work is completed.

The issue has been identified and a fix is being implemented.

We are continuing to work on a fix for this issue.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We are currently investigating this issue.

We were able to put measures in place to reduce load on the system. Sites and interfaces are now loading as normal. Team is still investigating root cause.

We are continuing to see intermittent spikes in CPU which causes the system to be briefly unavailable for a minute or so at a time. The team is still investigating.

The system was cycled with last night's release, bringing performance back to normal levels.

We are seeing another system slowdown. We’re actively working speed things up.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

Neon CRM syncs are delayed. A fix will be implemented tonight and all failed syncs will be manually re-initiated tomorrow.

Neon CRM sync is back up and running. Missed syncs have been initiated manually.

We are seeing another system slowdown, related to previous ones we’ve been seeing. We’re actively working to clear up queues and queries to free up load.

A fix has been implemented and we are monitoring the results.

Previous identification turned out to be incorrect. Team is still investigating.

Slowdown has cleared, the system has been operating smoothly for a few hours.

This incident has been resolved.

We are seeing another system slowdown, related to previous ones we’ve been seeing. We’re actively working to clear up queues and queries to free up load.

This incident has been resolved. We will be releasing updates tonight that should prevent such incidents from recurring.

We are currently investigating this issue.

Systems are operating normally. We are monitoring and exploring optimization options to prevent further slowdowns.

This incident has been resolved.

We are currently investigating reports of slowness.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We have identified the issue and are working to fix.

We are continuing to work on a fix for this issue.

We have implemented a fix. Sites should be back up. Monitoring currently.

The incident has been resolved. Sites are now loading correctly. Root cause: the CDN server was configured to use an older SSL certificate that expired today (5/19). We have since updated the server to use our more current certificate. With our recent move to AWS, we were now able to configure certificates to auto-renew, ensuring this issue does not happen again.

Some users are reporting increased load times in various interfaces. Our team is currently investigating.

We've increased system capacity to handle the additional load. Load times should be back to normal. Still investigating root cause.

Root cause has been identified. Increasing system capacity has resolved the issue.

Neon Fundraise's Facebook integration has been down for non-whitelabeled customers starting Monday evening. This impacts users' ability to login via Facebook, and to create new Facebook Fundraiser pages from within Fundraise. We have identified the issue as being related to our Facebook app and are working to resolve it as soon as possible.

The Facebook app has been updated and the integration is working as expected.

This incident has been resolved.