26 Greynoise incidents · settembre 2019 — official updates, affected components, duration and resolution details.
Our API is currently offline; we are investigating the cause and resolution.
The issue was tracked down to DoS-level traffic to our API. We’ve temporarily suspended the accounts generating that traffic, and will be implementing stricter rate limiting of API calls to prevent future occurrences.
We are experiencing delays in data processing impacting delivery of session, finished intelligence, and bulk data sets. Fixes are being applied to restore processing performance and burn down the backlog.
We're continuing to work on fixes for data processing. We expect to begin working through the backlog within the hour.
We are starting backlog processing. Following restoration of data availability for impacted hours, we will export impacted bulk data hourly buckets.
Data backlog processing is continuing as expected. We'll update the status page when we are fully caught up and bulk data exports are started.
Data backlog processing has completed and bulk data exports are available.
We are aware of an issue causing some requests to the IP Details Page to fail.
We have rolled back the change that resulted in these errors. We are monitoring recovery
We have confirmed resolution of the issue that led to the errors.
We are investigating an issue that is causing requests in the Visualizer to fail across most accounts.
We are rolling back the change that is causing errors in the Visualizer.
The rollback is complete, and Visualizer requests are successfully completing. We are monitoring the resolution.
The Visualizer is fully operational.
We are experiencing delays in our data processing pipeline. This is impacting the recency of data available via our API and the availability of bulk data files.
We are continuing to work on restoration of pipeline operations.
The team is continuing to work on recovery.
We are seeing recovery in our data processing pipeline. The team is monitoring recovery. Missing bulk data files will be generated as we catch up on the backlog.
Backlog reprocessing is continuing and expected rates. The team will process missing bulk data files at ~8AM ET.
Bulk data file processing is complete for all impacted hours and pipeline data delivery is stable.
We're seeing substantial delays in the handling of inbound packets and are investigating the cause.
The cause of data processing delays has been identified and remediated. The platform is working through the data backlog. Bulk data files will be updated as data for impacted hours is processed.
Pipeline performance is still showing signs of degradation. Data is backfilling but at slower than optimal rates. The team is continuing to identify and address performance issues.
The team is continuing to work on improving processing performance. Data is still delayed, but the pipeline is gradually making progress toward catching up.
At our current processing pace we expect to be through the backlog in roughly 12 hours, including backfill of bulk data. We will update the incident periodically as we monitor and work on improvements to processing performance.
We have released upgrades to a key part of our pipeline that has improved data processing speed. Our estimated time to recovery is now two hours. Data from ~5 hours ago is still being processed. We are starting bulk data jobs to publish for hours that have already completed backfill.
Bulk data for 2026-02-26_1400 - 2026-02-26_1600 hours are available.
Bulk data for 2026-02-26_1700 - 2026-02-26_1900 hours are available.
Bulk data for 2026-02-26_2000 - 2026-02-26_2200 hours are available.
Data backfill is complete. All bulk data hours are available.
API and Bulk Data sources are impacted by delays in data processing. We are monitoring the situation and taking corrective actions to catch up on the backlog
We are continuing to monitor recovery. Data processing is still delayed.
We are seeing recovery in data processing rates. At the current pace of recovery we expect normal operations and data backfill to be complete in roughly 4.5hrs. Our team will backfill bulk data as hourly windows finish catching up.
We are continuing to monitor recovery. Data processing rates have remained consistent with prior measurements. The team expects to backfill bulk data around 8AM ET.
Backfill jobs for bulk data have started for hours 00-07 UTC.
Data backfill for bulk data is continuing to process.
Bulk data backfill is complete. Overall platform data processing is performance is restored.
New logins to the Visualizer are not working. Existing sessions remain active.
We are working on rollback of infrastructure changes that are causing logins to fail.
Logins are working again. We are monitoring recovery.
The issue with Visualizer logins is resolved.
- Greynoise Bulk data delay - API Data freshness degraded
Our data service has been restored to real time.
1 hour behind on processing bulk data
Bulk data processing has caught up
We are experiencing occasional service disruptions across our platform as a result of AWS issues.
We're continuing to monitor AWS status and impacts.
AWS has recovered from their outage. GreyNoise services are fully operational. Greynoise sensor data from during the outage window has been processed.
GreyNoise Tags are not loading in the GreyNoise API. This is impacting most Visualizer pages and some API endpoints.
The GreyNoise API is recovering and calls for tags are succeeding. We are monitoring recovery.
The GreyNoise API is fully operational.
The GreyNoise Visualizer (viz.greynoise.io) is currently down; the team is investigating.
We have reverted the Visualizer to a prior version and are monitoring the service.
The Visualizer is stable.
The GreyNoise website (https://greynoise.io) is experiencing intermittent availability issues due to an issue with our third-party hosting provider.
The issue appears to have been resolved and the website is fully operational again.
The observed IP counts in our Vulnerability Prioritization Intelligence product are not accurate. We are investigating the cause.
The incident is resolved and the underlying service code has been updated.
Calls for some blocklist tags are failing to return IP addresses.
Tag blocklist API calls are successfully returning IP addresses. We are working to restore data recency back to the current hour.
Data processing is recovering. The platform is working through the backlog of data. We will resolve the incident once backlog processing is complete.
Data recency is continue to recover. Hours from 19:00 UTC on will be reprocessed following recovery to ensure completeness.
Data reprocessing is complete. All systems are operational.
We had an interruption in data processing; we've rolled back some changes to restore normal data processing.
Data processing is caught back up for real time data, and we're backfilling bulk data and the Visualizer/API data set.
The issue has been resolved.
Some Blocklists are currently returning no results. We are investigating the cause.
We believe we've resolved this issue and are monitoring performance.
This incident is resolved.
The generation of hourly bulk data files has been delayed. We are re-running the past few hours of files and are making changes to get generation back on schedule.
We increased capacity on the pipeline to address the surge of data. Processing has returned to normal.
We're experiencing a very heavy burst of traffic and are experiencing delays in generating bulk data files.
We’ve identified the source of the issue and have made changes. We will shortly update with expected recovery time.
The system is working through the backlog and should be recovered in a few hours. New hourly bulk files will be generated, and our team will re-run any missing hourly files.
The system is continuing to work through the backlog. Recovery is taking longer than initially expected. The team is work on mitigations to improve time to recovery. Missing hourly files will be re-run once the backlog is processed.
The incident has been resolved.