52 Pike13 incidents · febbraio 2022 — official updates, affected components, duration and resolution details.
SmartWaiver sta vivendo un outage di servizio. Sono il partner che i poteri rinunciano all'interno di Pike13. Tutte le funzionalità waiver sono attualmente interrotte.
Gli errori si stanno suddividendo. Stiamo monitorando.
Questo incidente è stato risolto.
Tradotto automaticamente dall'aggiornamento ufficiale dell'incidente.
Il problema è stato identificato e si sta attuando una correzione.
Questo incidente è risolto.
Tradotto automaticamente dall'aggiornamento ufficiale dell'incidente.
Le transazioni con carta di credito per alcuni fornitori stanno attualmente fallendo. Stiamo indagando sugli errori di connessione al gateway di pagamento.
Il gateway NMI ha confermato che hanno problemi di connettività. Potete seguire qui: https://status.transactiongateway.com/
NMI sta lavorando per risolvere il loro problema.
Non abbiamo visto errori NMI negli ultimi 10 minuti. NMI non ha ancora confermato che hanno risolto questo problema, quindi stiamo ancora monitorando.
Una soluzione è stata implementata e stiamo monitorando i risultati.
NMI ha risolto il problema della connettività. Questo è ora risolto.
Tradotto automaticamente dall'aggiornamento ufficiale dell'incidente.
We have contacted Celero about connectivity issues that started 5:00am PDT today.
Connectivity errors have ceased as of 30 minutes ago, but we have not yet received confirmation from Celero that the the issue is fully resolved.
This incident has been resolved.
Something isn't working right now. We're checking into it and will update you with more information ASAP.
We're back. Front Desk is up and running. Sorry for any trouble.
We are currently investigating this issue.
The issue has been resolved.
We are currently investigating this issue.
We are continuing to investigate this issue.
The incident has been resolved. Systems are back to normal. We are investigating root cause and prevention steps.
Our partner SmartWaiver is having technical issues. Waivers in Pike13, which all use the SmartWaiver service, are not working. We have opened a support ticket with them.
SmartWavier status can be viewed at https://status.smartwaiver.com/.
SmartWaiver says they have identified the problem and are working to resolve it. No estimated time to resolution yet.
There have been no waiver errors in the past 20 minutes. SmartWaiver says they have fixed the issue.
We are currently investigating this issue.
The database outage is over. Historical activities records will not display currently. We are starting the process of back filling those records into the rebuilt table.
Backfill of historical activities is progressing starting with most recent history to oldest. It will take several hours.
Activities data has been restored from present back to November 1, 2025. The restore is continuing and is processing approximately 1 million records per minute going backwards in historical order. All records going all the way back to the year 2012 will be restored by tomorrow.
This incident has been resolved.
Background processes exhausted system memory. We restarted services and background jobs caught up by 9:15am PST. We have added more redundancy to prevent future occurrences. We have also improved system monitoring for this event so that we will have faster resolution in the future.
Our engineers are investigating slow performance and page load times across the platform.
Performance has returned to normal, but we are continuing to monitor and investigate.
This incident has been resolved. The primary impact was to search (searching for clients, searching for staff member, etc.).
Something isn't working right now. We're checking into it and will update you with more information ASAP.
We are continuing to investigate this issue.
Systems have returned to normal status.
We are currently investigating this issue.
This incident has been resolved.
We are currently investigating an issue with reporting data not being updated.
The issue has been resolved. Replication is catching up. Stay tuned here for updates on latency reduction until fully caught up.
Reporting latency is currently approximately 24 hours behind.
Data latency in reporting continues to slowly decrease. We are constantly monitoring the situation and will update this page at least once every four hours with the current latency until reporting data reaches expected parameters. We expect full resolution by end of day today.
Latency continues to decrease slowly as the system processes all of the data from yesterday. It will decrease more rapidly once all of the data from peak activity hours yesterday has been processed. We still expect reporting to be fully caught up to normal latency by the end of the day today.
Reporting data is now loaded up to 4:30pm EDT yesterday (July 8). It still needs to process the remainder of July 8, and of course catch up on July 9.
Reporting data is now loaded up to 6:20pm EDT yesterday (July 8). Slow going as it churns through peak-time data.
Latency is dropping more rapidly. The system has now moved on to processing data from today (July 9).
Reporting data is now loaded up to July 9 6:15pm EDT. Still a ways to go.
Reporting data is now loaded up through end of day July 9 EDT.
Reporting data is now loaded up through today (July 10) 1:15pm EDT. Nearly caught up now.
Reporting data latency has returned to normal.
External partner SmartWaiver is having issues. This is preventing Pike13 users from signing waivers. https://status.smartwaiver.com/
SmartWaiver restored their service about 25 minutes ago.
Server pool failed to scale up with increasing load. Fixed.
Between 4:19pm and 4:22pm US Pacific Time, there was an intense spike in load on the database that cause Pike13 applications to stop responding.
Pike13 had been intermittently slow throughout the day. We are investigating.
We are going to replace the hardware for the database in 10 minutes from now. There will be a 5-minute outage.
Slightly delayed while complete a full database backup first. Estimated time of outage now 11:00pm EDT.
The hardware replacement is in progress.
Maintenance complete. Systems operating normally now. We will be monitoring for recurrence of the intermittent slowness.
We are currently investigating this issue.
Database restart. All system operational. Continuing to monitor performance.
Something isn't working right now. We're checking into it and will update you with more information ASAP.
There does not appear to be any operational impact, but we are checking to be certain.
One of the servers in the Reporting API cluster had connectivity issues and was removed from rotation. The other servers in the cluster were able to handle all of the traffic, which is the purpose of having the redundancy. Everything working normally.
