Kisi 장애 기록

1. 2026년 7월 15일 오전 2:27 UTCidentified

   문제는 확인되었습니다. 우리는 수정 작업을하고 있습니다.

2. 2026년 7월 15일 오전 6:27 UTCidentified

   Cisco Meraki 통합을 제외한 모든 통합에 주로 복구되었습니다. 우리는 수정에서 일하고 있습니다.

3. 2026년 7월 15일 오전 8:36 UTCidentified

   카메라 스냅 샷을 태칭하기위한 Cisco Meraki API 엔드 포인트는 예상대로 작동하지 않습니다. 우리는 Cisco Meraki 지원을 접촉했습니다.

4. 2026년 7월 16일 오전 3:38 UTCmonitoring

   Cisco Meraki는 그들의 끝에 문제점을 해결했습니다. 우리는 우리의 끝에 가공하는 스냅샷을 가능하게 하고 그것을 감시합니다. 참조 : https://status.meraki.net/incidents/6lb0s9fbny1c

5. 2026년 7월 16일 오전 4:42 UTCresolved

   Cisco Meraki Snapshots는 일반적으로 다시 일하고 있습니다. 우리는 backlog를 가지고 있지만, 그것은 몇시에 명확하게됩니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 6월 16일 오전 9:20 UTCinvestigating

   Mindbody API doesn't return any client information which subsequently led to revocation of access rights during the daily sync.

2. 2026년 6월 16일 오전 10:29 UTCidentified

   We have contacted the Mindbody support team and waiting for updates from their side. We are manually restoring access to as many as users as we can.

3. 2026년 6월 16일 오전 10:57 UTCidentified

   We are continuing to work on a fix for this issue.

4. 2026년 6월 16일 오전 11:00 UTCidentified

   We have restored access to all users who lost their access during the daily sync. Webhooks will continue to work as expected. We will pause daily syncs on Mindbody integration until the issue is resolved.

5. 2026년 6월 16일 오후 4:52 UTCresolved

   Mindbody team has confirmed that they have fixed the issue. Daily sync should proceed as usual. If you notice any issues, please reach out to Kisi Support team.

1. 2026년 6월 12일 오전 10:19 UTCmonitoring

   Between 1:15 AM and 6:15 AM UTC our API experienced a resource overload incident affecting a small subset of resources that require access to our caching backend. A fix has been deployed and we are actively monitoring the situation. We are working on restoring metadata. The services that were affected: - Event webhook delivery - Intercom calls - Incident alerts - Camera snapshots

2. 2026년 6월 12일 오후 2:12 UTCresolved

   This incident has been resolved.

1. 2026년 4월 30일 오후 12:12 UTCinvestigating

   We are currently experiencing issues in our events and incidents services due to an issue in our cloud services provider (GCP). We are actively working to mitigate the issue.

2. 2026년 4월 30일 오후 12:57 UTCmonitoring

   Our events and incidents services have been partially restored. We are working to restore missing events. Incidents dispatch may also be delayed.

3. 2026년 5월 1일 오전 7:16 UTCresolved

   This incident has been resolved.

1. 2026년 4월 10일 오후 4:03 UTCinvestigating

   We are currently investigating the issue. This affects all marketplace applications.

2. 2026년 4월 10일 오후 4:05 UTCinvestigating

   We are continuing to investigate this issue.

3. 2026년 4월 10일 오후 4:46 UTCmonitoring

   We have deployed a new version that fixes the issue. We are monitoring the recovery of the service.

4. 2026년 4월 10일 오후 6:58 UTCresolved

   Our systems have fully recovered.

1. 2026년 3월 25일 오전 5:50 UTCidentified

   We are currently investigating the issue. It appears to be related to the VORTEX API. We have already reached out to the VORTEX team to address this matter.

2. 2026년 3월 25일 오전 10:49 UTCidentified

   VORTEX team is working on a fix for the issue.

3. 2026년 3월 25일 오후 2:13 UTCresolved

   This incident has been resolved.

1. 2026년 3월 16일 오후 8:05 UTCinvestigating

   We are currently investigating this issue. We are coordinating with VORTEX team to resolve this issue as soon as possible.

2. 2026년 3월 17일 오후 12:15 UTCmonitoring

   The video exports from VORTEX cameras are working as expected now.

3. 2026년 3월 17일 오후 7:30 UTCresolved

   This incident has been resolved.

1. 2026년 3월 12일 오전 11:29 UTCinvestigating

   We are currently investigating the issue. It appears to be related to the VORTEX API. We have already reached out to the VORTEX team to address this matter.

2. 2026년 3월 12일 오후 3:33 UTCmonitoring

   The video exports are functioning as intended. We will continue to monitor the situation closely.

3. 2026년 3월 13일 오전 11:43 UTCresolved

   This incident has been resolved.

1. 2025년 10월 30일 오후 5:26 UTCresolved

   Between 13:10 and 16:50 UTC, customers may have experienced a delay in synchronization of reader, controller and door configurations from the cloud to the devices. The issue has since been resolved. All device configurations are now up-to-date.

1. 2025년 6월 26일 오전 10:55 UTCresolved

   On June 16, 2025, the Kisi production API experienced a temporary degradation in availability. Investigation revealed that the root cause was an unexpected behavior in Google Cloud Platform’s (GCP) Cloud Run during an internal stack migration and service redistribution process managed by GCP. According to GCP’s engineering response, a background migration and redistribution of workloads triggered a one-time instance restart in our Cloud Run service. This resulted in the application falling below its configured minimum instances temporarily, impacting our ability to serve requests reliably. While the redistribution process is part of GCP’s infrastructure optimization efforts, the side effect of under-scaling was not communicated in advance and not within our control. GCP has confirmed that a fix has been rolled out to address this issue and prevent recurrence in similar scenarios.

1. 2025년 6월 16일 오후 5:58 UTCinvestigating

   We’re currently investigating an increase in 5xx and 4xx errors affecting our public API. This may impact door unlocks, as well as access to the web and mobile dashboards. We’ll share more updates as we identify the root cause.

2. 2025년 6월 16일 오후 6:29 UTCinvestigating

   We are continuing to investigate this issue.

3. 2025년 6월 16일 오후 6:39 UTCmonitoring

   A fix has been implemented and we are monitoring the results.

4. 2025년 6월 16일 오후 6:49 UTCresolved

   The incident has been resolved. It lasted for approximately 30 minutes, from 17:15 to 17:45 UTC.

1. 2025년 6월 12일 오후 6:30 UTCinvestigating

   Google Cloud Platform is experiencing an outage. Unlocks are not affected, but there may be delays for event related processing (webhooks, event history, ...).

2. 2025년 6월 12일 오후 8:57 UTCresolved

   Google Cloud Platform has restored service, and all of our affected systems are operating normally again.

1. 2025년 6월 9일 오전 11:27 UTCresolved

   The Event Service—Kisi’s core system for capturing and forwarding events—experienced a degradation after a recent feature deployment. An unclosed-connection bug gradually exhausted CPU and memory resources, throttling database writes. Our monitoring detected elevated error rates. A redeployment addressed the bug, but due to a large backlog some delayed event webhooks were unfortunately dropped. However, no events were lost. Service Impact: - Event visibility in the Kisi Admin Dashboard and via API was degraded during the degradation window (new events were not yet persisted). - Event Webhook deliveries and report generation were paused until full recovery. Next Steps: - We will adapt our system to process all event webhooks and event related jobs regardless of backlog size - We will isolate our event pipeline to avoid spillover effects from new functionality

1. 2024년 10월 22일 오후 1:35 UTCresolved

   The most recent Kisi API release caused elevated error rates. The release has been rolled back and API error rates are back to normal. Unlocks were affected during this time.

1. 2024년 5월 31일 오전 8:33 UTCidentified

   An issue is ongoing causing email delays for all customers. We have identified our email delivery service as the cause of this incident. See https://status.mailgun.com/incidents/30w5l1zqcw5r Currently, a majority of our outgoing emails are affected.

2. 2024년 5월 31일 오전 9:01 UTCmonitoring

   A fix has been implemented by our email delivery service provider and we are monitoring results. As of 08:30 UTC a majority of emails are being delivered.

3. 2024년 5월 31일 오전 9:59 UTCresolved

   Fix implemented by our email service provider seems to have fixed the issue. All our outgoing emails have been delivered since 08:30 UTC.

1. 2024년 5월 27일 오후 1:32 UTCidentified

   We are investigating an issue causing email delays for all customers. The incident started 13:10 UTC, and is ongoing.

2. 2024년 5월 27일 오후 1:38 UTCidentified

   We have identified our email delivery service as the cause of this incident. See https://status.mailgun.com/incidents/8bn2px3nyfy4 Currently, a majority of our outgoing emails are affected.

3. 2024년 5월 27일 오후 1:55 UTCmonitoring

   A majority of emails are being delivered as of 13:52 UTC. We will continue to monitor the issue.

4. 2024년 5월 27일 오후 2:47 UTCresolved

   Our email delivery provider has implemented a fix and is monitoring the results. Emails are being delivered successfully since 13:52 UTC.

1. 2024년 4월 11일 오후 2:22 UTCidentified

   The most recent Kisi API release caused elevated error rates. We are rolling back and expect the issue to be resolved in a minute or two. Unlocks are affected.

2. 2024년 4월 11일 오후 2:23 UTCresolved

   The release has been rolled back. Unlock failure rates and API error rates are back to normal.

1. 2024년 3월 25일 오전 12:15 UTCinvestigating

   The rate of failed Cisco Meraki camera snapshots is higher than normal, we are investigating the issue. We will provide an update by Monday, 2023-04-25 09:00 UTC with current details.

2. 2024년 3월 25일 오전 10:15 UTCresolved

   Cisco Meraki camera snapshots are working normally. The elevated error rates could be attributed to a single Kisi organization.

1. 2024년 3월 7일 오후 2:13 UTCresolved

   We experienced a short (less than 10 minutes) API outage. As a result, unlocks via cloud were failing and web dashboard was unavailable. Offline cache unlocks (https://docs.kisi.io/concepts/offline_support) were working as usual. The issue has now been resolved and we are doing everything to make sure it will not happen again.

1. 2024년 1월 25일 오후 3:11 UTCmonitoring

   We discovered an issue that caused in-app unlocks to fail for a small group of users that belong to groups with restrictions between 2024-01-24 11:12 UTC and 2024-01-25 14:19 UTC. Although we have resolved the underlying problem causing unlock failures, there was a misreporting of these failures as successes in the unlock events. We are actively addressing and correcting the affected events.

2. 2024년 1월 26일 오전 10:48 UTCmonitoring

   We are currently correcting the affected unlock events.

3. 2024년 1월 26일 오후 2:53 UTCresolved

   All incorrect unlock events have been fixed.