Smarty 장애 기록

1. 2026년 9월 26일 오후 3:51 UTCmonitoring

   15:23 UTC에서 우리는 간헐적으로 사용할 수없는 일부 서비스의 조사 보고서를 시작했다. 서비스는 완전히 회복되고 모든 체계는 일반적으로 운영됩니다. 우리는 밀접하게 감시하고 뿌리 원인을 조사하기 위하여 계속됩니다.

2. 2026년 9월 26일 오후 4:49 UTCresolved

   사건이 해결되었습니다. 우리는 뿌리 원인을 발견하기 위해 더 철저한 조사를 수행하고 있습니다.

3. 2026년 9월 28일 오후 7:03 UTCpostmortem

   **공지 사항** 에 9월 26, 2026, 부터 15:28 에 15:39 UTC, 우리의 API에 요청은 sporadically 실패. 몇 분 동안 많은 요청이 잘못되었습니다. 15:30 UTC의 일부 용량을 간단히 회복했습니다. 무슨 일이 있었는지: 우리의 로드밸런서가 디스크 공간에서 ran에 의존하는 내부 시스템의 서버는 동료 노드에 제대로 실패하지 않았습니다. 그 시스템은 응답을 중단 할 때, 지역 전역의 다양한 부하 잔고가 자동으로 재시작되었습니다. 우리의 엔지니어는 15:37 UTC에서 건강 한 서버에 영향을 미쳤습니다. 서비스는 15:39 UTC에 의해 완전히 복원되었습니다. 우리가 하는 것은: 우리는 디스크 용량을 증가 하 고 영향을 받는 시스템에 로그 유지를 변경 그래서 그것은이 방법을 다시 실행할 수 없습니다. 우리는 또한 그 체계가 사용할 수 없을 때 우리의 로드밸런서가 어떻게 동작하는지, 그래서 그들은 차단 대신 트래픽을 유지하고, 우리는 당신을 영향을하기 전에 문제를 잡는 경고를 추가하고 있습니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 9월 10일 오후 8:14 UTCinvestigating

   우리는 현재 로그인 문제를 조사하고 곧 해결 희망입니다. 제품 API는 unaffected와 완전히 가동입니다.

2. 2026년 9월 10일 오후 8:18 UTCinvestigating

   이 문제를 조사하고 있습니다.

3. 2026년 9월 10일 오후 10:37 UTCresolved

   문제는 해결되었습니다. 영향을받은 유일한 서비스는 계정 관리 포털이었습니다. 이 사건은 우리의 APIs (주소 확인, autocomplete, geocoding, 등)에 생산 교통에 영향을 미치지 않았습니다

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 8월 19일 오전 12:49 UTCinvestigating

   미국 스트리트에 대한 503 오류의 작은 숫자 [주소] API는 몇 가지 요청에 영향을 미칩니다. 현재 조사 중입니다.

2. 2026년 8월 19일 오후 9:13 UTCresolved

   이 사건은 해결되었습니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 7월 21일 오후 10:30 UTCidentified

   19:00 UTC부터, 예기치 않은 입력은 US Street Address API에서 503 응답 코드를 반환하는 소량의 요청으로 표시됩니다. 해결은 현재 진행 중이며 업데이트가 완료되면이 페이지에 게시됩니다.

2. 2026년 7월 21일 오후 10:33 UTCidentified

   이 문제를 해결하기 위해 계속 노력하고 있습니다.

3. 2026년 7월 21일 오후 10:56 UTCmonitoring

   수정이 실행되고 함대에 굴러있다. 우리는 결과를 감시하고 있습니다.

4. 2026년 7월 21일 오후 11:10 UTCmonitoring

   롤아웃이 완료됩니다. 우리는 감시자를 계속합니다.

5. 2026년 7월 22일 오전 12:34 UTCresolved

   이 사건은 해결되었습니다.

공식 인시던트 업데이트를 자동 번역했습니다.

1. 2026년 6월 3일 오후 8:17 UTCresolved

   SUMMARY We are rotating the Let's Encrypt intermediate certificates used to issue the TLS certificates that secure our services. Our certificates will continue to chain to the same trusted root — ISRG Root X1 — but through Let's Encrypt's new "Generation Y" intermediate hierarchy. For the vast majority of customers, no action is required. If your systems validate our certificates against the public root store — the default for virtually all browsers, operating systems, and HTTP client libraries — this change is completely transparent. The only customers who may be affected are those who pin a specific Let's Encrypt intermediate certificate. WHY WE'RE MAKING THIS CHANGE The intermediate certificates currently in our chain are approaching the end of their lifecycle, with the existing intermediate path expiring in approximately nine months. Rotating well ahead of that deadline guarantees uninterrupted certificate issuance and renewal with no disruption to your integrations. THE CHAIN OF TRUST A TLS certificate is never validated on its own. It is verified through a chain that links the connection back to a root certificate your device already trusts. Today, our certificates chain like this: Your connection → Smarty leaf certificate → Let's Encrypt intermediate → ISRG Root X1 After this update, they will chain like this: Your connection → Smarty leaf certificate → Let's Encrypt Generation Y intermediate (YR / YE) → ISRG Root YR / YE → ISRG Root X1 (via cross-sign) The destination is unchanged. Both the old and new paths terminate at ISRG Root X1, the RSA root that ships in every current major trust store. Let's Encrypt cross-signed its new Generation Y roots with the existing X1 and X2 roots specifically so that anything already trusting X1 continues to work without modification. The path is longer; the anchor of trust is identical.

1. 2026년 4월 24일 오후 5:39 UTCresolved

   At 00:00 UTC on April 17, A scheduled certificate renewal updated the CA chain served by api.smartystreets.com. The change was made without advance notice to integrators, and the new chain is anchored by Sectigo roots (USERTrust ECC Certification Authority and USERTrust RSA Certification Authority) that are not present in some older trust stores. Clients on current operating systems, browsers, and runtimes continue to work without change. Connections may fail for systems that: - Use a custom or pinned TLS trust bundle that does not include USERTrust ECC Certification Authority. - Run Java 8 installations predating update 51 (mid-2015) or earlier Java versions — these do not ship USERTrust ECC Certification Authority in the default cacerts keystore. - Run other environments with outdated trust stores, including older Android devices, legacy embedded and IoT hardware, and air-gapped systems. To restore connectivity, affected systems should either update to a current runtime or add the Sectigo roots (USERTrust ECC Certification Authority and USERTrust RSA Certification Authority) to their trust store. Please reach out to support for assistance.

1. 2026년 3월 16일 오후 10:35 UTCresolved

   Around 22:00 UTC the US [Address] Autocomplete Pro API service in one of our datacenters went offline due to a misconfiguration. The problem was quickly identified and rectified. All other datacenters were unaffected.

1. 2026년 3월 6일 오후 5:59 UTCinvestigating

   We are currently investigating an increase in 503 Service Unavailable errors. We will provide an update as soon as more information is available.

2. 2026년 3월 6일 오후 9:16 UTCmonitoring

   The issue has been identified and we are currently monitoring overall system health.

3. 2026년 3월 10일 오후 3:58 UTCresolved

   The issue has been resolved. A small number of nodes experienced large memory consumption which resulted in a few 503 errors being returned. This has been resolved and measures have been put in place to mitigate this in the future. Additional alerts and monitors have also been put in place.

1. 2026년 2월 18일 오전 12:40 UTCresolved

   Around 19:15 UTC an incorrect configuration was deploy to our US Extract API service. The problem was quickly detected and reverted.

1. 2025년 11월 27일 오전 12:32 UTCmonitoring

   We've identified an issue affecting one of our redundant facilities. As a precaution, it has been taken out of active rotation while we investigate.

2. 2025년 12월 1일 오후 5:07 UTCresolved

   This incident has been resolved.

1. 2025년 10월 30일 오후 10:36 UTCmonitoring

   During a routine data deployment, we noticed a brief spike in 5xx error responses. Systems have since stabilized, and we're investigating the root cause to confirm the issue is fully resolved and to prevent similar behavior in future deployments.

2. 2025년 11월 4일 오후 3:35 UTCresolved

   This incident has been resolved.

1. 2025년 7월 18일 오후 4:25 UTCresolved

   We have become aware that Google services have been exhibiting some intermittent failures. As a result, anyone using "Login with Google" to access their account dashboard on our website may also experience a failure to login. When Google is able to resolve the issues they are having, the login will continue to work as normal. In the meantime, if you have created a username and password, you should be able to use those to login.

1. 2025년 5월 15일 오후 2:32 UTCresolved

   We discovered that the "API Keys" page in our Account dashboard web site was not able to retrieve the lists of keys from the backend service. This has now been fixed.

1. 2025년 3월 18일 오후 10:11 UTCresolved

   Around 19:10 UTC, increased traffic exposed resource limitations caused by a previously implemented algorithm update, leading to issues on several servers hosting some of our services (US Street API, US Extract API) in one of our data centers due to insufficient RAM. To ensure stability and service reliability, the affected servers have been decommissioned and replaced with upgraded hardware.

1. 2025년 2월 26일 오후 8:31 UTCresolved

   The US Enrichment API intermittently returned HTTP 500 codes in a single data center on 2025 February 26 around 17:00 UTC. The issue has been resolved as of 17:35 UTC.

1. 2025년 2월 7일 오후 9:15 UTCresolved

   Around 18:00 UTC, one of our facilities experienced a network routing issue that lasted for about 30 minutes. The facility was removed from DNS rotation until the network issue had been resolved. If you have not yet read through our "Technical Requirements" and "Best Practices" documentation, this could be a good opportunity to become familiar with those important and helpful documents. - https://www.smarty.com/docs/cloud/requirements#retry - https://www.smarty.com/docs/cloud/best-practices

1. 2025년 1월 10일 오후 9:12 UTCresolved

   From 01/08 12pm UTC to 01/09 9am UTC some requests to the US Street [Address] API instances experienced an intermittent issue with certain malformed addresses that caused some requests to not get a response, returning HTTP 503 errors to those requests. This affected a small number of requests and the error has since been resolved.

1. 2024년 12월 30일 오후 10:32 UTCresolved

   During a regular upgrade of our control plane, several controller nodes became unresponsive. This resulted in various routing issues which were manifest by intermittent HTTP 503 errors visible in the HTTP response to some callers. Our monitoring systems alerted our Operations Engineering Team of the anomaly and, as a result, the affected cluster, located in our US Central region (Chicago), was immediately removed from production rotation. The affected cluster has been restored to full health by provisioning new cloud resources using our IaC (infrastructure as code) systems. Following this, the cluster was confirmed to be was healthy via our automated monitoring and was subsequently re-introduced into active rotation.

1. 2024년 12월 20일 오후 4:11 UTCresolved

   On 2024 December 19, around 22-24:00 UTC, some users may have received the HTTP 299 code in response to requests from one of our loadbalancers. The reason for this response has been identified and remediated. We would like to take this opportunity to remind our users to familiarize themselves with our best practices. https://www.smarty.com/docs/cloud/best-practices

1. 2024년 12월 9일 오후 7:45 UTCresolved

   Around 15:00 (3pm) UTC on 2024 November 28, a single IP address at one of our datacenters experienced a routing issue that caused some requests to be "null routed", which may have resulted in some clients experiencing 500 level errors during that time. This issue persisted until 17:00 (5pm) UTC on 2024 November 28, at which time our upstream provider resolved the issue. Our recommendation to clients to mitigate an issue such as this in the future is to maintain a pool of active TCP connections to the server nodes and to only send traffic over active, healthy TCP connections. Then a background "thread" (or programmatic equivalent) could attempt to connect to the suspect IPv4 address. Once that connection has determined to be healthy, regular traffic can be resumed over that TCP connection.