Ιστορικό διακοπών του Treasure Data

1. 27 Ιουλίου 2026 στις 8:29 π.μ. UTCresolved

   Μεταξύ 7/25 και σήμερα, 7/27 στις 11:30 π.μ. JST, βιώσαμε ένα πρόβλημα υποβάθμισης της απόδοσης που επηρεάζει BulkLoad (εισαγωγή Συνδετών Δεδομένων), εξαγωγή αποτελεσμάτων (δραστηριοποίηση) και μαζική εισαγωγή. Μετά από αναφορές που ελήφθησαν σήμερα το πρωί, η ομάδα μηχανικών μας ερεύνησε το θέμα και προσδιόρισε το υψηλό φορτίο του συστήματος υπό συγκεκριμένες συνθήκες (αυξημένη καθυστέρηση στην εσωτερική επικοινωνία συστατικών) ως τη βασική αιτία. Το θέμα έχει επιλυθεί με πλήρη περιστροφή των κόμβων server μας, και η πλήρης απόδοση αποκαταστάθηκε από 11:30 AM JST. Ζητάμε ειλικρινά συγγνώμη για την ταλαιπωρία που προκλήθηκε. Αυτή τη στιγμή επανεξετάζουμε τους κανόνες παρακολούθησης και προειδοποίησης για την πρόληψη παρόμοιων ζητημάτων στο μέλλον.

Αυτόματη μετάφραση από την επίσημη ενημέρωση του συμβάντος.

1. 7 Ιουλίου 2026 στις 5:04 μ.μ. UTCinvestigating

   Παρατηρούμε υποβαθμισμένες επιδόσεις στο Treasure Work και tdx στην περιοχή του Τόκιο. Το ζήτημα αυτό περιορίζεται στο μοντέλο Claude Haiku 4.5 μόνο στην περιοχή του Τόκιο. Οι χρήστες μπορούν να παρατηρήσουν ότι αιτήματα χρησιμοποιώντας Claude Haiku 4.5 λάθη επιστροφής ή time out. Αυτή τη στιγμή, δεν βλέπουμε κανένα πρόβλημα με άλλα μοντέλα. Ως εργασία, οι χρήστες μπορούν να επιλέξουν ένα διαφορετικό μοντέλο. Ερευνούμε ενεργά αυτό το ζήτημα και θα στοχεύουμε να παρέχουμε ενημέρωση σε 30 λεπτά.

2. 7 Ιουλίου 2026 στις 5:40 μ.μ. UTCinvestigating

   Συνεχίζουμε να ερευνούμε αυτό το ζήτημα και θα παρέχουμε νέα ενημέρωση σε 30 λεπτά.

3. 7 Ιουλίου 2026 στις 6:07 μ.μ. UTCidentified

   Συνεχίζουμε να διερευνούμε αυτό το ζήτημα και αξιολογούμε επί του παρόντος μια πιθανή λύση. Θα ενημερωθούμε μόλις ολοκληρωθούν οι δοκιμές.

4. 7 Ιουλίου 2026 στις 7:42 μ.μ. UTCidentified

   Συνεχίζουμε να εργαζόμαστε μέσω ενός δυνητικού επιδιόρθωσης και εργασίας για τον εντοπισμό των πιθανών παραγόντων που συμβάλλουν. Και πάλι, καθώς το ζήτημα εντοπίζεται σε μοντέλα Haiku 4.5 στην περιοχή του Τόκιο, συνιστούμε στους χρήστες να χρησιμοποιήσουν προσωρινά ένα διαφορετικό μοντέλο αν βιώνουν υψηλά ποσοστά σφάλματος. Ζητάμε συγγνώμη για την ταλαιπωρία και σύντομα θα έχουμε νέα ενημέρωση.

5. 7 Ιουλίου 2026 στις 8:27 μ.μ. UTCmonitoring

   Εφαρμόστηκε μια ρύθμιση και παρακολουθούμε τα αποτελέσματα.

6. 7 Ιουλίου 2026 στις 8:59 μ.μ. UTCresolved

   Το θέμα αυτό έχει επιλυθεί. Treasure Work and tdx λειτουργούν κανονικά, και το μοντέλο Claude Haiku 4.5 στην περιοχή του Τόκιο δεν βιώνει πλέον λάθη ή διαλείμματα. Σας ευχαριστώ για την υπομονή σας.

Αυτόματη μετάφραση από την επίσημη ενημέρωση του συμβάντος.

1. 26 Ιουνίου 2026 στις 9:48 μ.μ. UTCidentified

   Βιώνουμε την υποβάθμιση της απόδοσης των ερωτημάτων Hive στην περιοχή των ΗΠΑ. Το τεύχος αυτό ξεκίνησε στις 25 Ιουνίου 2026. Οι χρήστες μπορούν να παρατηρούν περισσότερο από τους αναμενόμενους χρόνους εκτέλεσης των ερωτημάτων, ιδίως για μακροχρόνιες θέσεις εργασίας με μεγάλο όγκο δεδομένων. Αναγνωρίσαμε την αιτία και δουλεύουμε πάνω σε μια λύση. Θα σας ενημερώσουμε σε 30 λεπτά.

2. 26 Ιουνίου 2026 στις 10:22 μ.μ. UTCidentified

   Βιώνουμε την υποβάθμιση της απόδοσης των ερωτημάτων Hive στην περιοχή των ΗΠΑ. Το τεύχος αυτό ξεκίνησε στις 25 Ιουνίου 2026. Οι χρήστες μπορούν να παρατηρούν περισσότερο από τους αναμενόμενους χρόνους εκτέλεσης των ερωτημάτων, ιδίως για μακροχρόνιες θέσεις εργασίας με μεγάλο όγκο δεδομένων. Αναγνωρίσαμε την αιτία. Για να το αντιμετωπίσουμε αυτό, επικυρώνουμε μια αλλαγή διαμόρφωσης συστάδων πριν την εφαρμογή της στο περιβάλλον παραγωγής. Θα σας ενημερώσουμε σε 30 λεπτά.

3. 26 Ιουνίου 2026 στις 10:57 μ.μ. UTCidentified

   Συνεχίζουμε να δουλεύουμε πάνω σε ένα σημείο. Η ομάδα μας επικυρώνει μια αλλαγή διαμόρφωσης συστάδων πριν την εφαρμογή της στο περιβάλλον παραγωγής. Θα σας ενημερώσουμε σε 30 λεπτά.

4. 26 Ιουνίου 2026 στις 11:57 μ.μ. UTCmonitoring

   Έχουμε παράσχει μια νέα ομάδα με επαρκή ικανότητα και τώρα δρομολογούμε θέσεις εργασίας σε αυτήν. Η απόδοση του ερωτήματος αναμένεται να ανακάμψει σταδιακά καθώς το σμήνος σταθεροποιείται. Παρακολουθούμε στενά την κατάσταση. Θα σας ενημερώσουμε σε 1 ώρα.

5. 27 Ιουνίου 2026 στις 1:13 π.μ. UTCresolved

   Η υποβάθμιση της απόδοσης στην περιοχή των ΗΠΑ έχει επιλυθεί. Έχουμε προμηθεύσει με επιτυχία μια νέα ομάδα και δρομολογήσαμε όλη την κυκλοφορία προς αυτήν. Τα συστήματα λειτουργούν κανονικά. Σας ευχαριστώ για την υπομονή σας.

Αυτόματη μετάφραση από την επίσημη ενημέρωση του συμβάντος.

1. 25 Ιουνίου 2026 στις 11:46 π.μ. UTCresolved

   Επίλυση — Ορισμένες θέσεις εργασίας εξαγωγής (εξαγωγή δεδομένων) στις ΗΠΑ, το Τόκιο και τις περιφέρειες της ΕΕ μπορεί να έχουν αποτύχει ή να έχουν καθυστερήσει μεταξύ 24 Ιουνίου 2026 21:00 UTC και 25 Ιουνίου 2026 05:26 UTC. Κατά τη διάρκεια αυτής της περιόδου, υποκείμενα υπολογιστικοί κόμβοι ήταν διαλείπτως ασταθής, η οποία θα μπορούσε να προκαλέσει τις εξαγωγικές θέσεις εργασίας να τερματιστούν πριν την ολοκλήρωση. Σε ορισμένες περιπτώσεις μια επηρεασμένη εργασία μπορεί να έχει τελειώσει χωρίς ένα σαφές μήνυμα σφάλματος. Το θέμα έχει επιλυθεί και όλα τα συστήματα λειτουργούν κανονικά. Η ακεραιότητα των δεδομένων δεν επηρεάζεται — τα δεδομένα των πηγών στα δεδομένα του θησαυρού δεν επηρεάστηκαν· μόνο η παράδοση των εξαγωγικών θέσεων εργασίας σε εξωτερικούς προορισμούς κατά τη διάρκεια αυτού του παραθύρου θα μπορούσε να έχει διαταραχθεί. Αν τρέξατε μια εξαγωγή αποτελεσμάτων/στοιχείων κατά τη διάρκεια αυτού του παραθύρου, σας συνιστούμε να επιβεβαιώσετε ότι ολοκληρώθηκε και να το εκτελέσετε εκ νέου εάν η έξοδος φαίνεται ελλιπής. Επίσης θα επικοινωνήσουμε άμεσα με τους πελάτες που επηρεάζονται. Σας ευχαριστώ για την υπομονή σας.

Αυτόματη μετάφραση από την επίσημη ενημέρωση του συμβάντος.

1. 24 Ιουνίου 2026 στις 11:11 π.μ. UTCresolved

   Εντοπίσαμε μια καθυστέρηση στην καταγραφή δεδομένων γεγονότων ηλεκτρονικού ταχυδρομείου (όπως το ανοικτό, κλικ, αναπήδηση, τα γεγονότα παράδοσης, και τα γεγονότα λάθους) στους πίνακες εκδηλώσεων πελατών. Επηρεαζόμενη περίοδος (κατά προσέγγιση): - 22 Ιουνίου 03:45 UTC – 23 Ιουνίου 04:15 UTC - 23 Ιουνίου 07:00 UTC – 24 Ιουνίου 07:30 UTC Κατά τη διάρκεια αυτών των περιόδων, η καταγραφή των δεδομένων των γεγονότων ηλεκτρονικού ταχυδρομείου καθυστέρησε. Η ίδια η αποστολή ηλεκτρονικού ταχυδρομείου δεν επηρεάστηκε — τα μηνύματα εστάλησαν και παραλήφθηκαν κανονικά. Το θέμα έχει επιλυθεί και όλα τα συστήματα λειτουργούν κανονικά. Καθυστερημένα γεγονότα έχουν υποβληθεί σε πλήρη επεξεργασία. Αν τρέξατε ερωτήματα σε πίνακες event ή error event κατά τη διάρκεια των επηρεαζόμενων περιόδων, προτείνουμε την επανάληψη αυτών των ερωτήσεων για να πάρετε ακριβή αποτελέσματα. Ζητάμε συγγνώμη για την ταλαιπωρία.

Αυτόματη μετάφραση από την επίσημη ενημέρωση του συμβάντος.

1. 5 Μαΐου 2026 στις 4:20 μ.μ. UTCidentified

   Incident: Real-time Profile Deletion Workflow Failures Status: Identified Impacted Regions: EU01, US01, Tokyo, AP03 (USA) Update (May 5, 2026 - 11:19 AM CDT) We have identified the root cause of the failures affecting Real-time (RT) bulk profile deletion, load, and stitch workflows in the affected regions. Summary of the Issue: A backend configuration issue resulted in a loss of necessary permissions for the service responsible for processing Real-time bulk profile operations. As a result, impacted workflows are failing with an "Authorization" or "400" error when attempting to invoke internal functions. Current Status: Our engineering team has developed a fix and is currently in the process of deploying it to all affected regions. We expect service to be restored shortly after the deployment is completed. We will provide a further update once the fix has been fully deployed and we have confirmed that workflow processing has returned to normal.

2. 5 Μαΐου 2026 στις 4:23 μ.μ. UTCidentified

   We are continuing to work on a fix for this issue.

3. 5 Μαΐου 2026 στις 5:43 μ.μ. UTCresolved

   We have applied the fix, and are observing that traffic is flowing correctly now.

1. 27 Απριλίου 2026 στις 5:31 μ.μ. UTCmonitoring

   Between 8:05 AM and 8:30 AM PDT, customers operating in the EU01 (Europe) region may have experienced intermittent errors (5xx) affecting two primary areas: Data Ingestion: Errors when attempting to send records via the Ingest API. Real-Time Personalization: Request failures for personalization and profile lookup services. The disruption was caused by an service interruption with an upstream provider in our data processing layer within the eu-central-1 region. This prevented our systems from successfully persisting incoming data to our streaming infrastructure and executing real-time lookups during the affected window. The upstream provider has confirmed the issue is resolved. Our engineering team monitored the recovery, and we have confirmed that all services in the EU01 region have returned to normal operating parameters. Data Integrity: For data ingestion, records that were rejected with a retryable error code during this window should be resubmitted. Records that have already been successfully ingested following the recovery require no further action. Personalization: Real-time request success rates have returned to 100%. We are continuing to monitor the stability of the region. We apologize for any inconvenience this may have caused to your operations. For further questions, please contact our support team at [email protected].

2. 27 Απριλίου 2026 στις 6:25 μ.μ. UTCresolved

   The upstream provider has restored service.

1. 18 Φεβρουαρίου 2026 στις 8:45 μ.μ. UTCinvestigating

   We are investigating a possible problem currently affecting Treasure Insights. We will provide an update as soon as we know more.

2. 18 Φεβρουαρίου 2026 στις 9:27 μ.μ. UTCidentified

   We have partially resolved the incident. We are continuously working to resolve the incident.

3. 18 Φεβρουαρίου 2026 στις 9:32 μ.μ. UTCmonitoring

   The performance went back to normal, and no errors are observed now. We will monitor at least for 15 minutes.

4. 18 Φεβρουαρίου 2026 στις 10:00 μ.μ. UTCresolved

   This incident has been resolved.

1. 21 Ιανουαρίου 2026 στις 2:55 π.μ. UTCinvestigating

   Starting at approximately 17:00 PST, we observed increased latency affecting all operations accessing table resources in Plazma. At around 19:00 PST, we applied temporary tuning to the database managing table resources and confirmed recovery based on internal performance metrics. We are currently continuing to monitor the system to ensure overall performance remains stable. Visible impacts are: - Streaming, Mobile, and JavaScript/Browser imports may delay - Jobs execution may delay - Table creation may error - Console execution may delay - Plazma Public API may return errors - Treasure Workflow may delay and fail - Workflow REST API may be unavailable - Workflow operations in Console and CLI may become unavailable - Presto JDBC/ODBC queries and CDP segmentation queries may fail - Console Table preview update may delay - The REST API to submit and cancel jobs may be unavailable - Data Connector Integrations may become unavailable - ADH (Ads Data Hub) and DCR (Data Clean Room) service may be unavailable - ADL (Active Data Layer) service may be unavailable We will send an additional update in 30 minutes

2. 21 Ιανουαρίου 2026 στις 3:37 π.μ. UTCidentified

   We identified the same issue in the Tokyo region and applied database parameter tuning. We are currently assessing the impact of this change on system performance. As a result of this temporary mitigation applied in both the US and Tokyo regions, the following impact is observed: In Data Workbench, updates to table metadata for row counts and preview data are temporarily paused. We will continue monitoring and provide further updates as we make progress.

3. 21 Ιανουαρίου 2026 στις 3:40 π.μ. UTCidentified

   We are continuing to work on a fix for this issue.

4. 21 Ιανουαρίου 2026 στις 4:06 π.μ. UTCidentified

   Performance in the Tokyo region improved at approximately 12:50 JST. The Tokyo region had been experiencing gradually increasing performance degradation since around 09:00 JST. We are currently proceeding with the steps to resume updates of table metadata in Data Workbench. Due to the temporary pause of metadata updates, application metrics that rely on this information, such as Parent Segment size, have also been temporarily not updated. We will continue monitoring the system and share further updates as needed.

5. 21 Ιανουαρίου 2026 στις 4:12 π.μ. UTCidentified

   In the US region, table metadata updates have been resumed. At this point, the system in the US region is expected to be fully recovered, and we are continuing to monitor it closely. For the Tokyo region, metadata updates will be resumed after we complete confirmation of system performance. We will provide another update when this step begins.

6. 21 Ιανουαρίου 2026 στις 4:41 π.μ. UTCmonitoring

   The system in the US region is operating normally with no observed issues. In the Tokyo region, table metadata updates have also been resumed. At this point, the Tokyo region is expected to be fully recovered, and we will continue to monitor system performance. Once stability is fully confirmed, we plan to close this status page.

7. 21 Ιανουαρίου 2026 στις 5:07 π.μ. UTCresolved

   Incident Impact Summary (Plazma / Table Resources) We have confirmed that the system in the Tokyo region is operating normally with no remaining issues. What happened - All operations accessing Plazma table resources experienced increased latency. When - US region: 17:00 PST – 18:40 PST - Tokyo region: 09:00 JST – 12:50 JST Temporary impact during recovery - Updates to table row counts and preview data were temporarily paused. Recovery status - US region: Table metadata updates resumed at 20:00 PST. - Tokyo region: Table metadata updates resumed at 13:40 JST. Both regions are now fully recovered. Data integrity - No data loss or data corruption occurred. We will prepare and publish a post-mortem document to explain the circumstances under which this incident occurred one day after the recent system maintenance. With this confirmation, the incident is now closed.

8. 26 Ιανουαρίου 2026 στις 6:36 μ.μ. UTCpostmortem

   ## Post-Mortem Summary This incident was related to a database maintenance performed on January 19, during which management of Plazma table resources was migrated from the existing RDBMS to a dedicated Aurora database. The purpose of this change was to better isolate table resource workloads and improve long-term scalability. The migration itself completed successfully, and both the original database and the newly introduced Aurora database were operating normally immediately after the maintenance. On the following day, when internal batch processes began running, the newly introduced database experienced unexpected load. Investigation determined that the internal batch system was configured to connect to the master database endpoint instead of a reader endpoint. As a result, internal batch processing competed with user-facing requests, causing increased latency when accessing table resources. To mitigate the impact and enable rapid recovery, temporary database tuning was applied. This tuning was intended solely as a short-term measure to stabilize the system and has since been fully reverted. The permanent fix consisted of correcting the configuration of the internal batch system so that it accesses the appropriate database endpoint. Following these actions, system performance recovered in both the US and Tokyo regions, and all services returned to normal operation. No data loss or data corruption occurred. To prevent similar issues in the future, we are reviewing our configuration and deployment practices, including reducing configuration differences between staging and production environments and strengthening validation of database endpoints used by internal workloads.

1. 10 Δεκεμβρίου 2025 στις 6:03 π.μ. UTCresolved

   An unexpected behavior occurred in the REST API "GET /v3/job/result/{job_id}" which caused it to return only a partial set of results. This issue stemmed from an update to the job result download mechanism implemented around 2025/12/09 01:00 (UTC). The issue was resolved after a fix was applied around 2025/12/10 04:36(UTC). The affected API is used through the following methods: - Directly calling the REST API - Using functions in libraries (e.g., pytd) that reference job results - Toolbelt commands that refers to job results - Setting store_last_results:true in the td>:/td_run> operator in Treasure Workflows - Using the td_for_each>, td_wait> and td_wait_table>: operators in Treasure Workflows Unfortunately, we are unable to identify which API requests were affected. Please compare the number of results retrieved by your job with the download results to see if you were affected. We sincerely apologize for the inconvenience this has caused.

1. 5 Δεκεμβρίου 2025 στις 9:18 π.μ. UTCinvestigating

   We are currently experiencing issues accessing Treasure Insights. Our investigation has identified that this is caused by an outage with a third-party service provider utilized by Treasure Insights. We are monitoring the situation and working with the provider to resolve the issue. We will provide updates as soon as more information becomes available.

2. 5 Δεκεμβρίου 2025 στις 9:22 π.μ. UTCmonitoring

   The third-party service provider has implemented a fix. We are monitoring the results.

3. 5 Δεκεμβρίου 2025 στις 9:30 π.μ. UTCresolved

   We confirmed the third-party service is now operational. The incident has been resolved.

1. 5 Νοεμβρίου 2025 στις 3:49 π.μ. UTCinvestigating

   We have encountered a temporary issue with the availability of Premium Audit Logs for Artificial Intelligence services (AI Agent Foundry and Audience Agent) during the period of November 4th 23:19:26 UTC to November 5th 00:57:55 UTC. While some data during this period may not have been captured, our engineering team is actively investigating the situation. We are taking all necessary steps to restore full service and ensure that all future logs are recorded without issue. We apologize for any inconvenience this may have caused and will keep you updated on our progress.

2. 5 Νοεμβρίου 2025 στις 3:55 π.μ. UTCmonitoring

   A fix has been implemented and we are monitoring the results.

3. 5 Νοεμβρίου 2025 στις 7:58 π.μ. UTCresolved

   Services have been restored to normal operations, and all of our systems are reporting healthy once again. If you observe any lingering issues, please connect with our support team, and we would be happy to help: https://docs.treasuredata.com/articles/#!pd/contacting-treasure-data-support

1. 20 Οκτωβρίου 2025 στις 8:53 π.μ. UTCinvestigating

   We are aware of an issue affecting log writing and reading for the Profile API in all regions. This is due to an incident with an upstream service. Our team is monitoring the provider's status. We will provide an update shortly.

2. 20 Οκτωβρίου 2025 στις 10:15 π.μ. UTCinvestigating

   We are also investigating reports indicating that this incident might be causing login errors for customers using SSO (Single Sign-On) to log into our service. Customers attempting a new login via SSO may be unable to access the service. We believe users who are already logged in (have an active session) and users logging in with methods other than SSO (e.g., username/password) are not affected by this login issue. We are working to confirm this.

3. 20 Οκτωβρίου 2025 στις 3:09 μ.μ. UTCinvestigating

   Our upstream service provider is gradually recovering its services, and our services are gradually recovering too. SSO login is working fine, however profile API 1.0 is still recovering its service with latency. As the upstream service provider incident is still ongoing, you might still encounter unexpected errors or issues. We will closely monitoring our services and work with our upstream service provider.

4. 20 Οκτωβρίου 2025 στις 6:53 μ.μ. UTCidentified

   We are continuing to work with our service provider to track this issue. Customers may continue to observe higher latency than normal and may notice delays in processing. We are monitoring the situation closely and are adding additional capacity in anticipation of a recovery. As this situation develops, we will keep you updated on any changes in expected behavior.

5. 20 Οκτωβρίου 2025 στις 9:32 μ.μ. UTCresolved

   Delays should be improving for all customers, and services should be returning to normal. Given the similarity of the two issues, we'll continue tracking any concerns related to this interruption as part of the "[US Region] Issue Affecting Treasure Data Services" issue. For further updates, please track here: https://status.treasuredata.com/incidents/w7v5wtmkdllj If you experience continued issues specifically relating to our Profiles API or SSO Logins, please contact our support team: https://docs.treasuredata.com/articles/#!pd/contacting-treasure-data-support

1. 20 Οκτωβρίου 2025 στις 8:27 π.μ. UTCinvestigating

   We are currently investigating an issue originating from an upstream service provider used by our infrastructure. This may be affecting the normal operation of some Treasure Data services. We are working to determine the scope of the impact. Observed symptoms may include, but are not limited to, the failure of various jobs (such as Trino, Hive, Result Export, Import, and Custom Script execution) to start. Our team is actively monitoring the situation and working with the provider. We will provide further updates as more information becomes available.

2. 20 Οκτωβρίου 2025 στις 3:05 μ.μ. UTCinvestigating

   Our upstream service provider is gradually recovering its services, and our services are gradually recovering too. Most of our Treasure Data services are back to normal except for Custom Script, autoML and realtime services such as profile API 1.0, and realtime 2.0. As the upstream service provider incident is still ongoing, you might still encounter unexpected errors or issues. We will closely monitoring our services and work with our upstream service provider.

3. 20 Οκτωβρίου 2025 στις 6:49 μ.μ. UTCinvestigating

   We are continuing to monitor ongoing recovery efforts, and most of our systems remain operating. As the situation with our upstream provider develops we will react if we discover any new issues. At this time, we are observing delays in our systems, including Profile API 1.0 and Realtime 2.0. Requests are being processed as they have been received, but users may see a slowdown in results, and may notice delays when updating configurations or journeys. You may notice continued slower performance or unexpected errors. We are working with our service provider to address these issues and will provide an update with new information soon.

4. 20 Οκτωβρίου 2025 στις 7:35 μ.μ. UTCmonitoring

   We are seeing continued recovery across our systems, and are currently working to process any buffered or delayed messages. Customers may continue to observe delays in data processing and may notice slower-than-usual processing of configuration changes. We expect these delays to decrease as our systems catch up. Our team is continuing to track recovery closely, and we'll inform you of any new developments.

5. 20 Οκτωβρίου 2025 στις 9:42 μ.μ. UTCmonitoring

   Service levels continue to improve as upstream services become more stable. Users may still see processing delays, though those should be decreasing as we process the backlog of messages from this event. We are continuing to monitor our systems and deploy capacity as needed. We will update you with additional information as it develops.

6. 20 Οκτωβρίου 2025 στις 11:15 μ.μ. UTCresolved

   Services have been restored to normal operations, and all of our systems are reporting healthy once again. If you observe any lingering issues, please connect with our support team, and we would be happy to help: https://docs.treasuredata.com/articles/#!pd/contacting-treasure-data-support

1. 15 Οκτωβρίου 2025 στις 6:16 μ.μ. UTCinvestigating

   We are currently investigating an issue where some customers are experiencing intermittent “query expired” errors when retrieving segment counts in Audience Studio. In many cases, queries may succeed after a retry, but this behavior can impact the ability to view or use segment results reliably. Our engineering teams are actively investigating the root cause and working to mitigate the issue. We will provide updates here as soon as more information becomes available. In the meantime, affected users may be able to work around the issue by retrying the query after a short interval.

2. 16 Οκτωβρίου 2025 στις 5:05 π.μ. UTCmonitoring

   We've identified that an internal component between our query engine which extracts the number of records that match Segment rules and Audience Studio ran into a capacity issue during a specific time. We have increased the capacity and will continue to monitor the situation. If the issue does not reoccur by approximately 2025/10/17 01:00 (+00:00), we will consider this incident resolved and close this.

3. 17 Οκτωβρίου 2025 στις 1:55 π.μ. UTCresolved

   After a successful monitoring period, we can confirm that our fix is working and the issue is now resolved. Thank you for your patience.

1. 15 Οκτωβρίου 2025 στις 2:05 π.μ. UTCinvestigating

   We are currently experiencing an issue with email delivery for Treasure Data system emails, such as password resets. The affected email deliveries are: * Default email notification of Treasure Workflow * Password reset * New user invitation * Password change * Email address change * User lock notifications The following emails are NOT affected: * Email delivery with the mail>: operator of Treasure Workflow * Email delivery with Engage Studio Our team is actively working to resolve this issue. We sincerely apologize for any inconvenience or disruption this may cause.

2. 15 Οκτωβρίου 2025 στις 2:52 π.μ. UTCidentified

   The issue has been identified and a fix is being implemented.

3. 15 Οκτωβρίου 2025 στις 4:38 π.μ. UTCmonitoring

   The issue has been addressed with an applied fix. We are currently monitoring the system to ensure the services are fully restored and stable. Thank you for your patience while we addressed this.

4. 15 Οκτωβρίου 2025 στις 5:06 π.μ. UTCresolved

   The issue has been fully resolved. Thank you for your patience.

1. 30 Σεπτεμβρίου 2025 στις 3:37 π.μ. UTCinvestigating

   We are currently investigating an issue where AI agents in the AI Agent Foundry may prematurely stop their response without providing a final conclusion, especially during their thinking process. This behavior does not generate an error message. Our initial findings show it is more likely to occur in agents handling long or extensive conversations, though we have not yet identified a specific threshold. Our engineering team is actively investigating the root cause. We will provide another update as soon as more information is available. We apologize for any inconvenience this may cause.

2. 30 Σεπτεμβρίου 2025 στις 6:08 π.μ. UTCidentified

   We have an update on the root cause of the issue causing AI agents to provide incomplete responses. Our investigation has found that when an agent uses its tools, the user's original prompt was being sent multiple times to the underlying AI model. This duplication caused the model to incorrectly process the tool results, leading it to return an empty response instead of a complete answer. The hotfix we are preparing will correct this behavior by ensuring the user's prompt is sent only once for each task. This will allow the AI to properly process all tool results and generate a complete, final response. We are working to deploy this fix as quickly as possible and will provide another update once it is ready. Thank you for your continued patience.

3. 30 Σεπτεμβρίου 2025 στις 6:53 π.μ. UTCmonitoring

   The hotfix for the incomplete agent responses has been deployed. This should prevent the duplicate prompt issue during tool use. We are now monitoring the platform to ensure everything is working as expected. We will resolve this incident once we've confirmed stability.

4. 30 Σεπτεμβρίου 2025 στις 7:22 π.μ. UTCresolved

   After a successful 30-minute monitoring period, we can confirm that our fix is working and the issue is now resolved. All AI agents are functioning as expected. Thank you for your patience.

1. 26 Σεπτεμβρίου 2025 στις 7:12 μ.μ. UTCidentified

   We are currently experiencing delays in our Profile API replication pipeline in the Tokyo region. This issue began at approximately 13:00 UTC and is affecting all customers in this region. Impact: - Profile API responses may return stale data (up to several hours behind) - Lookup functionality remains available but data may be inconsistent Our engineering team is actively working on recovery. We estimate normal service will be restored within 8-10 hours. We will continue to provide updates as we make progress toward resolution.

2. 26 Σεπτεμβρίου 2025 στις 11:02 μ.μ. UTCmonitoring

   A fix has been implemented and we are monitoring the results.

3. 26 Σεπτεμβρίου 2025 στις 11:24 μ.μ. UTCresolved

   This incident has been resolved.

1. 24 Σεπτεμβρίου 2025 στις 6:53 π.μ. UTCresolved

   We experienced a partial issue on September 24th, between 01:55 and 02:40 UTC, where some users may have encountered incorrect permission errors when accessing their tables and databases in Hive and Trino. The issue was caused by a high load on our table API, which temporarily struggled to retrieve correct permission data. All underlying data remained secure and unaffected. The API load has returned to normal, and all services are operating as expected. If you had any jobs that failed with a permission error during this period, please try running them again. We apologize for any confusion this may have caused.

1. 18 Αυγούστου 2025 στις 9:03 μ.μ. UTCinvestigating

   Treasure Data Insights is currently unavailable; we are investigating this issue.

2. 18 Αυγούστου 2025 στις 9:49 μ.μ. UTCresolved

   The issue was resolved; it was linked to a vendor issue. The vendor has resolved the problem that was impacting TD Insights.