Database

Problem: The client is preparing for an iPhone launch event, anticipating traffic spikes up to 200%. They require proactive monitoring of their Apache Cassandra production system during specified timeframes, with an upgrade to Severity 1 for immediate response during those periods. Process: Ticket Acknowledgment: We confirmed availability for the requested support dates and asked for […]

Problem: The client reported encountering a request timeout error when querying the PLDT Cassandra database in a production environment. The specific query involved selecting records from the jesi.service_monitoring table, which was attached along with a screenshot for further context. Process: Upon receiving the issue, the support team initiated an investigation. They first inquired about the […]

Problem: The client wanted to perform a rolling upgrade of the underlying operating system from RHEL 7 to RHEL 9 for their ETCD nodes in a Patroni-managed PostgreSQL cluster. The cluster contained three ETCD nodes and three Patroni-managed PostgreSQL instances (one primary and two standby). With a Recovery Point Objective (RPO) and Recovery Time Objective […]

Problem: The client encountered a significant issue with their PostgreSQL database (PGDB). They reported that the filesystem (FS) utilization suddenly increased from 74% to 94% without any new objects being created. Despite their efforts to recreate the replication slot and restart PGPool, the filesystem remained at 94%. Logs revealed a termination error related to another […]

Problem: The client reported a replication issue in their A1 BG Production environment, consisting of a Patroni cluster with two PostgreSQL instances (Leader and Replica). Replication stopped, causing the leader’s /pgcluster file system to fill up with pg_wal files, leading to a full disk. The client requested help to identify the root cause of the […]

Problem: Following an upgrade from Cassandra 4.0.9 to 4.1.3, the client reported a noticeable increase in CPU utilization. The average CPU usage on their systems jumped from around 20% to approximately 37%. This escalation in CPU usage adversely impacted system performance and stability. The issue was notably more severe on servers running Red Hat Enterprise […]

Problem: After performing a manual failover in PostgreSQL, the client encountered the following error when running a query on a partitioned table ‘ac1_control’: ERROR: could not access the status of transaction 613182547; DETAIL: Could not open file ‘pg_xact/0248’: No such file or directory. Despite restarting the PostgreSQL instance, the issue persisted. The client was operating […]

Problem: The client faced an issue with Apache Cassandra where an error occurred while building indexes. The error message was: Unexpected error while building indexes: Could not find type for id: 12297. The client had also partially configured dual data centers, which added complexity to the problem. Process: Initial Identification: The expert determined that the […]

Problem: The client was experiencing issues with their Wiki.js application running on an AWS EC2 instance, utilizing PostgreSQL as the database and Docker for deployment. After a system crash, attempts to restart the application via the Docker Compose file resulted in errors. The client requested assistance in investigating the problem, stating that while the application […]