Database

Problem: The client reported high memory usage on a production Apache Cassandra node, accompanied by frequent errors related to the ThreadPoolExecutor shutting down. This led to instability in the Cassandra service, including errors like java.util.concurrent.RejectedExecutionException, and resulted in a failure to execute repairs. Process: Step 1: Initial Identification The error logs provided by the client […]

Problem: The client needed to upgrade their production Apache Cassandra system from version 3.11.7 to 4.1.0 to take advantage of new features and improvements. They requested guidance on upgrade procedures and a reliable source for downloading the required RPM. Zero downtime was a critical requirement to ensure uninterrupted operations during the upgrade process. Process: Step […]

Problem: The client was experiencing performance issues with a self-managed Cassandra database cluster hosted on Azure VMs. A recent surge in data traffic led to high CPU utilization, causing significant system slowness and increased latency. The environment utilized SSDs for storage, but earlier attempts at recommended SSD optimizations yielded no significant improvement. In light of […]

Problem: The client planned to migrate a 5-node Cassandra cluster from an on-premises environment (version 3.11.8) to AWS (target version 4.1.5). The client requested guidance on the best migration strategy that ensures no downtime. Additionally, the client requested information on backup and restore procedures for the migration. Solution: The expert recommended a step-by-step approach. First […]

Problem: The client experienced issues with uneven data distribution across nodes after adding new nodes to an existing Cassandra cluster. Upon reviewing the “nodetool status” output, it was observed that new nodes were not receiving equal data compared to existing ones, resulting in significant data discrepancies between nodes. The client sought assistance in understanding why […]

Problem: The client reported a critical issue with the Cassandra cluster after adding a new data center and a rack containing three nodes. Despite bringing the new data center online, no data was being transferred from the source data center. Additionally, attempts to run a repair operation on the nodes were unsuccessful, which prevented the […]

Problem: The client needed to add a new data center to their existing Cassandra DB cluster for a critical project. However, upon starting Cassandra on the new server, it encountered a shutdown error due to a required node being offline. The error message, “A node required to move the data consistently is down,” indicated an […]

Problem: The client is encountering a “SSTable corruption” issue when starting Cassandra in a new PLAB environment created using a CloudFormation template. After copying EBS volumes from a disaster recovery (DR) environment and making necessary adjustments in the cassandra.yaml file, they receive a series of NullPointerExceptions related to the SSTableReader while attempting to open SSTables. […]

Problem: The client experienced an authentication failure during health checks in their Pgpool-II setup, which led to a failover event. Despite updating the password in pool_passwd and pgpool.conf using the pg_md5 utility, the client continued to face the same issue. They observed that after failing over the node due to the authentication issue, they successfully […]