Database

Problem: The client faced performance challenges while running nodetool cleanup on an Apache Cassandra 4.1.5 cluster during a node addition activity in a production environment. Specifically, the cleanup process was taking an unexpectedly long time on nodes with over 600GiB of data load, raising concerns about the overall timeline and impact on production workflows. The […]

Problem: The client reported high CPU utilization (up to 90%) across all nodes in their 3-node Apache Cassandra 4.1.3 cluster during full or incremental repair operations initiated from any single node. Despite relatively low data volumes (~25 GB per node), the CPU spike raised concerns about system performance, stability, and potential downtime during repairs. Process: […]

Problem: The client reported a sudden and significant drop in Apache Cassandra performance on a 4-node cluster. The issue appeared without any recent configuration or infrastructure changes. The application started experiencing frequent timeouts, and restarting Cassandra services on all nodes did not resolve the problem. The client provided various monitoring graphs, system logs, and other […]

Problem: The client needed to scale their production Cassandra cluster from 6 nodes to 12 nodes (3 to 6 nodes per data center) without any downtime. Their existing setup includes Cassandra version 4.1.6, with two data centers (PROD and DR), each containing 3 nodes, forming a 6-node cluster with a replication factor of 3 and […]

Problem: The client needed to implement audit logging for their PostgreSQL 15 databases. Specifically, they wanted to track user actions such as: Configuration changes Creation, deletion, or modification of objects (documents, users, settings) Attempts to access forbidden resources Privilege escalation attempts Additionally, they requested that auditing be limited only to administrative users, not applied globally […]

Problem: The client was using Apache Cassandra 4.1.5 installed via a tarball extraction on an AWS EC2 machine and wanted to upgrade both their Cassandra version and the operating system. The installation was done manually using the tarball method, and the client needed to understand the feasibility and potential challenges involved in upgrading the OS […]

Problem: The client experienced a failure in running repairs in Apache Cassandra due to corruption in hint files. Additionally, a node in the cluster went down and could not be brought back up, causing concerns about data consistency and cluster stability. Process: Step 1: Initial Investigation The client observed errors related to corrupted hint files, […]

Problem: The client experienced a problem with query slowness in their PostgreSQL database. Several queries were running slowly, and the application became unresponsive during the issue. The client required assistance in diagnosing and optimizing the queries contributing to the performance issues. Process: Step 1 – Initial Investigation The expert reviewed the PGAWR reports for the […]

Problem: The client encountered a high availability issue in their Cassandra cluster, consisting of five nodes deployed on AWS EC2. After shutting down two servers (10.51.44.25 and 10.51.46.144), it became impossible to connect to the database, even though the other nodes remained online. The issue manifested as an authentication error when trying to connect to […]