Database

Problem: The client reported a sudden and significant drop in Apache Cassandra performance on a 4-node cluster. The issue appeared without any recent configuration or infrastructure changes. The application started experiencing frequent timeouts, and restarting Cassandra services on all nodes did not resolve the problem. The client provided various monitoring graphs, system logs, and other […]

Problem: The client needed to scale their production Cassandra cluster from 6 nodes to 12 nodes (3 to 6 nodes per data center) without any downtime. Their existing setup includes Cassandra version 4.1.6, with two data centers (PROD and DR), each containing 3 nodes, forming a 6-node cluster with a replication factor of 3 and […]

Problem: The client needed to implement audit logging for their PostgreSQL 15 databases. Specifically, they wanted to track user actions such as: Configuration changes Creation, deletion, or modification of objects (documents, users, settings) Attempts to access forbidden resources Privilege escalation attempts Additionally, they requested that auditing be limited only to administrative users, not applied globally […]

Problem: The client was using Apache Cassandra 4.1.5 installed via a tarball extraction on an AWS EC2 machine and wanted to upgrade both their Cassandra version and the operating system. The installation was done manually using the tarball method, and the client needed to understand the feasibility and potential challenges involved in upgrading the OS […]

Problem: The client experienced a failure in running repairs in Apache Cassandra due to corruption in hint files. Additionally, a node in the cluster went down and could not be brought back up, causing concerns about data consistency and cluster stability. Process: Step 1: Initial Investigation The client observed errors related to corrupted hint files, […]

Problem: The client experienced a problem with query slowness in their PostgreSQL database. Several queries were running slowly, and the application became unresponsive during the issue. The client required assistance in diagnosing and optimizing the queries contributing to the performance issues. Process: Step 1 – Initial Investigation The expert reviewed the PGAWR reports for the […]

Problem: The client encountered a high availability issue in their Cassandra cluster, consisting of five nodes deployed on AWS EC2. After shutting down two servers (10.51.44.25 and 10.51.46.144), it became impossible to connect to the database, even though the other nodes remained online. The issue manifested as an authentication error when trying to connect to […]

Problem: The client faced intermittent downtimes in their PostgreSQL cluster, which is managed by Patroni for high availability. These downtimes were particularly prominent during failover events when the system failed to transition smoothly between nodes during leader elections. As a result, PostgreSQL was unable to maintain continuity of service, affecting the application performance. Logs from […]

Problem: The client observed inconsistent behavior in Elasticsearch search results when searching for strings containing reserved characters, such as colons, slashes, parentheses, and curly braces. These inconsistencies were most notable when the query string included special characters without proper escaping or when using quotes around the search values. This caused mismatches in expected results, with […]