Database

Problem: The client encountered challenges connecting to the cqlsh on several nodes within their Cassandra cluster. Additionally, discrepancies were noted in the output of “nodetool status” across different nodes, with certain nodes appearing as down. Seeking assistance, the client provided output files for analysis, prompting intervention to rectify the connectivity issues. Process: Check Network Connectivity: […]

Problem: The client reported high memory consumption on both leader (Node 1) and replica (Node 2) nodes in PostgreSQL version 13. Memory utilization on both nodes was observed to be significantly elevated. On Node 1, high memory usage was associated with PostgreSQL processes such as checkpoint and background writer operations, while Node 2 was undergoing […]

Problem: The client reported that Cassandra 2.2.5, running in a single-node configuration, crashed and failed to start. The error logs pointed to a “CorruptSSTableException” in the “system_traces.events” table, indicating corruption in an SSTable file. Since this was a critical system table, the client needed a way to bypass the corrupted data and restart the Cassandra […]

Problem: The client reported a critical increase in pending tasks on one of the nodes within their Cassandra cluster. This issue was causing concern, and the client sought assistance in understanding the root cause and implementing a resolution. Process: The client initially executed the nodetool compaction-stats -H command on the affected node and restarted it, […]

Problem: The client reported recurring crashes of a Cassandra node with errors related to “too many open files”. Despite increasing the maximum open files limit, the issue persisted. The problem was observed primarily during high server load, with regular crashes around 01:15 AM. The client suspected that network instability or heavy operations, such as running […]

Problem: The client has master and standby configuration and connections are managed by pgpool (version 4.1.5) in the PostgreSQL (version 11.5) production environment. Recently the client faced a switchover/failover when an “idle_in_transaction” session was killed using “pg_terminate_backend” after connecting via pgpoolvip. Process: Step 1 – Initial investigation: The expert team gathered the information and started […]