Case Studies

Problem: PostgreSQL 15.17 running in a Patroni-managed cluster experienced autovacuum tasks running for five hours or more on very large, non-partitioned tables. Customer-provided artifacts included table size and index statistics, autovacuum logs, pg_settings, and pg_stat_all_tables. Symptoms reported: multi-hour autovacuum/vacuum runs on large tables, continuous cleanup script execution, and 10-hour REINDEX operations on bloated indexes. The […]

Problem: The customer requested a concrete Disaster Recovery (DR) strategy for a Debezium-based CDC platform covering supported DR architectures, Active‑Active vs Active‑Passive tradeoffs, replication/failover mechanisms, Kafka dependencies, metadata/offset/schema‑history recovery, RTO/RPO targets, infrastructure prerequisites, and DB‑specific guidance for Oracle, Postgres and MySQL. The existing environment used Debezium connectors, Kafka, cross‑cluster replication (MirrorMaker2), and a managed schema […]

Problem: Production PostgreSQL clusters (PostgreSQL community edition 15.17) are running under Patroni (v3.3.2) with HAProxy (2.6.21) in front. All client connections go through HAProxy using virtual IPs and virtual ports; on the PostgreSQL side the reported client address is the HAProxy VIP rather than the originating physical client IP. The operations team needed a reliable […]

Problem: A production PostgreSQL cluster (community PostgreSQL 15.8, managed by Patroni v2.1.4) experienced an unplanned automatic switchover. Symptoms reported included an unexpected promotion of the standby after the primary stopped responding to health checks. Monitoring and OS traces showed memory usage climbing to ~95% on the primary node, PostgreSQL failing to respond to Patroni probe […]

Problem: A security scan of a deployed Apache Spark binary distribution reported two vulnerabilities located in Hive artifacts bundled with Spark: CVE-2024-29869 (org.apache.hive:hive-exec, fixed in Hive 4.0.1) and CVE-2024-23953 (org.apache.hive:hive-llap-common, fixed in Hive 4.0.1). The scan showed the vulnerable artifacts present in the shipped Spark distribution (the Spark 3.5.x line shipped a Hive 2.3.x fork […]

Problem: A production alert arrived with the subject indicating “client replied sms” and a Severity 3 priority: inbound reply messages were not being recorded by the downstream processing pipeline. The affected pipeline is an Apache Spark Structured Streaming job that consumes reply events from a message broker and uses a persistent checkpoint directory to track […]

Problem: An automated notification opened a Severity 2 case for Apache Spark with minimal details from the reporter. The only information attached to the alert was the product (Apache Spark) and the high-severity flag; no application logs or job identifiers were provided. Initial observable behavior from follow-up checks showed repeat Spark application failures on the […]

Problem: A high-traffic Patroni-managed PostgreSQL cluster experienced an automatic failover: the former primary (node A) lost leadership, and a secondary (node B) was successfully promoted. However, after the failover, node A failed to automatically rejoin the cluster as a replica. The symptoms unfolded in three stages. First, direct streaming replication was immediately rejected by the […]

Problem: A customer running Patroni-managed PostgreSQL (PostgreSQL 15.17, Patroni v3.3) had production databases ranging from >2-3 TB up to a ~52 TB physical replica. Rebuilding the empty DR site from production relied on manual pg_basebackup executions and manual post-backup recovery steps, which was too slow, operationally intensive, and prone to human error for such large volumes. The […]