Case Studies

Problem: After restoring a PostgreSQL 15.8 database from Commvault, the client noticed multiple errors during startup. Autovacuum began trying to remove temporary tables, but instead reported issues such as: “Dropping orphan temp table” “pg_attribute catalog is missing attributes” Indexes and constraints appearing only partially restored About 30 temp tables were affected, and several indexes or […]

Problem: The client, reported repeated network timeouts and instability in their Apache Kafka environment, resulting in missing Call Detail Records for specific time periods. The issue manifested as persistent connectivity errors between Kafka brokers and 100% CPU utilization on the node hosting critical services (UMS, App, and ODF pods). Process: Step 1: Initial Identification The […]

Problem: The client operates a 6-node multi-DC replication setup for Cassandra, consisting of 3 nodes in the PROD datacenter and 3 nodes in the DR datacenter. They are planning to perform DDL changes, including altering tables to adjust the default_time_to_live parameter and dropping and recreating a table with a new definition. However, they are unsure […]

Problem: The client encountered a critical issue while starting one of their production pods during a Cassandra migration. Although the PostgreSQL migration completed successfully, the Cassandra migration failed with a com.datastax.oss.driver.api.core.AllNodesFailedException, indicating that the driver could not connect to any Cassandra nodes. This blocked the production deployment. Process: Step 1: Initial Analysis The logs revealed […]

Problem: The client using Argo Workflows identified a compliance issue during a Twistlock (Prisma Cloud) scan. The image argoproj/argoexec:v3.7.1 failed the CIS Docker Benchmark 4.1 rule, which requires containers to run as a non-root user. Running containers as root posed a risk of privilege escalation and potential account takeover. The client wanted to resolve the […]

Problem: A Prometheus deployment failed a security compliance scan when Prisma Cloud flagged a high-severity vulnerability in the sidecar container image kiwigrid/k8s-sidecar:1.30.9. The issue was linked to the pip package, which was identified as vulnerable to arbitrary code execution. This raised concerns about both compliance and the security of the monitoring stack. Process: Step 1: […]

Problem: A client deploying Harbor, the open-source container registry, wanted to replace the default Hossted-assigned domain with their own custom domain. Despite following setup instructions, Harbor continued to resolve to the old hostname. This resulted in SSL certificate mismatch errors, failed authentication requests, and missing features in the UI such as Proxy Cache and Helm […]

Problem: The client reported that their Apache Airflow Scheduler pod was not automatically detecting updates from a Persistent Volume Claim (PVC) named airflow-dags. Meanwhile, a related airflow-triggerer pod using the same PVC was updating files as expected. To temporarily resolve the issue, the client had to manually restart (“bounce”) the scheduler pod to force it to recognize new […]

Problem: The client, preparing to launch Argo Workflows v3.6.0 for production, faced a critical RBAC issue when enabling Single Sign-On (SSO) with Azure Active Directory (Entra ID) via Dex. When users attempted to log in through SSO, Argo Workflows returned the error: unknown name groups This prevented role-based authorization through group membership, effectively blocking all […]