Case Studies

Problem: The client was facing a situation where fluent-bit containers jeopardized the k8s host by overusing the temporary space which is on the host. Solution: After the investigation, the expert team suggested the next solution to the client: Resource Limitations: Checked for resource limitations on the fluent-bit containers. Insufficient CPU or memory limits could have […]

Problem: The client experienced timeout errors when running adhoc queries in an Apache Cassandra cluster. Specifically, a query with multiple conditions was timing out due to the coordinator node not receiving responses from replica nodes. Process: Here are the steps taken by the expert in the process of resolving the problem: Assessment of Query Efficiency: […]

Problem: A client encountered connection errors while attempting to connect to a Cassandra database cluster using cqlsh. The error messages indicated issues related to protocol version compatibility and SSL certificate verification failures (“This version of the driver does not support protocol version 21”). Process: Initial Diagnosis: The support team analyzed the error messages provided by […]

Problem: The production OpenSearch cluster encountered frequent shutdowns due to Out of Memory (OOM) errors across all four nodes. The exact cause of the OOM issue needed further investigation, requiring specific data and logs to diagnose the root cause. Process: Resource Assessment: Evaluate RAM, CPU, and disk usage on each node to identify potential resource […]

Problem: The client’s billing system experienced around 10 billing cycles each month. During each billing cycle, the billing process was initiated and connected to the database cluster with 20 concurrent streams. However, upon starting the process, both the replication and the billing process failed, displaying the following error: “Could not send data to the client: […]

Problem: The client seeks to configure SSL communication within an existing Patroni/etcd/Postgres cluster, specifically aiming to switch to HTTPS in the Patroni configuration file to secure communication between components. Solution: After a thorough analysis, the following recommendations were made: Certificate Generation Utilize OpenSSL or obtain a certificate from a trusted Certificate Authority (CA). For self-signed […]

Problem: The client is experiencing high read latency in their production cluster monitoring. They are seeking assistance in identifying the cause of this latency and resolving it to prevent potential outages. Process: Steps and measures undertaken to investigate the issue: Initial Assessment: Requested logs/config from all nodes. Observed server overload or potential network issues. Configuration […]

Problem: Java client for Apache Kafka is missing error logs when attempting to publish messages to a Kafka topic that is full on the broker side. Despite expecting error logs in such cases, the logs only show warnings and traces, hindering quick identification of the root cause. Solution: Main solution steps suggested to resolve the […]

Problem: The client’s Prometheus pod, despite having substantial memory resources, is experiencing prolonged startup times, likely due to extended WAL (Write-Ahead Logging) loading durations. This delay leads to readiness probe failures and leaves the pod in a failed state. The client seeks a resolution to mitigate this performance issue and ensure prompt pod initialization. Solution: […]