Case Studies

Problem: The client encountered a recurring issue within the Docker Swarm environment, wherein containers sporadically crashed with exit code 137. This behavior, indicative of potential memory-related issues, was exacerbated by the absence of corresponding container logs, complicating the diagnostic process. Process: Initial Inquiry and Investigation: Prompted by the client’s request for a Root Cause Analysis […]

Problem: The client is experiencing daily automatic restarts of their Cassandra database, potentially causing issues with application connectivity. Additionally, they’re encountering errors related to JVM memory, degraded mode, connection resets by peer, and null pointer exceptions on indexes. Solution: After a thorough analysis of the provided logs, the following findings and recommendations were made: Identify […]

Problem: A client with an Elasticsearch cluster consisting of three nodes was experiencing a recurring issue where one of the nodes is disconnecting from the cluster automatically. This disruption was resulting in numerous unassigned shards, impacting the overall stability and performance of the Elasticsearch environment. Process: Data Collection for Further Analysis: Request: Provide detailed system […]

Problem: The client’s Elastic server is experiencing sporadic crashes, evident from StackOverflowError logs linked to “GraphTokenStreamFiniteStrings.” While a potential fix exists in Lucene 9.7, the product’s certification with Elastic 7.17 using Lucene 8.11.1 complicates the implementation of the fix. The client seeks assistance to evaluate the possibility of backporting Lucene 9.7 changes to Lucene 8.11.1. […]

Problem: The problem identified in Apache Airflow version 2.5.0 was the lack of password confirmation during password changes, posing a significant security risk to users. This vulnerability could potentially lead to unauthorized access and session hijacking within the Airflow application. Solution: Based on the client’s request and the provided information, the recommended solution steps to […]

Problem: The client reported instances of Cassandra databases experiencing unavailability and performance degradation, leading to application connection errors. Specifically, nodes were found to be unresponsive, and memory and CPU utilization spiked unexpectedly. Additionally, there were concerns about data inconsistencies between data centers and excessive tombstones affecting performance. Process: Initial Investigation: Upon initial investigation, it was […]

Problem: After deleting 52 million rows from a table, the client performed a VACUUM FULL and REINDEX on the table. Following these maintenance operations, the performance of large joins involving this table has significantly deteriorated, with query execution times increasing from minutes to hours. The task is to identify the cause of this performance regression […]

Problem: The client is unable to register to a FreeIPA server, encountering the error message “Cannot obtain CA certificate.” The error log indicates issues with LDAP access and Kerberos database. Process: The expert took the following steps to address the client’s registration issue with the FreeIPA server: Reproduction of the Setup: The expert attempted to […]

Problem: The client had 10 monthly bill cycles, and during such bill cycle days, the billing process was connected to the DB cluster in 20 streams. After the process had started the replication the above-mentioned billing process failed with the “could not send data to the client: Connection reset by peer” error. The client reduced […]