Case Studies

Problem: The client reported that data storage space was being exhausted because the incremental backups were being saved in the same location as the data storage. This issue forced the client to delete incremental backups to free up space for new data. The client requested a consultation on how to configure incremental backups to be […]

Problem: The client reported several security vulnerabilities in Airflow version 2.5.0, including weak password policies such as allowing passwords with less than 8 characters, lack of password expiration, and absence of enforced password changes during the first login. These weaknesses compromise overall system security and user account integrity. Solution: To address these issues, the expert […]

Problem: The client reported encountering a recurring issue when attempting to execute scripts within a Docker container. The client consistently received an error code 137, indicating an Out of Memory (OOM) condition. Despite attempts to resolve the issue by restarting and reinstalling Docker, the problem persisted. Process: Gathering System Information: The Docker version being used; […]

Problem: The client requested to review cluster configuration and advise any changes to the configuration parameters to avoid any potential issues proactively. Additionally, the client requested advice on how to identify Cassandra database cluster’s workload. Process: 1) Data Collection: Gathered configuration files Collected the last 5,000 lines of the system log 2) Expert Review: Conducted […]

Problem: Client reported an issue with Cassandra version 4 after upgrading from version 3. Running the “nodetool repair” command on a cluster with 2 data centers (3 nodes each) resulted in an error indicating that the incremental repair session failed. This issue did not occur with version 3, and all nodes showed no pending compaction […]

Problem: On July 8th, 2024, all Docker containers on all nodes within a Docker Swarm cluster suddenly crashed. The cluster consisted of 13 nodes: 1 master, 2 reachable, and 10 worker nodes. The initial logs indicated a problem with the RAFT consensus algorithm attempting and failing to elect a leader multiple times. Process: Upon receiving […]

Problem: The Jenkins server experienced significant performance issues characterized by excessive thread creation and inadequate resource allocation. Symptoms included system freezes, failures to execute commands, and frequent application errors related to memory and resource limits. Process: Initial Investigation: Error Identification: Logs and system monitoring revealed critical errors related to insufficient memory and resource limits. Key […]

Problem: Argo CD, version v2.10.4, deployed in a high-load environment with approximately 480 applications, encountered severe performance issues. Specifically, the refresh operations were inconsistent, sometimes taking up to 16 minutes, impacting deployment efficiency and causing significant CPU consumption by the controller pod. Despite efforts to adjust configuration parameters based on Argo CD documentation, the issue […]

Problem: Some client servers are not receiving updated data from our IPA servers. For example, listing hosts in a specific host group on one client server shows missing hosts: ~]# ipa host-find --in-hostgroups=rhel9_hosts | grep Host | grep -i ra Host name: india In contrast, the same command on another server shows additional hosts: ~]# […]