DevOps

Problem: The client using Argo Workflows identified a compliance issue during a Twistlock (Prisma Cloud) scan. The image argoproj/argoexec:v3.7.1 failed the CIS Docker Benchmark 4.1 rule, which requires containers to run as a non-root user. Running containers as root posed a risk of privilege escalation and potential account takeover. The client wanted to resolve the […]

Problem: A client deploying Harbor, the open-source container registry, wanted to replace the default Hossted-assigned domain with their own custom domain. Despite following setup instructions, Harbor continued to resolve to the old hostname. This resulted in SSL certificate mismatch errors, failed authentication requests, and missing features in the UI such as Proxy Cache and Helm […]

Problem: The client, preparing to launch Argo Workflows v3.6.0 for production, faced a critical RBAC issue when enabling Single Sign-On (SSO) with Azure Active Directory (Entra ID) via Dex. When users attempted to log in through SSO, Argo Workflows returned the error: unknown name groups This prevented role-based authorization through group membership, effectively blocking all […]

Problem: A telecommunications customer operating a production Kubernetes cluster deployed via Kubespray encountered an infrastructure challenge. One of their original control-plane nodes (kz-bss-k8om01) had previously failed and was replaced with a new node named kz-bss-k8om04. Later, the client requested to rename this node back to its original FQDN (kz-bss-k8om01) and ideally retain the original IP […]

Problem: After upgrading the Jenkins Active Choices Plugin from version 2.6.1 to 2.8.1, the client’s Jenkins instance began exhibiting critical malfunctions in jobs that utilized multi-level reactive reference parameters. These parameters, implemented via Active Choices Reactive Reference Parameter fields, rely on Groovy scripts to dynamically populate choices based on the values of one or more […]

Problem: The client encountered an issue with desynchronization of nodes in an ETCD cluster running on RHEL 8.8. Error logs indicated significant disk write delays (slow fdatasync), which caused the Patroni cluster to fail and become unavailable. Process: Step 1 – Initial Analysis The expert asked the client to check the health of the cluster […]

Problem: The client encountered frequent master (manager) re-elections in their production Docker Swarm cluster, despite having the dispatcher-heartbeat value set to 2 minutes. These re-elections were happening within fractions of a second, causing concerns around Swarm stability and service availability. The client’s Docker environment was based on version 1.13.1 running on RHEL 7.9. Key symptoms […]

Problem: The client faced an issue integrating Jenkins with Keycloak for authentication. While the Jenkins UI successfully authenticated users via Keycloak, API calls from backend services were failing. According to Jenkins’ documentation, API requests should be authenticated using an API token, but despite following the recommended steps, the client encountered authentication failures (403 Forbidden & […]

Problem: The client experienced issues with Docker Swarm configuration in production. Specifically, when a container restarted, the application failed to recover properly. The client requested a review of the configuration to identify the root cause and potential improvements to enhance the cluster’s functionality. Process: Step 1: Initial Investigation The client provided details of the Docker […]