Data Management and Analytics

Problem: Production Patroni-managed PostgreSQL 15.17 cluster (Patroni 3.3.2) experienced intermittent performance degradation that caused application delays and connection resets. The environment uses a connection pooler (PGBouncer) fronting ~1,600 read-only connections while the primary accepted ~1,600 direct read-write connections. The team reported long query latencies, mass connection timeouts, and large temporary file creation observed in database […]

Problem: A production Patroni-managed PostgreSQL (community 15.17, Patroni 3.3.2) cluster experienced recurrent nightly performance degradation. Symptoms included large numbers of LWLock wait events and high query latency during night operations reported by the application and visible in the database analytical report. Patroni and PostgreSQL logs were provided for analysis. Operational context: PgBouncer was present as […]

Problem: A production Patroni-managed PostgreSQL 15 cluster experienced periodic heavy queries that threatened availability. An example slow job ran for ~84 seconds and performed a full scan of a 1.8 TB partitioned table (arbor.CDR_DATA) that uses daily partitions starting in early April. Most clients connect through generic application users rather than distinct personal accounts. The […]

Problem: A production Patroni-managed PostgreSQL cluster (PostgreSQL v15.17, Patroni 3.3.2) experienced a primary process abort with SIGABRT during normal operation. Server logs reported that a server process was terminated by signal 6 (Aborted) and that the failed process was executing a COMMIT when the postmaster began terminating other server processes. Subsequent messages showed PostgreSQL could […]

The rapid evolution of generative artificial intelligence has created a significant need for systems that can store and retrieve information with human-like semantic understanding. ChromaDB has emerged as a pivotal technology in this landscape, acting as a specialized storage layer that allows applications to “remember” and reason over vast amounts of unstructured data. By bridging […]

In today’s data-driven landscape, organizations are constantly handling large volumes of documents—from invoices and contracts to logs and reports. Extracting, processing, and analyzing this information efficiently is critical for productivity and decision-making. Docling offers a powerful, modern solution that automates document processing and transforms unstructured data into actionable insights. What is Docling? Docling is an […]

In today’s data-driven world, delivering fast, relevant, and personalized search and recommendation experiences at scale is a competitive necessity. Vespa is a powerful open-source engine designed to handle large-scale search, real-time analytics, and machine learning inference in a single, unified platform. Built for performance and flexibility, Vespa enables organizations to serve intelligent applications with low […]

In the era of artificial intelligence and large language models, the ability to process and search through massive amounts of unstructured data has become a competitive necessity. This data includes images, video, and text. Milvus emerges as a leader in this landscape. As an open source vector database, Milvus is specifically designed to manage embedding […]

Modern data science and machine learning workflows rely heavily on notebooks for experimentation, analysis, and collaboration. However, traditional notebook tools often suffer from hidden state, execution order issues, and poor version control support. Marimo addresses these challenges by rethinking how Python notebooks work. Designed for reliability, reproducibility, and developer-friendly workflows, Marimo is especially relevant for […]