Problem:

A production PostgreSQL 15.8 deployment running inside a Patroni-managed HA cluster (Patroni 2.1.4) reported transient application failures where processes could not connect and surfaced the message “PostgreSQL connection error, Cannot connect to FX DB”. Database logs showed repeated messages indicating SSL accept-stage failures with the text “could not accept SSL connection: EOF detected”. The failures appeared intermittently during normal DB operations and caused task failures in the application layer. The customer provided PostgreSQL, Patroni and system logs for the time window when the errors occurred.

Process:

Step 1: Context and initial log correlation

Initial analysis began from the PostgreSQL server log entries reporting “could not accept SSL connection: EOF detected” correlated with the customer-reported connection errors. Postgres configuration and Patroni runtime were reviewed to confirm versions (PostgreSQL 15.8, Patroni 2.1.4), SSL enabled on the server and that the errors occurred at the connection accept stage rather than inside SQL processing. This established the problem surface as SSL/TLS handshake failures occurring before session initialization, which explained the immediate application connection errors.

Step 2: Cross-check of Patroni and system logs for lifecycle events

Patroni and system message logs were examined for node restarts, role changes, or resource exhaustion that could explain abrupt disconnects. No leadership failovers or database restarts coincided with the SSL errors, and system metrics did not show resource saturation. The timing pattern and lack of server-side restarts suggested that incoming connections were being closed by the client side or an intermediary during TLS negotiation, making non-server-side causes likely.

Step 3: Inspecting TLS handshake behavior

OpenSSL-based probing was performed from application-tier hosts against the database port to observe handshake behavior. Successful handshakes occurred when a proper TLS client initiated the connection; failed handshakes matched the EOF behavior when a client opened a plain TCP connection to an SSL-enabled listener (or closed the socket immediately). This produced a reproducible distinction between correctly configured clients and probes/clients that did not perform TLS, matching the server log pattern.

Step 4: Reviewing connection sources and intermediary components

Connection-source patterns were derived from available anonymized logs and connection counts. A subset of short-lived connection attempts showed no completed TLS negotiation before socket close. Those attempts aligned with monitoring/health-check and connection-pooler behavior typical of external probes or misconfigured health checks. This indicated an intermediary (load-balancer, health checker, monitoring probe or an application-side pooler) performing non-TLS probes to the DB port rather than a native PostgreSQL client initiating TLS.

Step 5: Determining remediation approach

Given the mismatch between expected TLS traffic and non-TLS probes, remediation options were evaluated focusing on minimal production change: adjust probes to use TLS, change probes to a non-DB endpoint, or change the intermediary to TCP passthrough. A decision matrix prioritized fixes that required no change to PostgreSQL’s TLS configuration and limited application downtime. This analysis led to changes on the probing/intermediary side and improved server-side logging to prevent recurrence.

Step 6: Implemented change and verification

The implemented action was to reconfigure health checks and any load-balancer/monitoring probes that targeted the database SSL port so they either performed a TLS handshake or targeted a non-SSL health endpoint. Application connection strings and pooler settings were also verified and updated where needed to use sslmode=require. After the probe and client-side changes, repeat OpenSSL probes and live traffic produced no further EOF-on-accept events and application connection failures stopped occurring, confirming the fix.

Solution:

PostgreSQL remained configured with ssl = on. The remediation consisted of aligning all connection sources with that configuration: health checks and load-balancer probes were updated to perform TLS or redirected to a non-DB probe endpoint, and application/client pools were set to use sslmode=require where needed. Server-side logging was slightly tightened to capture failed SSL handshakes for faster future diagnosis.

Architecturally, the fix works because PostgreSQL’s SSL-enabled listener expects a TLS handshake at TCP accept. Non-TLS probes or plain-TCP connectivity attempts terminate the handshake and appear on the server as EOF during accept; ensuring probes either complete a TLS handshake or avoid the SSL listener prevents these premature socket closures and eliminates associated connection errors.

Conclusion:

The targeted configuration changes removed the SSL accept-stage EOF events and eliminated the intermittent connection errors observed by the application. Operational stability improved with no DB restarts required, fewer failed application tasks, and improved observability for any future TLS handshake issues.