05.10.2026 Problem:
A production PostgreSQL 15.17 cluster managed by Patroni 3.3.2, with PgBouncer 1.25.1 in front of it, showed intermittent application failures during night operations. The application logged ODBC connection errors with SQLSTATE 08P01: “server login has been failing, cached error: connect failed (server_login_retry)”. The customer suspected PostgreSQL connection exhaustion. Metrics showed otherwise: max_connections was 4,000 with 1,434–1,685 active sessions, and PgBouncer reported cl_waiting = 0 and maxwait = 0.
Process:
Step 1: Rule out connection exhaustion
Compared the application errors with the customer’s PostgreSQL session statistics and PgBouncer stats. PostgreSQL had more than half of its connection limit free and no clients were queued in PgBouncer, so neither max_connections nor pool exhaustion could explain the rejections.
Step 2: Find where the failing connections were going
Analyzed the full PgBouncer log for 27 September to 2 October. All 147 server-side “connect failed” events targeted the IPv6 loopback [::1]:5432, and none targeted 127.0.0.1. The PostgreSQL log for the same night contained no mention of ::1, which showed the connections never reached PostgreSQL and were refused by the operating system.
Step 3: Explain the mismatch
PgBouncer’s database entries used host=localhost, which on these servers resolves to both 127.0.0.1 and ::1. PgBouncer 1.25 defaults to load_balance_hosts = round-robin and rotates between the resolved addresses when opening server connections. Patroni starts PostgreSQL with listen_addresses = 0.0.0.0, which is IPv4 only, so nothing listens on ::1:5432 and every attempt to that address is refused immediately.
Step 4: Match the failure pattern to PgBouncer’s retry behavior
After a failed server connect, PgBouncer caches the failure for server_login_retry (15 seconds) and rejects every new client login to that pool with the cached error. When the retry later picks 127.0.0.1, the pool recovers on its own, which is why the problem comes and goes. The log timeline confirmed this: on 2 October at 00:12:38, one ::1 failure was followed by 99 client rejections within 0.1 seconds. The same pattern recurred on 29 September, 30 September, 1 October and again on 2 October, including 210 rejections at 01:45:48 for a second pool.
Step 5: Choose the lowest-risk fix
Two options were considered. Making PostgreSQL listen on IPv6 (listen_addresses = '0.0.0.0,::1' through Patroni plus a pg_hba.conf entry for ::1/128) would require a PostgreSQL restart on a production cluster. Pointing PgBouncer directly at the IPv4 address needs only a configuration reload, so it was recommended: set host=127.0.0.1 for each database entry in pgbouncer.ini on both nodes and run RELOAD; on the PgBouncer admin console. A reload keeps client connections open and replaces server connections gradually, with no application change.
Step 6: Provide verification steps
The customer received commands to confirm the cause before the change and the effect after it: getent ahosts localhost to show both addresses, ss -lnt sport = :5432 to show the IPv4-only listener, psql -h ::1 -p 5432 to reproduce the refusal, and a count of [::1]:5432 entries in the PgBouncer log, which should stop growing after the fix.
Step 7: Flag secondary findings
Several issues unrelated to the root cause were reported for follow-up: 129 “prepared statement already prepared” errors from the ODBC client, suggesting a review of UseServerSidePrepare under transaction pooling; about 1,400 idle sessions connecting directly to PostgreSQL on port 5432 and bypassing PgBouncer; repeated password authentication failures for one application user, indicating an outdated password on the application host; and a separate heavy-load period on 29–30 September with query_wait_timeout events and slow server logins.
Solution:
The recommended fix was to replace host=localhost with host=127.0.0.1 in the PgBouncer database definitions and reload PgBouncer, leaving PostgreSQL and Patroni unchanged. This works because PgBouncer can no longer pick the IPv6 loopback address that PostgreSQL does not listen on, so no server connect fails and the cached server_login_retry state that rejected client logins is never triggered.
Conclusion:
What looked like PostgreSQL connection exhaustion turned out to be a name-resolution mismatch between PgBouncer and an IPv4-only PostgreSQL listener. Log analysis identified the cause, showed it was recurring rather than a one-time event, and led to a reload-only fix that avoids restarting a production database. The customer received the fix, verification commands, and follow-up recommendations to route idle direct sessions through PgBouncer, review ODBC server-side prepare settings, and update the outdated application credentials, and then closed the case.