post_parse_analyze hook reads pg_extension inside an aborted transaction; the locks leak into the PGPROC and every later backend on it fails with "failed to re-find shared lock object"

[SpongeRobert]

Versions. PostgreSQL 18.6, Apache AGE at the head of branch PG18,
e43dc1a12b78fba4acef9835b2b10379b8d243b4 (the 1.8.0 release commit, head on
2026-09-24), shared_preload_libraries = 'age'. The extension does not have to
be created in any database. is_age_extension_exists() is the same on master
(fa109ef1dd); the check came with a17bf457fe ("Make age extension usable
from shared_preload_libraries").

Symptom. Backends fail at connect with FATAL: failed to re-find shared lock object (SQLSTATE XX000), in every database and under every role, until
the server restarts. Later the server log may also show
lock AccessShareLock on object <db>/3079/0 is already held.

What happens.

1. A session fails a statement inside BEGIN and waits for ROLLBACK
   (TBLOCK_ABORT: AbortTransaction() has run, the resource owner of the
   transaction is released).
2. Its caches are reset: the shared invalidation queue overflowed while the
   backend was not reading it. The relcache callback clears
   extension_cache_is_valid, and the catcache entry of pg_extension is gone.
3. ROLLBACK is parsed. post_parse_analyze calls is_age_extension_exists(),
   which calls get_extension_oid("age", true). The catcache miss opens
   pg_extension and then pg_class. Their locks are granted on the fast path,
   then ResourceOwnerEnlarge() fails:
   ERROR: ResourceOwnerEnlarge called after release started.
4. Nothing releases those locks any more. In TBLOCK_ABORT the transaction is
   already aborted, so the backend exits with the two relations still in the
   fast-path array of its PGPROC.
5. The next backend on that PGPROC takes the same relation. It reuses the slot
   without counting it, and its LockRelease() goes to the main lock table,
   which has no entry: FATAL: failed to re-find shared lock object at connect.
   Its exit does not clear the slot, so every later occupant fails the same way.

The core does not read the catalog in an aborted transaction. The hook does so
on every parse, ROLLBACK included.

Evidence.

• An assertion of the core. PostgreSQL 18.6 built from REL_18_6 with
  --enable-cassert --enable-debug, AGE at e43dc1a, no patch. The first
  reproduction run aborts backend A on its ROLLBACK:

  TRAP: failed Assert("IsTransactionState()"), File: "relcache.c", Line: 2073
  postgres: ... ROLLBACK(ExceptionalCondition+0xa8)
  postgres: ... ROLLBACK(AssertCouldGetRelation+0x2e)
  ...
  postgres: ... ROLLBACK(SearchCatCache+0x45)
  postgres: ... ROLLBACK(SearchSysCache+0x95)
  postgres: ... ROLLBACK(GetSysCacheOid+0x3e)
  postgres: ... ROLLBACK(get_extension_oid+0x40)
  age.so(+0x11d3f)
  age.so(+0x2900f)
  postgres: ... ROLLBACK(parse_analyze_fixedparams+0xe5)
  LOG:  client backend (PID 109) was terminated by signal 6: Aborted
  DETAIL:  Failed process was running: ROLLBACK;
  

  AssertCouldGetRelation() is documented as "In code that reads catalogs in
  the event of a cache miss, call this before checking the cache". The same
  build with the fix below gives no TRAP, no error on ROLLBACK and no XX000.

• The stack on a release build at the first error. Taken with gdb on the
  backend of session A, with pgdg debug symbols; the breakpoint is errstart
  at level ERROR:

  #0  errstart (elevel=21, domain=0x0) at elog.c:354
  #1  errstart_cold (elevel=21, domain=0x0) at elog.c:329
  #2  ResourceOwnerEnlarge (owner=<optimized out>) at resowner.c:469
  #3  RelationIncrementReferenceCount (rel=0x795a669e11b8) at relcache.c:2191
  #4  RelationIdGetRelation (relationId=1259) at relcache.c:2121
  #5  relation_open (relationId=1259, lockmode=1) at relation.c:58
  #6  table_open (relationId=1259, lockmode=1) at table.c:44
  #7  ScanPgRelation (targetRelId=3079, indexOK=true, force_non_historic=false) at relcache.c:371
  #8  RelationBuildDesc (targetRelId=<optimized out>, insertIt=true) at relcache.c:1115
  #9  RelationIdGetRelation (relationId=3079) at relcache.c:2144
  #10 relation_open (relationId=3079, lockmode=1) at relation.c:58
  #11 table_open (relationId=<optimized out>, lockmode=1) at table.c:44
  #12 SearchCatCacheMiss (cache=0x5c875436e300, nkeys=1, ...) at catcache.c:1554
  #13 SearchCatCacheInternal (...) at catcache.c:1499
  #14 SearchCatCache (...) at catcache.c:1357
  #15 SearchSysCache (key1=1, cacheId=<optimized out>) at syscache.c:217
  #16 GetSysCacheOid (cacheId=27, oidcol=1, ...) at syscache.c:459
  #17 get_extension_oid (extname="age", missing_ok=true) at extension.c:192
  #18 is_age_extension_exists () at src/backend/catalog/ag_catalog.c:88
  #19 is_age_extension_exists () at src/backend/catalog/ag_catalog.c:74
  #20 post_parse_analyze (pstate=..., query=..., jstate=<optimized out>) at src/backend/parser/cypher_analyze.c:90
  #21 parse_analyze_fixedparams (..., sourceText="ROLLBACK;", ...) at analyze.c:128
  #22 pg_analyze_and_rewrite_fixedparams (...) at postgres.c:683
  #23 exec_simple_query (query_string="ROLLBACK;") at postgres.c:1190
  

• The reproduction (repro.sh below: psql, pgbench and kill, on a fresh
  server). Ten fresh servers per row:

  Build	rollback	quit (control: the session leaves without ROLLBACK)
  AGE e43dc1a as is	10 of 10 reproduced	0 of 10
  the same with the fix below	0 of 10	not run

| AGE master (fa109ef1dd) as is | 10 of 10 reproduced | not run |
| master with the fix | 0 of 10 | not run |

Without the overflow of the invalidation queue (step 2) it is 0 of 10: the
cache of is_age_extension_exists() is still valid, so the catalog is not
read.

Suggested fix (fix.diff below, against e43dc1a; it applies to
master with an offset of 2 lines, and make installcheck on master with it
passes all 42 tests). A pull request against master follows. At the top of is_age_extension_exists():

if (!IsTransactionState())
    return false;

Nothing is cached, so the next live transaction asks the catalog again. In an
aborted transaction only the statements that leave it reach the parser, and AGE
has nothing to analyse in them, so "no extension" is the right answer for that
parse.

The other callers of is_age_extension_exists() are is_age_drop() (a
DROP statement), object_access() (DDL) and set_rel_pathlist()
(planning). By our reading of exec_simple_query(), an aborted transaction
rejects those statements before parse analysis, so only post_parse_analyze
reaches the catalog there. That is from reading the code: we reproduced and
traced only the post_parse_analyze path. The fix covers every caller anyway.

Related: #2520, another hook of AGE that misbehaves when age is in
shared_preload_libraries.

repro.sh

#!/usr/bin/env bash
# Reproduction: the post_parse_analyze hook of Apache AGE reads pg_extension
# while parsing the ROLLBACK of a failed transaction; the locks it takes leak
# into the PGPROC, and the next backend on that PGPROC fails at connect with
# "FATAL: failed to re-find shared lock object".
#
# Needs: a FRESH server with shared_preload_libraries = 'age' (the extension
# does not have to be created), a superuser, psql and pgbench, and a shell on
# the server host as the OS user of the server (step 2 stops a backend with
# kill). Connection parameters come from the usual PG* environment variables.
# A poisoned PGPROC lives until the server restarts: use a throwaway server.
#
# Exit code: 1 when XX000 was seen (reproduced), 0 when every connection of
# step 4 succeeded, 3 when the run could not be carried out.
set -uo pipefail
export PGUSER="${PGUSER:-postgres}" PGDATABASE="${PGDATABASE:-postgres}"
END="${1:-rollback}" # rollback | quit (the control: the session leaves without ROLLBACK)

fifo=$(mktemp -u); out=$(mktemp); mkfifo "$fifo"
trap 'rm -f "$fifo" "$out" /tmp/age-repro-one.sql' EXIT

# 1. Session A fails a statement inside a transaction and stays in the
#    failed block; its input is a fifo, so it stays open between the steps.
psql -XAt <"$fifo" >"$out" 2>&1 &
exec 3>"$fifo"
echo "SELECT pg_backend_pid();" >&3
for _ in $(seq 50); do pid=$(grep -m1 -E '^[0-9]+$' "$out") && break; sleep 0.1; done
[ -n "${pid:-}" ] || { echo "session A did not report its pid"; exit 3; }
echo "BEGIN; SELECT 1/0;" >&3
sleep 0.3

# 2. Its backend is stopped while another session overflows the shared
#    invalidation queue; on SIGCONT backend A resets its caches, the cached
#    "is age installed" flag and the catcache entry of pg_extension among them.
kill -STOP "$pid"
psql -Xq -c "DO \$\$ BEGIN
	FOR i IN 1..1000 LOOP EXECUTE format('CREATE TABLE age_repro_%s (a int)', i); END LOOP;
	FOR i IN 1..1000 LOOP EXECUTE format('DROP TABLE age_repro_%s', i); END LOOP;
END \$\$" >/dev/null || { kill -CONT "$pid"; echo "the overflow did not run"; exit 3; }
kill -CONT "$pid"
sleep 0.3

# 3. ROLLBACK is parsed: the hook reads pg_extension outside a live transaction
#    and fails with "ResourceOwnerEnlarge called after release started".
[ "$END" = rollback ] && echo "ROLLBACK;" >&3
echo '\q' >&3
exec 3>&-
wait
grep -E 'ResourceOwnerEnlarge|ERROR' "$out" || true

# 4. Every free PGPROC is taken at once (the free list is FIFO, so the one
#    session A gave back is taken last).
max=$(psql -XAtc "SHOW max_connections")
used=$(psql -XAtc "SELECT count(*) FROM pg_stat_activity WHERE backend_type = 'client backend'")
n=$((max - used))
echo 'SELECT 1;' >/tmp/age-repro-one.sql
res=$(pgbench -n -c "$n" -j 8 -t 1 -f /tmp/age-repro-one.sql 2>&1)
if grep -qE 'failed to re-find shared (lock|proclock) object' <<<"$res"; then
	grep -m1 -E 'failed to re-find' <<<"$res"
	echo "REPRODUCED (end=$END, connections=$n)"; exit 1
fi
if grep -q "actually processed: $n/$n\$" <<<"$res"; then
	echo "not reproduced (end=$END, connections=$n)"; exit 0
fi
echo "the saturation did not run as intended:"; tail -3 <<<"$res"; exit 3

fix.diff (against e43dc1a)

--- a/src/backend/catalog/ag_catalog.c
+++ b/src/backend/catalog/ag_catalog.c
@@ -75,6 +75,15 @@
 {
     static bool callback_registered = false;
 
+    /*
+     * Outside a live transaction (the ROLLBACK of a failed one) the catalog
+     * must not be read: the resource owner is already released, and the locks
+     * a catalog scan takes here would leak into the PGPROC. Nothing is cached,
+     * so the next live transaction asks the catalog again.
+     */
+    if (!IsTransactionState())
+        return false;
+
     if (extension_cache_is_valid)
         return age_extension_exists;