Transactions & Isolation
> How databases keep concurrent work correct — ACID guarantees, isolation levels, MVCC, locking, and the anomalies each level prevents.
What is a Transaction?
A transaction is a group of statements that succeed together or fail together. The classic example: transferring money between two accounts — debit one, credit the other. Either both happen, or neither does.
BEGIN;
UPDATE accounts SET balance = balance - 100 WHERE id = 1;
UPDATE accounts SET balance = balance + 100 WHERE id = 2;
-- If anything went wrong, roll back the whole thing:
-- ROLLBACK;
COMMIT;ACID
- Atomicity — all or nothing. Partial writes never persist.
- Consistency — constraints (PK, FK, CHECK) hold before and after.
- Isolation — concurrent transactions don't step on each other.
- Durability — once COMMIT returns, the change survives a crash.
Concurrency Anomalies
Isolation levels are defined by which of these they prevent:
- Dirty read — you see another transaction's uncommitted change.
- Non-repeatable read — the same row, read twice in your transaction, shows different values because someone else committed between reads.
- Phantom read — a query's result set grows or shrinks when re-run because someone inserted/deleted matching rows.
- Write skew — two transactions read overlapping data, then each writes based on assumptions the other invalidates.
Isolation Levels
| Level | Dirty | Non-repeat | Phantom | Write skew |
|---|---|---|---|---|
| READ UNCOMMITTED | possible | possible | possible | possible |
| READ COMMITTED (PG default) | prevented | possible | possible | possible |
| REPEATABLE READ | prevented | prevented | prevented (PG) | possible |
| SERIALIZABLE | prevented | prevented | prevented | prevented |
-- Set the level for the current transaction
BEGIN ISOLATION LEVEL SERIALIZABLE;
-- ... your work ...
COMMIT;
-- Or set the default for the session
SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL REPEATABLE READ;MVCC — Multi-Version Concurrency Control
Postgres (and most modern engines) don't lock rows for reads. Instead, every row has multiple versions tagged with the transaction IDs that created and deleted them. Your transaction sees the version visible at its snapshot point — readers never block writers, writers never block readers. The cost: dead tuples accumulate and need VACUUM.
Explicit Locking
Use SELECT ... FOR UPDATE to lock rows you intend to modify. This prevents lost updates when read-then-write happens across transactions.
BEGIN;
-- Lock the row so nobody else can update it until we COMMIT
SELECT balance FROM accounts WHERE id = 1 FOR UPDATE;
-- Now we can safely compute and write
UPDATE accounts SET balance = balance - 100 WHERE id = 1;
COMMIT;
-- Variants:
-- FOR UPDATE - exclusive, blocks readers asking FOR UPDATE
-- FOR NO KEY UPDATE - lighter, allows FK references
-- FOR SHARE - shared, blocks FOR UPDATE
-- SKIP LOCKED - useful for job queues (skip rows others are processing)
-- NOWAIT - fail immediately instead of waitingDeadlocks
Two transactions each hold a lock the other needs. The database detects the cycle and aborts one with an error. Avoidance:
- Always acquire locks in a consistent order (e.g. ascending by id).
- Keep transactions short — open BEGIN, do work, COMMIT fast.
- Use
SELECT ... FOR UPDATEupfront rather than escalating later.
Savepoints — Partial Rollback
BEGIN;
INSERT INTO orders(customer_id, total) VALUES (42, 100);
SAVEPOINT before_items;
INSERT INTO order_items(order_id, sku) VALUES (currval('orders_id_seq'), 'BAD-SKU');
-- Oops, FK violation. Roll back just the items, keep the order.
ROLLBACK TO SAVEPOINT before_items;
INSERT INTO order_items(order_id, sku) VALUES (currval('orders_id_seq'), 'GOOD-SKU');
COMMIT;Autocommit vs Explicit Transactions
By default, most clients run in autocommit mode — every statement is its own transaction. This is convenient but dangerous for multi-statement operations: if the second statement fails, the first is already committed and cannot be rolled back.
-- Autocommit (default): each statement is standalone
UPDATE accounts SET balance = balance - 100 WHERE id = 1; -- auto-committed
UPDATE accounts SET balance = balance + 100 WHERE id = 2; -- if this fails, the debit is already permanent
-- Explicit transaction: both succeed or both fail
BEGIN;
UPDATE accounts SET balance = balance - 100 WHERE id = 1;
UPDATE accounts SET balance = balance + 100 WHERE id = 2;
COMMIT;
-- Disable autocommit for the session (psql behavior)
SET autocommit = off;Write-Ahead Logging (WAL)
Durability is not magic — it is engineering. Postgres (and InnoDB, SQL Server) use a Write-Ahead Log: before any data page is modified on disk, a record describing the change is appended to a sequential log and fsync'd to disk. The log is the source of truth; data pages are updated lazily by background processes.
- Speed — appending to a sequential log is orders of magnitude faster than random writes to data pages.
- Crash recovery — on restart, replay the log forward from the last checkpoint to restore consistency.
- Replication — standby servers stream the WAL and apply changes, keeping replicas in sync.
A transaction is "committed" the moment its commit record is durably written to the WAL — not when data pages are updated. This is why COMMIT can be fast even for large transactions.
Two-Phase Commit (2PC)
When a transaction spans multiple databases or services, a simple BEGIN / COMMIT is not enough — one node could crash after another has already committed. Two-Phase Commit solves this with a coordinator.
-- Phase 1: Prepare
-- Coordinator asks all participants: "Can you commit?"
-- Each participant writes a prepare record to its own WAL and replies YES/NO.
PREPARE TRANSACTION 'txn-uuid-123';
-- Phase 2: Commit or Abort
-- If all participants said YES, coordinator sends COMMIT.
-- If any said NO, coordinator sends ROLLBACK.
COMMIT PREPARED 'txn-uuid-123';
-- or: ROLLBACK PREPARED 'txn-uuid-123';- Phase 1 (Prepare) — all participants durably log their intent and acquire locks. No participant can unilaterally abort after this point.
- Phase 2 (Commit/Abort) — the coordinator makes the final decision and broadcasts it. Participants obey.
- The catch — if the coordinator crashes after Phase 1, transactions can be left in-doubt, holding locks indefinitely until the coordinator recovers. This is why modern distributed systems often prefer Saga patterns or eventual consistency over 2PC.
Advisory Locks — Application-Level Coordination
Sometimes you need mutual exclusion that has nothing to do with rows — e.g., "only one process should run the nightly report." Postgres advisory locks are perfect for this: they are lightweight, transactional or session-scoped, and never block the autovacuum.
-- Session-level: held until explicitly released or session ends
SELECT pg_advisory_lock(42); -- blocks until acquired
SELECT pg_advisory_unlock(42); -- release it
-- Try-lock: returns true/false immediately instead of blocking
SELECT pg_try_advisory_lock(42);
-- Transaction-level: auto-released at COMMIT/ROLLBACK
SELECT pg_advisory_xact_lock(42);
-- Common pattern: lock on a hash of a business key
SELECT pg_advisory_xact_lock(hashtextext('report:nightly'));
-- Use SKIP LOCKED + advisory locks for robust job queues
UPDATE jobs
SET worker_pid = pg_backend_pid(), started_at = now()
WHERE id = (
SELECT id FROM jobs
WHERE status = 'pending'
ORDER BY created_at
FOR UPDATE SKIP LOCKED
LIMIT 1
)
RETURNING *;Read-Only Transactions
Marking a transaction as read-only tells the database it will not write. This enables optimizations: no WAL records for your work, no lock conflicts with writers, and the engine can use a slightly stale snapshot on some replicas.
BEGIN TRANSACTION READ ONLY ISOLATION LEVEL REPEATABLE READ;
-- Run complex reporting queries here with a stable, consistent snapshot
COMMIT;
-- Postgres: set default for a long-running analytics connection
SET default_transaction_read_only = on;Transaction Monitoring & Diagnostics
Long-running transactions are a leading cause of table bloat, replication lag, and lock contention. Know how to find them.
-- Postgres: find long-running transactions
SELECT pid, usename, application_name, state, query_start, now() - query_start AS duration, query
FROM pg_stat_activity
WHERE state = 'active'
AND now() - query_start > interval '5 minutes'
ORDER BY duration DESC;
-- Postgres: find idle-in-transaction connections (often worse than active!)
SELECT pid, usename, now() - xact_start AS xact_duration, query
FROM pg_stat_activity
WHERE state = 'idle in transaction'
ORDER BY xact_duration DESC;
-- Postgres: what is blocking what?
SELECT blocked_locks.pid AS blocked_pid,
blocked_activity.usename AS blocked_user,
blocking_locks.pid AS blocking_pid,
blocking_activity.usename AS blocking_user,
blocked_activity.query AS blocked_statement,
blocking_activity.query AS blocking_statement
FROM pg_catalog.pg_locks blocked_locks
JOIN pg_catalog.pg_stat_activity blocked_activity ON blocked_activity.pid = blocked_locks.pid
JOIN pg_catalog.pg_locks blocking_locks
ON blocking_locks.locktype = blocked_locks.locktype
AND blocking_locks.relation = blocked_locks.relation
AND blocking_locks.pid != blocked_locks.pid
JOIN pg_catalog.pg_stat_activity blocking_activity ON blocking_activity.pid = blocking_locks.pid
WHERE NOT blocked_locks.granted;Real-World Case Study: E-Commerce Inventory
Consider a flash sale: 10,000 users try to buy 100 remaining units. A naive implementation creates lost updates or overselling. Here is a correct pattern.
-- WRONG: read, check, then update (lost update / oversell)
SELECT stock FROM products WHERE id = 99; -- reads 100
-- app checks stock > 0
UPDATE products SET stock = stock - 1 WHERE id = 99; -- race condition!-- CORRECT: atomic decrement with a condition
UPDATE products
SET stock = stock - 1
WHERE id = 99 AND stock > 0
RETURNING stock;
-- If 0 rows returned, the item is sold out. No race, no oversell.-- CORRECT (with reservation): pessimistic lock during checkout flow
BEGIN;
-- Reserve the item for this user for 15 minutes
UPDATE products
SET reserved = reserved + 1, stock = stock - 1
WHERE id = 99 AND stock > 0
RETURNING *;
INSERT INTO reservations (product_id, user_id, expires_at)
VALUES (99, $user_id, now() + interval '15 minutes');
COMMIT;
-- A cron job releases expired reservations back to stock
UPDATE products
SET stock = stock + r.amount, reserved = reserved - r.amount
FROM (
SELECT product_id, count(*) AS amount
FROM reservations
WHERE expires_at < now()
GROUP BY product_id
) r
WHERE products.id = r.product_id;Retry Strategies for Serialization Failures
Under SERIALIZABLE or high contention, the database may abort your transaction with SQLSTATE 40001 (serialization failure) or 40P01 (deadlock detected). Your application must retry.
import time
import random
def run_with_retry(conn, operation, max_attempts=5):
for attempt in range(1, max_attempts + 1):
try:
with conn.begin(): class=class="str">"com"># BEGIN / COMMIT block
return operation(conn) class=class="str">"com"># your business logic
except Exception as e:
if class="str">"40001" not in str(e) and class="str">"40P01" not in str(e):
raise class=class="str">"com"># not a retryable error
if attempt == max_attempts:
raise
class=class="str">"com"># Exponential backoff + jitter
sleep = (2 ** attempt) * 0.01 + random.uniform(0, 0.05)
time.sleep(sleep)
class=class="str">"com"># Usage
run_with_retry(pool, lambda c: c.execute(
class="str">"UPDATE inventory SET count = count - 1 WHERE sku = %s", (sku,)
))Snapshot Isolation Deep Dive
Snapshot Isolation is what Postgres REPEATABLE READ actually implements (and SQL Server's SNAPSHOT). It guarantees: every transaction sees a consistent snapshot of the database as of its start time, and no transaction sees uncommitted changes.
The anomaly it does not prevent is write skew: two transactions read overlapping snapshots, make disjoint writes, and both commit — each write is valid against the snapshot, but the combined result violates a business invariant.
-- Write skew example: doctor on-call constraint (at least one must be on call)
-- Transaction A and B both start under Snapshot Isolation.
-- T1 reads: Alice=true, Bob=true
-- T2 reads: Alice=true, Bob=true
-- T1: set Alice=false (valid because Bob is still true in T1's snapshot)
-- T2: set Bob=false (valid because Alice is still true in T2's snapshot)
-- Both COMMIT. Result: nobody is on call. Invariant violated.
-- Fix 1: use SERIALIZABLE (detects the dependency cycle and aborts one)
-- Fix 2: restructure with an explicit lock on a guard row
SELECT * FROM oncall_guard WHERE hospital_id = 1 FOR UPDATE;
-- Now only one transaction at a time can modify on-call status.Practical Checklist
- Disable autocommit for multi-statement operations; always wrap writes in
BEGIN / COMMIT. - Pick the weakest isolation that prevents the anomalies your code cares about.
- Use
SERIALIZABLEfor money, inventory, and uniqueness invariants you cannot enforce with a constraint. - Handle serialization failures (
40001) and deadlocks (40P01) by retrying with exponential backoff. - Never hold a transaction open across a network call to a slow external service.
- Monitor for
idle in transactionconnections — they hold snapshots and prevent vacuum progress. - Use
SELECT ... FOR UPDATEearly in a transaction, not at the end, and always in a consistent order. - Prefer optimistic concurrency (version columns) for low-contention update patterns.
- Use advisory locks for application-level coordination, not row locks on dummy rows.
- Test retry logic under load — serialization failures only appear under real contention.