Skip to content
Md. Moudud Hassan
rss

12 min readArchitecture

The Saga Pattern in Java: Distributed Transactions Without Two-Phase Commit

The first time a team splits a monolith along service boundaries, someone eventually asks the question that has no comfortable answer: what happens if the payment succeeds and the inventory reservation fails?

In the monolith this was not a question. One transaction, one database, ROLLBACK. Across services it becomes the central design problem, and the honest answer is that you cannot have the guarantee you used to have. What you can do is choose deliberately what to replace it with.

Why not just use XA

Two-phase commit exists and Java has supported it for decades through JTA. It is tempting precisely because it promises the semantics you lost.

It is the wrong tool here, for reasons that are structural rather than a matter of taste.

XA holds locks across the entire protocol. A distributed transaction spanning three services keeps rows locked in all three until every participant has voted and the coordinator has decided. That window includes network latency between services, so lock duration is now a function of your slowest participant’s health. Under load, that becomes lock contention that no single service can diagnose from its own metrics.

Worse is the coordinator. If it fails after participants have voted to commit but before it tells them the outcome, those participants are stuck holding locks with no idea what to do — the classic blocking problem. Solving it properly means a highly available, persistent coordinator, which is a piece of infrastructure whose failure modes you now own.

And practically, much of what you integrate with cannot participate anyway. A payment gateway’s REST API has no notion of preparing to commit. Kafka is not an XA resource in any useful sense. The moment one participant cannot vote, the guarantee is gone regardless.

So: sagas. A saga replaces one distributed transaction with a sequence of local ones, each committing independently, and each paired with a compensating action that semantically undoes it if a later step fails.

Choreography or orchestration

Two ways to structure it, and the choice matters more as the saga grows.

Choreography has no central coordinator. Each service listens for events and emits its own. Order Service publishes OrderCreated; Payment Service consumes it and publishes PaymentCompleted; Inventory consumes that, and so on. Services are decoupled and there is no single point of failure.

The cost shows up around the fourth or fifth step. There is no single place that describes the business process — the flow exists only as an emergent property of who happens to subscribe to what. Answering “what happens when a refund fails” means reading five codebases. Cyclic dependencies creep in unnoticed. I would use choreography for short flows of two or three steps with genuinely independent services, and not much beyond that.

Orchestration puts one component in charge. The orchestrator holds saga state, decides the next step, issues commands, and handles failures. The flow is written down in one place, which means you can read it, test it, and show it to someone who does not write code.

For anything with real business consequence — payments, land records, licensing, anything a regulator might ask about — orchestration wins, for a reason that has nothing to do with elegance: you can answer questions about it. When a citizen calls asking why their application is stuck, “let me query the saga state” is a very different conversation from “let me correlate logs across five services”.

The trade is a component that must be highly available and whose state must be durable. That is a real cost, but a well-understood one.

An orchestrated saga in practice

The shape that has served me well is a persisted state machine. State lives in the orchestrator’s own database, updated in the same local transaction as the decision to move forward.

@Entity
@Table(name = "order_saga")
public class OrderSaga {

    @Id
    private UUID sagaId;

    private UUID orderId;

    @Enumerated(EnumType.STRING)
    private SagaState state;

    @Enumerated(EnumType.STRING)
    private SagaState compensatingFrom;   // null unless unwinding

    private Instant startedAt;
    private Instant stateChangedAt;       // drives the stuck-saga sweeper
    private int attempts;
    private String failureReason;

    @Version
    private long version;                 // optimistic locking; see below
}

public enum SagaState {
    STARTED,
    PAYMENT_PENDING,   PAYMENT_COMPLETED,
    INVENTORY_PENDING, INVENTORY_RESERVED,
    SHIPPING_PENDING,  COMPLETED,
    COMPENSATING,      FAILED
}

The @Version field is not incidental. Sagas are advanced by message handlers, and messages get redelivered and processed concurrently. Optimistic locking is what stops two deliveries of PaymentCompleted from both advancing the saga and dispatching the inventory command twice.

The orchestrator is then a handler per inbound event:

@Component
@RequiredArgsConstructor
public class OrderSagaOrchestrator {

    private final OrderSagaRepository sagas;
    private final CommandPublisher commands;

    @Transactional
    public void on(PaymentCompleted event) {
        OrderSaga saga = sagas.findByOrderIdForUpdate(event.orderId());

        // Idempotency: a redelivery finds the saga already past this state.
        if (saga.getState() != SagaState.PAYMENT_PENDING) {
            log.debug("Ignoring {} for saga {} in state {}",
                      event.getClass().getSimpleName(), saga.getSagaId(), saga.getState());
            return;
        }

        saga.transitionTo(SagaState.INVENTORY_PENDING);
        commands.send(new ReserveInventory(saga.getOrderId(), saga.getSagaId()));
    }

    @Transactional
    public void on(InventoryReservationFailed event) {
        OrderSaga saga = sagas.findByOrderIdForUpdate(event.orderId());
        if (saga.getState() != SagaState.INVENTORY_PENDING) return;

        saga.beginCompensation(event.reason());
        // Unwind in reverse: payment was the last thing that succeeded.
        commands.send(new RefundPayment(saga.getOrderId(), saga.getSagaId()));
    }
}

The guard clause at the top of each handler is the whole idempotency story, and it is deliberately boring. Rather than tracking processed message IDs separately, the saga’s own state answers the question “have I already handled this?” — because a saga in INVENTORY_PENDING has demonstrably already processed PaymentCompleted.

Compensations are not rollbacks

This is the conceptual jump that trips teams up, and it is worth being precise about.

A rollback erases history. A compensation is a new business action that offsets a previous one, and it is visible. Refunding a payment does not remove the charge — the customer sees both entries on their statement. Cancelling a booking does not unbook it; it emits a cancellation. Sometimes the compensation cannot fully restore the prior state at all: a notification email cannot be unsent.

Three consequences follow, and all three need designing for.

Compensations can fail. Your refund call can time out. Retry with backoff, and when retries are exhausted, escalate — a dead-letter queue plus an alert plus a manual resolution path. Money stuck in an intermediate state is a business problem before it is a technical one, and someone needs to be told.

Compensations must be idempotent. They will be retried. A refund handler that issues a second refund because it did not recognise a redelivery has turned a reliability mechanism into financial loss. Use an idempotency key derived from the saga, and let the database enforce it:

@Transactional
public void refund(RefundPayment cmd) {
    // The unique constraint is the actual guarantee. The check is an optimisation.
    if (refunds.existsBySagaId(cmd.sagaId())) return;

    Refund refund = gateway.refund(cmd.orderId(), cmd.amount(), cmd.sagaId().toString());
    refunds.save(refund);   // UNIQUE (saga_id)
}

Note that the gateway call takes the saga ID as its idempotency key too. Most payment providers support this, and it is the difference between “we probably won’t double-refund” and “we cannot double-refund”.

Some steps cannot be compensated. Order the saga so irreversible steps come last. If shipping physically dispatches goods, everything reversible — payment authorisation, inventory reservation — should have committed and been verified before that point. This is a design constraint on the business process, not a coding detail, and it is worth raising with product people explicitly.

The dual-write problem, and the outbox

Here is the bug that will find you in production regardless of how careful the saga logic is.

@Transactional
public void createOrder(CreateOrder cmd) {
    orders.save(new Order(cmd));                       // database
    kafka.send("orders", new OrderCreated(cmd.id()));  // broker — NOT in the transaction
}

Two systems, no shared transaction. If the broker publish fails after the commit, the order exists and no saga starts — it sits forever in a pending state. If the process dies between the two, same result. If you reverse the order, you can publish an event for an order that was never persisted, which is worse.

Wrapping this in @Transactional does nothing: Kafka is not enrolled in the database transaction. This is not a race condition to be narrowed, it is a correctness gap.

The transactional outbox closes it. Write the event to a table in the same local transaction as the business data. A separate process reads the table and publishes.

@Transactional
public void createOrder(CreateOrder cmd) {
    orders.save(new Order(cmd));
    outbox.save(OutboxEvent.of("Order", cmd.id(), new OrderCreated(cmd.id())));
}
CREATE TABLE outbox_event (
    id             BIGINT PRIMARY KEY AUTO_INCREMENT,
    aggregate_type VARCHAR(64)  NOT NULL,
    aggregate_id   VARCHAR(64)  NOT NULL,
    event_type     VARCHAR(128) NOT NULL,
    payload        JSON         NOT NULL,
    created_at     TIMESTAMP(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
    published_at   TIMESTAMP(6) NULL,
    INDEX idx_unpublished (published_at, id)
);

Now one atomic commit covers both. Either the order and its event are durable together, or neither is.

For the relay, change data capture — Debezium reading the MySQL binlog — is the better option at scale: no polling, no added load, and it captures the commit order faithfully. A polling publisher is simpler and perfectly adequate for moderate volume, provided you remember it delivers at least once. Which brings us back to idempotent consumers: the outbox does not remove that requirement, it makes it mandatory.

The anomaly nobody warns you about

Sagas give up isolation, and this is the part that gets skipped in most explanations.

Between a saga’s first commit and its last, other transactions can see partial results. Classic ACID would have hidden that. Two failure modes follow:

Dirty reads. Another transaction reads data the saga will later compensate away. A customer’s balance shows a charge that gets refunded a second later; a report includes an order that never completes.

Lost updates. A saga writes based on a value it read earlier, while another transaction has changed it in between. The saga’s compensation then restores a value that was never correct.

There are established countermeasures, and the useful ones in my experience:

  • Semantic lock. Mark records in an explicit pending state — PAYMENT_PENDING rather than a boolean — so other transactions can see the saga is mid-flight and decide what to do. This is the most broadly useful of the set, and cheap.
  • Commutative updates. Prefer operations whose order does not matter. balance = balance - 100 composes safely; balance = 400 does not.
  • Reread value. Re-read and verify before writing, as an optimistic-locking check. Cheap protection against lost updates.
  • By value. Route by business risk: use a saga for a £10 order, and a stricter mechanism for a £10 million transfer. Not every operation deserves the same consistency budget.

The semantic lock has a side benefit worth mentioning: it makes the intermediate state visible in your data model. When something goes wrong, “show me every order stuck in PAYMENT_PENDING for more than ten minutes” is a query, not an investigation.

Sagas get stuck. Plan for it.

A saga waiting on a reply that never arrives will wait forever unless you build the way out. Every step needs a deadline, and something has to enforce it.

@Scheduled(fixedDelay = 60_000)
@Transactional
public void sweepStuckSagas() {
    Instant cutoff = Instant.now().minus(Duration.ofMinutes(15));

    for (OrderSaga saga : sagas.findStuck(cutoff)) {
        log.warn("Saga {} stuck in {} since {}",
                 saga.getSagaId(), saga.getState(), saga.getStateChangedAt());

        if (saga.getAttempts() < 3) {
            retryCurrentStep(saga);
        } else {
            saga.beginCompensation("Timed out in " + saga.getState());
            compensate(saga);
        }
    }
}

Two things make this work in practice. Timeouts belong in the saga’s state, not in the messaging layer — a broker redelivery is not the same thing as a business step exceeding its allowed duration, and conflating them produces surprising behaviour. And every escalation path must end somewhere a human can see it. A saga that has exhausted retries and failed compensation is not a technical event; it is a customer whose money is in limbo.

What I would reach for

Hand-rolling an orchestrator is roughly a week of work and entirely reasonable for one or two sagas — you get exactly the semantics you want and no framework to learn. Past that, the state management, timeout handling and observability start to justify something purpose-built.

Temporal is what I would look at first for complex, long-running workflows. It handles durable execution properly, so the workflow reads like sequential code while surviving process restarts, and the visibility tooling is genuinely good. The cost is running Temporal, which is not trivial.

Axon Framework fits naturally if you are already committed to event sourcing and CQRS. If you are not, adopting it for sagas means adopting a much larger set of ideas than you asked for.

Camunda or another BPMN engine is worth serious consideration when the process is genuinely a business process that non-engineers need to see and change. In government and enterprise contexts that visibility can be the deciding factor, well ahead of any technical criterion.

The framework is the smaller decision. Whichever you pick, the same problems remain yours: compensations that are correct and idempotent, an outbox for the dual-write, deliberate handling of isolation anomalies, and timeouts with a human escalation path. No library solves those, because they are properties of your business process rather than your code.

The honest summary

A saga trades atomicity for availability. That is a real trade, not a free lunch, and the resulting system is genuinely more complex than the monolith it replaced.

Which is why the most useful question is not “how do I implement a saga” but “do these operations need to be in separate services at all?” Data that must change together is often evidence of a boundary drawn in the wrong place. Redrawing it so a single service owns the transaction is not a retreat — it is usually the better design, and it costs nothing to implement.

Reach for a saga when the boundaries are genuinely right and the operations genuinely span them. Then take the compensations, the outbox and the isolation anomalies seriously, because that is where the real engineering lives — not in the happy path, which is the easy part and the part every tutorial shows you.