Two ways to pay, one payment: idempotency in Request to Pay

At DPL I led delivery of Request to Pay, which 4,000+ merchants went on to adopt. The merchant enters the customer's MSISDN, a name-check API confirms who it is, they set the amount and send. The customer then receives the request twice: as a push notification in the app, and as a USSD prompt.

Sending it down both channels is the point. The push is convenient on a smartphone; the USSD prompt reaches phones without data. But two channels are also two chances to pay, and the whole feature rests on one of them always losing.

The failure you're designing against

A customer opens the notification and pays in the app. The USSD prompt is still on their phone, so they answer that too. Or the network is slow, and they try again. Either way, two payment attempts for the same request arrive close together, and the obvious code lets both through:

var request = await db.PaymentRequests.FindAsync(requestId);
if (request.Status != RequestStatus.Pending)       // both callers see Pending...
    throw new BusinessException("REQUEST_ALREADY_PAID");

request.Status = RequestStatus.Paid;               // ...and both pay
await db.SaveChangesAsync();

This is check-then-act. Between the read and the write there is a window, and under real traffic someone will land in it.

Make the state change the check

The fix is to stop asking whether the request is still pending and then acting on the answer. Ask the database to move it from Pending to Paid only if it is still Pending, in one statement, and look at how many rows changed. One row: this caller won. Zero rows: someone already paid.

public async Task PayAsync(Guid requestId, PaymentChannel channel, CancellationToken ct)
{
    await using var tx = await db.Database.BeginTransactionAsync(ct);

    // The check and the state change are one statement, so only one caller can win.
    var claimed = await db.PaymentRequests
        .Where(r => r.Id == requestId && r.Status == RequestStatus.Pending)
        .ExecuteUpdateAsync(s => s
            .SetProperty(r => r.Status, RequestStatus.Paid)
            .SetProperty(r => r.PaidVia, channel), ct);

    if (claimed == 0)
        throw new BusinessException("REQUEST_ALREADY_PAID");

    await wallet.DebitAsync(requestId, ct);   // inside the same transaction as the claim
    await tx.CommitAsync(ct);
}

That sketch assumes the debit and the request live in the same database, so one transaction covers both. When the money moves in another system, claim the request first (Pending to Processing), make the payment, then finish it (Processing to Paid), and release the claim if the payment fails. The rule doesn't change: the transition itself is the lock.

"Already paid" is an answer, not an error

The losing attempt isn't a bug, and it shouldn't look like one. It's a business exception with a clear message: this request has already been paid. The customer who paid in the app and then answered the USSD prompt should be told exactly that, not shown a generic failure that makes them wonder whether they paid twice.

The callback is a second writer

USSD payments finish asynchronously. Once the customer confirms, a callback arrives and updates the request record in the database. That makes the callback a writer too, and callbacks get retried: timeouts, redeliveries, a gateway being careful. So the callback uses the same conditional transition. A repeated callback finds nothing left to update, and should still answer with success, because from the gateway's point of view the job is done.

What to take from it

• Every channel goes through one transition. App, USSD, and callback all use the same conditional update, never their own copy of the rules.
• Let the database arbitrate. A conditional update or a unique constraint beats any check you do in application code.
• Design the loser's experience. "Already paid" is the most reassuring thing you can tell someone who tried twice.
• Assume every callback arrives more than once.