System design interview untuk fintech: payment gateway, exactly-once processing, saga pattern, outbox pattern, eventual consistency — lengkap dengan diagram dan trade-off analysis.

System Design Interview

Format

Durasi: 45-60 menit
Tipe: Whiteboard/diagram, bukan coding
Ekspektasi: Kamu bisa desain sistem dari high-level sampai detail

Framework Menjawab

1. Clarify Requirements (5 menit)

Functional: Apa yang sistem ini lakukan? Siapa usernya?
Non-functional: Berapa TPS? Latency budget? Availability?
Constraints: Compliance (PCI-DSS, GDPR)? SLA?

2. Back-of-Envelope Estimation (5 menit)

Traffic: X requests/day → Y QPS
Storage: X bytes/record → Y TB/month
Bandwidth: X bytes/response → Y Mbps

3. High-Level Design (10 menit)

Database choice (SQL vs NoSQL vs cache)
API design (REST vs gRPC vs message queue)
Service boundaries

4. Deep Dive (15-20 menit)

Data model & schema
Core algorithm / flow diagram
Failure modes & handling
Scalability bottlenecks

5. Wrap-Up (5 menit)

Summary
"What would you improve if you had more time?"
Monitoring, alerting, deployment strategy

Soal Klasik: Payment Gateway

Requirements

Functional: Process payment, handle retry, idempotency guarantee
Non-functional: 1000 TPS, p99 < 500ms, 99.99% availability
Constraint: PCI-DSS compliance, no double charge

High-Level Architecture

Client → API Gateway → Payment Service → Payment Processor (Midtrans/Xendit)
                          │
                          ├── Idempotency Store (Redis)
                          ├── Outbox Table (PostgreSQL)
                          ├── Retry Queue (RabbitMQ/Kafka)
                          └── Reconciliation Worker

Core Flow — Happy Path

1. Client POST /payments { idempotency_key, amount, source }
2. Payment Service cek idempotency_key di Redis:
   - Ada → return cached response (idempotent)
   - Gak ada → lanjut
3. INSERT ke outbox table: { id, status: PENDING, ... }
4. Publish event ke Retry Queue
5. Call Payment Processor API
6. UPDATE outbox: status = SUCCESS
7. Cache response di Redis (TTL 24h)
8. Return ke client

Core Flow — Retry & Failure

Scenario: Payment Processor timeout/down

1. Retry Queue consumer pickup event
2. Cek retry_count < max_retries (3x)
3. Exponential backoff: 1s → 2s → 4s
4. Kalau masih gagal → status = FAILED
5. Reconciliation Worker (jalan tiap 30 menit):
   - Query semua PENDING payment > 5 menit
   - Check status ke Payment Processor
   - Update status di DB

Exactly-Once Processing

Kenapa susah: Distributed systems inherently at-least-once. Network failure bisa bikin client retry, tapi payment processor udah proses transaksi pertama.

Solusi — Idempotency Key:

func (s *PaymentService) ProcessPayment(ctx context.Context, req PaymentRequest) (*PaymentResponse, error) {
    // 1. Check cache (Redis) — 1ms
    cached, err := s.cache.Get(ctx, req.IdempotencyKey)
    if err == nil {
        return cached, nil  // Return cached response
    }

    // 2. Acquire distributed lock — mencegah race condition
    lock, err := s.locker.Acquire(ctx, req.IdempotencyKey, 30*time.Second)
    if err != nil {
        return nil, ErrConcurrentRequest  // Another request is processing
    }
    defer lock.Release()

    // 3. Double-check cache (another goroutine might have processed)
    cached, err = s.cache.Get(ctx, req.IdempotencyKey)
    if err == nil {
        return cached, nil
    }

    // 4. Process payment
    resp, err := s.processor.Charge(ctx, req)
    if err != nil {
        return nil, fmt.Errorf("processor.Charge: %w", err)
    }

    // 5. Cache response
    s.cache.Set(ctx, req.IdempotencyKey, resp, 24*time.Hour)

    return resp, nil
}

Outbox Pattern

Kenapa: Jangan publish event ke queue SEBELUM transaksi DB commit. Kalau DB rollback, event udah terkirim → inconsistent.

Instead of:
  DB.INSERT(payment)
  QUEUE.PUBLISH(payment_event)  ← Kalau ini gagal? Atau DB rollback?

Use:
  DB.INSERT(payment)
  DB.INSERT(outbox: { event_type, payload, status: PENDING })  ← SAME transaction!

  Outbox Worker:
    SELECT * FROM outbox WHERE status = PENDING
    QUEUE.PUBLISH(event)
    UPDATE outbox SET status = SENT

Database Schema

CREATE TABLE payments (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    idempotency_key VARCHAR(64) UNIQUE NOT NULL,
    amount BIGINT NOT NULL,           -- dalam cents (hindari float)
    currency VARCHAR(3) NOT NULL,     -- ISO 4217
    status VARCHAR(20) NOT NULL,      -- PENDING, SUCCESS, FAILED
    payment_method VARCHAR(50),
    processor_reference VARCHAR(255), -- reference dari Midtrans/Xendit
    created_at TIMESTAMPTZ DEFAULT NOW(),
    updated_at TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX idx_payments_idempotency ON payments(idempotency_key);
CREATE INDEX idx_payments_status ON payments(status) WHERE status = 'PENDING';

CREATE TABLE outbox_events (
    id BIGSERIAL PRIMARY KEY,
    aggregate_id UUID NOT NULL,
    event_type VARCHAR(100) NOT NULL,
    payload JSONB NOT NULL,
    status VARCHAR(20) DEFAULT 'PENDING',
    created_at TIMESTAMPTZ DEFAULT NOW()
);

Topik Lanjutan yang Sering Ditanya

Saga Pattern

Kalau payment melibatkan multiple service (misal: deduct balance + send notification + update ledger), saga pattern maintain consistency tanpa distributed transaction.

Choreography: Setiap service publish event, service lain listen
Orchestration: Central coordinator yang atur urutan langkah + compensate kalau gagal

Database Sharding

Untuk scale write di atas 10K TPS: shard by merchant_id.

Trade-off: query cross-shard jadi mahal. Solusi: summary table atau async aggregation.

Rate Limiting di Gateway

Jangan cuma rate limit di service — API gateway HARUS punya rate limiting:

Per API key
Per IP
Per endpoint

Token bucket algorithm paling cocok: allow burst, smooth traffic.

Monitoring & Alerting

Golden metrics:

Latency: p50, p95, p99 per endpoint
Traffic: request rate per second
Errors: error rate (4xx + 5xx)
Saturation: goroutine count, DB connection pool, Redis connection pool

Alert:

Payment success rate < 99% → P1
Payment latency p99 > 1s → P2
Outbox pending > 100 events → P2
Dead letter queue not empty → P1

System Design Interview — Fintech Focus