Faisalaffan Blog
|
🇬🇧
Faisalaffan Blog
Career8 min read

What I Did as a Mid-Level Engineer — Shipping, Ownership & Production Scars

I shipped real features to real users. I broke production and fixed it alone at 2 AM. I built a mobile banking app with 680K downloads, a mining fleet system saving $200K/year, and a field operations platform for 4.1M users. This is what mid-level engineering actually looks like — the projects, the metrics, and the production incidents that made me a real engineer.

Faisal Affan
Faisal Affan
Published
What I Did as a Mid-Level Engineer — Shipping, Ownership & Production Scars

What I Did as a Mid-Level Engineer

"Mid-level is where you stop writing code that works and start writing code that survives." — Me, after the 14th production rollback

The mid-level years transform you from "someone who can code" to "someone who can ship." The shift is fundamental: you stop being evaluated on whether your code runs. You're evaluated on whether your code survives production — at scale, under load, when things go wrong at 3 AM.


The Mid-Level Proposition

What Changed

Before (Junior): "Can you write code that works?" Now (Mid): "Can you own a feature end-to-end — design, build, test, deploy, monitor, debug, and iterate — without someone holding your hand?"


Case Study 1: SiTepat — Mobile Banking Integration

Role: Solo Backend Engineer Stack: Go, Flutter, ISO 8583, PostgreSQL Scale: 680K+ Google Play downloads, zero downtime during Eid transaction rush

This was my first "real" production system. Not a WordPress site. Not a CRUD app. A mobile banking application that had to integrate directly with a bank's core system via ISO 8583 protocol.

What I Built

  • ISO 8583 Bridge: A Go service that translated REST API calls into ISO 8583 messages for the bank's legacy mainframe. This was the critical path — if this bridge went down, 680K users couldn't check their balance.
  • Real-Time Financing Dashboard: Field agents could simulate IIR (Internal Interest Rate) calculations in real-time, replacing a manual Excel process that took 3-5 business days.
  • Smart Payment Reminder Engine: Automated reminders via Firebase Cloud Messaging, reducing late payments by 18%.

What Went Wrong

Production Incident #1: The ISO 8583 Timeout

What happened: During the Eid al-Fitr holiday (highest transaction volume of the year), the ISO 8583 bridge started timing out on 30% of requests. The bank's mainframe was responding correctly — our bridge was the bottleneck.

Root cause: We were opening a new TCP connection for every ISO message. At 500+ concurrent requests, the connection pool exhausted. The fix was implementing persistent connections with health checks.

Lesson: Protocol-level integration isn't "just another API." ISO 8583 has quirks around connection management, byte ordering, and field encoding that will destroy you if you treat it like REST.

Production Incident #2: The Firebase Token Expiry

What happened: Payment reminders stopped sending. 0% delivery rate for 6 hours. Users missed payment deadlines and were charged late fees.

Root cause: Firebase Cloud Messaging tokens expire every 30 days. We stored tokens on first login but never refreshed them. Simple fix — but took 6 hours to diagnose because the error logs were buried in Firebase's dashboard, not our monitoring.

Lesson: External service dependencies need health checks in YOUR monitoring, not theirs. If you depend on Firebase, monitor Firebase.

Impact

Metric Before After
App downloads 0 680K+
Transaction processing time 3-5 business days (manual) Real-time
Late payment rate 22% 18%
System uptime N/A (new system) 99.7%

Case Study 2: HPU Digital Mining Platform

Role: Full-Stack Engineer Stack: Go microservices, React PWA, MQTT, PostgreSQL Scale: 3 provinces, 100+ trucks, 50K telemetry events/second

This was my introduction to IoT — and everything I thought I knew about "reliable systems" got tested against reality: remote mining sites with intermittent satellite connectivity, GPS trackers that report garbage data, and users who've never used a smartphone before.

What I Built

  • Real-Time Fleet Management: Live GPS tracking for 100+ dump trucks, excavators, and support vehicles
  • Fuel Management & Anomaly Detection: Caught fuel theft within the first month of rollout
  • HSE Digital Management: Replaced paper-based safety reporting with a mobile PWA
  • Multi-Site Command Center Dashboard: Real-time visibility from pit to HQ with drill-down to individual truck telemetry

What Went Wrong

IoT Lesson #1: GPS Trackers Lie

We bought 50 GPS trackers from a reputable vendor. 12 of them reported coordinates that were physically impossible — trucks "teleporting" 500 meters in 2 seconds. After 2 weeks of debugging our code, we discovered: the trackers were configured to report at 1Hz (once per second), but when they lost satellite lock, they'd buffer readings and dump them all at once when reconnected. Our system interpreted the burst as real-time data.

Fix: Server-side velocity validation. If a coordinate change implies >120 km/h for a mining truck, flag it and use the last known good position. Saved us from chasing phantom trucks for a month.

IoT Lesson #2: The Edge Has No Internet

Mining sites in remote Kalimantan have connectivity that makes 3G look like fiber. We designed for "eventual connectivity" — MQTT with QoS 1, local SQLite buffer on edge gateways, batch upload when the satellite link comes back. The architecture worked, but we underestimated how long the "offline window" actually was: sometimes 6-8 hours.

Lesson: Design for the worst-case offline window, not the average. Your users won't care that "on average" connectivity is good if their 8-hour shift ends without data sync.

Impact

Metric Before After
Truck utilization 65% 79%
Fuel theft incidents Unknown (undetected) Detected in month one
Annual savings — Rp 3.2B ($200K)
HSE report turnaround 5 days (paper) Real-time (digital)

Case Study 3: Tepati — Field Operations Platform

Role: Solo Full-Stack Engineer Stack: Go monolith, Flutter mobile, SQLite, MQTT Scale: 4.1M end-users, built solo in 30 days

This was the project that taught me the difference between "building" and "owning." When you're the only developer, there's no backend team to blame when the database is slow. No frontend team to fix the UI. No DevOps team to handle deployment.

What I Built

  • Offline-First Sync Engine: Field agents in areas with zero connectivity could fill forms, capture photos, collect signatures — everything synced when they returned to network range
  • Dynamic Form Engine: 50+ field types with conditional logic, configurable without code changes
  • Multi-Level Approval Pipeline: 4 hierarchy levels with role-based routing and full audit trail
  • IIR Simulation Engine: Real-time financing calculations that previously took 3-5 business days

The Solo Developer Reality

  • You deploy your own code. At 2 AM. On a Saturday.
  • You debug your own production incidents. There's no L2 support.
  • You write your own documentation. If you don't, nobody else will — and you'll regret it in 6 months.
  • You make architectural decisions alone. This is both liberating and terrifying — there's nobody to sanity-check your choices.

Impact

Metric Before After
Data loss rate 23% 0%
End-users served 0 4.1M
Time to build N/A 30 days (MVP)
Financing decision time 3-5 business days Real-time

My Mid-Level Scorecard

Portfolio Item Did I Do It? Notes
2-3 work case studies ✅ SiTepat, HPU, Tepati — all with measurable impact
Measurable metrics ✅ 680K downloads, 65→79% utilization, 23% data loss eliminated
Open source contribution ❌ Biggest career gap — started too late
Technical articles ⚠️ Wrote internal documentation, never published publicly
Production incident handling ✅ Survived 4 major incidents, learned more from them than any feature

Advice for Current Mid-Level Engineers

  • Own the full lifecycle. Don't just write the feature — deploy it, monitor it, debug it, improve it. The engineer who owns the full stack is worth 3x the engineer who only writes code.
  • Measure before and after. "Faster" is an opinion. "API latency dropped from 800ms to 200ms" is a portfolio item. Start measuring today.
  • Write about your production incidents. The bug you fixed at 2 AM is tomorrow's blog post. The architecture decision you made after the incident is next month's conference talk.
  • Start open source now. Even one PR to a project you use. The skills you gain — reading unfamiliar codebases, following contribution guidelines, receiving code review from strangers — are exactly what senior+ interviews test for.

Mid-level is the most underrated career stage. It's where you accumulate the production scars that make your senior portfolio authentic. Every incident, every rollback, every 2 AM fix — that's not trauma. That's content.

Related stories

Career
Career
Every Role I've Played — Junior to CTO, What I Actually Did at Each Level

Every Role I've Played — Junior to CTO, What I Actually Did at Each Level

A candid career retrospective tracing my journey through every engineering role: Junior developer grinding through fundamentals, Mid-level shipping production features, Senior designing architectures, Staff building org-wide leverage, and CTO-level strategic decisions. Each section maps to the portfolio strategy framework with real projects, real metrics, and real lessons.
Faisal Affan
Faisal Affan
Published
Career
What I Did at the CTO Level — Business Strategy Through Technology

What I Did at the CTO Level — Business Strategy Through Technology

At this level, your portfolio isn't about technology — it's about how technology created business value. I reduced AI inference costs by 90%, built an MCP orchestration that 3x'd developer output, and established a FOSS strategy where every repo is a 24/7 portfolio asset. This is what engineering leadership actually delivers.
Faisal Affan
Faisal Affan
Published
Career
What I Did as a Junior Developer — The Fundamentals That Shaped Everything

What I Did as a Junior Developer — The Fundamentals That Shaped Everything

No production experience. GitHub full of tutorial repos. One asset: the ability to learn fast and zero fear of breaking things. This is what I actually built as a junior engineer — WordPress sites, a PHP university system, and the hardcoded credentials mistake that taught me security forever.
Faisal Affan
Faisal Affan
Published
Career
What I Did as a Senior Engineer — Architecture, Trade-Offs & Technical Judgment

What I Did as a Senior Engineer — Architecture, Trade-Offs & Technical Judgment

I stopped being evaluated on lines of code. I was evaluated on decisions — what architecture to use, what trade-offs to accept, and how to guide other developers. This is the real story of designing IoT telemetry for 50K events/sec, microservices for Australia's largest insurer, and the stack decision that became my most-read article.
Faisal Affan
Faisal Affan
Published