Staff Distributed Systems Engineer — Real-Time Financial Infrastructure
Harrison Clarke is working with a category-defining fintech company — one rebuilding the infrastructure layer beneath modern capital markets — to hire a Staff Distributed Systems Engineer who will architect and operate systems where a misplaced millisecond is money, and a dropped message is a regulatory event.
This is distributed systems at its most consequential. Not eventual consistency for a social feed — strong consistency guarantees across a globally distributed financial system processing billions of dollars in daily throughput.
Most distributed systems roles tolerate trade-offs between consistency, availability, and partition tolerance with a shrug and a retry. Financial infrastructure doesn't get that luxury. Your system needs to be exactly-once in a world that's at-least-once, auditable to the transaction, and compliant across jurisdictions — all while matching the throughput expectations of a modern tech company, not a legacy bank. The companies winning this space are replacing 40-year-old COBOL plumbing with systems designed from first principles, and they need the engineers who can actually do it.
What You'll Own
- Core transaction processing systems — designing and evolving the distributed architecture that handles order matching, settlement, ledger reconciliation, and position management with zero tolerance for data loss or inconsistency
- Event-driven infrastructure at scale — building and operating high-throughput event streaming platforms (Kafka, Pulsar, or custom) that serve as the central nervous system across trading, risk, and compliance domains
- Consensus and coordination — implementing distributed consensus protocols, leader election, and distributed locking for systems where split-brain isn't a hypothetical — it's a P0 you prevent by design
- Low-latency networking and serialization — optimizing inter-service communication, wire protocols, and data serialization (Protobuf, FlatBuffers, custom binary formats) for microsecond-sensitive paths
- Storage engine design — building or integrating purpose-built storage layers for time-series financial data, audit logs, and real-time materialized views across distributed state
- Operational resilience — chaos engineering, failure injection, capacity modeling, and graceful degradation patterns for systems that regulators expect to never go down
What You Bring
- 8+ years of software engineering experience with deep, hands-on distributed systems expertise — not just using distributed databases, but understanding the theory and trade-offs beneath them
- Strong command of distributed systems fundamentals — consensus algorithms (Raft, Paxos, or variants), replication strategies, vector clocks, CRDTs, distributed transactions, and the practical limits of the CAP theorem
- Production experience designing and operating high-throughput, low-latency systems — you've built systems where p99 is measured in single-digit milliseconds or lower
- Mastery of at least one systems language — Java, Go, Rust, or C++ — with a strong opinion on when each is the right tool
- Deep experience with event streaming infrastructure (Kafka, Pulsar, Kinesis, or custom) at serious scale, including exactly-once semantics, partitioning strategies, and backpressure management
- Familiarity with distributed storage engines — you've worked with or contributed to systems like CockroachDB, TiDB, FoundationDB, Vitess, ScyllaDB, or comparable
Nice to Have
- Previous experience in financial services, capital markets, or payments infrastructure — understanding of order books, settlement cycles, ledger design, or clearing systems
- Background in database internals — B-trees, LSM trees, WAL design, MVCC, transaction isolation levels implemented (not just configured)
- Familiarity with regulatory and compliance frameworks (SOC 2, PCI-DSS, MiFID II, SEC reporting) and how they shape systems architecture
- Experience with multi-region active-active architectures and the coordination headaches that come with them
- Contributions to open-source distributed systems projects or published work on distributed systems topics
Why This Opportunity
- Greenfield with gravity — you're not patching legacy systems; you're designing from scratch, but for a domain where the requirements are unforgiving and the problems are intellectually deep
- Staff-level influence — this is a technical leadership seat. You'll set architectural direction, mentor senior engineers, and make decisions that shape the platform for years
- Rare problem density — the intersection of ultra-low-latency, strong consistency, regulatory auditability, and global distribution is one of the richest problem spaces in systems engineering. Most companies only deal with one or two of those constraints. You'll handle all of them simultaneously
- Meaningful scale, real stakes — billions in daily volume, real money, real counterparties. The systems you build will be load-bearing infrastructure for modern financial markets