One Binary, One Bucket: When to Build Services on Object Storage

You can now ship entire services with a single stateless binary and a bucket. No database. No queue. No block volumes. Just object storage. In the last few weeks, Show HN posts like lightweight Git servers on top of S3-like stores, durable streams over HTTP backed by object storage, and self-hosted libraries that run entirely on buckets have all trended. This is not a novelty hack. It is an architecture pattern that finally has the primitives to be production-grade—if you apply it where it fits.

If you are a CTO trying to cut infra blast radius and vendor risk, this pattern is worth a serious look. But it is not a religion. It is a deliberate trade: you swap sub-10 ms latencies and ACID transactions for extreme durability, portability, and operational simplicity. Below is a decision framework and some hard numbers so you can decide where “one binary, one bucket” belongs in your stack.

Why this is viable now

• Strong consistency arrived. Since 2020, major object stores like Amazon S3 offer strong read-after-write consistency for PUT, DELETE, and LIST. You are no longer coding around eventual consistency for basic CRUD.
• Cost and portability improved. S3 requests are cheap on a per-million basis, Cloudflare R2 removed egress tolls, and S3-compatible APIs (MinIO, R2, Wasabi) make “lift your bucket somewhere else” a real hedge.
• Edge and serverless matured. VPC endpoints, regional edge caches, and per-request compute mean you can keep latency reasonable without a farm of stateful servers.

What “one binary, one bucket” actually means

Patterns you will see:

• Append-only logs. Events are written as immutable objects partitioned by time or shard. Consumers list and read ranges; periodic compaction merges small files.
• Content-addressed blobs. Artifacts, images, model weights, or packfiles addressed by hash; indices are small manifests.
• Snapshot pointers. A tiny “latest” manifest points to immutable objects for the current state; updates are serialized through a single writer or a lease.
• Range-readable formats. Store Parquet, packfiles, or tar-like bundles so you can serve small slices via HTTP Range instead of creating millions of tiny objects.

The common thread is immutability first, compaction second, pointers last.

Where it shines

• Regulated audit trails and WORM requirements. Object Lock and versioning give you write-once, retain-for-N-years semantics out of the box. If you live under SEC 17a-4 style retention or need provable immutability, this is your friend.
• Artifact registries and media libraries. Large, immutable blobs with occasional metadata updates are exactly what buckets are for.
• Low to moderate write rates with bursty reads. 5–50 writes/sec per service with 10–100x read fan-out is comfortable, especially if you batch writes and cache reads.
• Cross-cloud hedge. Want to run the same service on AWS, GCP, or on-prem? S3-compatible APIs plus a stateless binary make portability real.

Where it hurts

• Hot counters and high-contention state. You do not have atomic increments or multi-row transactions. If you need ACID, you still need a database.
• Sub-20 ms tail latency. Typical p50 object GETs land in the tens of milliseconds; p99 can be in the 100–400 ms range. With caching, you can hide some of it, but not all.
• Very high write rates of tiny objects. Per-request costs and metadata overhead can dwarf storage costs if you hammer the bucket with 1000s of objects per second.

The cost math most teams skip

Let’s model the naive and the sane versions of a bucket-backed queue-like service.

Naive object-per-message

• Write rate: 200 messages/sec (moderate).
• Read rate: 1000 reads/sec (5x fan-out).
• Monthly ops: ~518 million writes, ~2.6 billion reads.
• Using typical S3 request prices: $0.005 per 1000 PUT; $0.0004 per 1000 GET.

Cost: writes ≈ 518,400k / 1k × $0.005 = $2,592/month; reads ≈ 2,592,000k / 1k × $0.0004 = $1,037/month. Total ≈ $3,629/month in request charges alone, before storage. That is not catastrophic, but it is wasteful—because you are paying per tiny object.

Sane: Batch and bundle

• Batch writes: 100 messages per object → 2 writes/sec instead of 200.
• Range reads: Read specific messages via offsets in a 8–64 MB bundle; cache bundles at the edge.
• Monthly ops: ~5.18 million writes, reads drop by 10–100x with caching.

Cost: writes ≈ 5,184k / 1k × $0.005 = ~$26/month; reads maybe ~$100–$300/month depending on hit ratio. Net: 10–50x cheaper in request charges. Storage costs barely change.

The lesson: with object storage, granularity is your budget. If you cannot batch, you will pay.

Latency real talk

Assume warm-region access with a VPC endpoint:

• GET p50: 10–30 ms is typical.
• GET p95–p99: 100–400 ms spikes happen. Design for it.
• PUT p50: 10–50 ms; p99 can flirt with a few hundred ms.

If you need single-digit millisecond tail for hot paths, this is the wrong tool. If you can hide the tails behind caches and asynchronous work, you are fine.

Consistency and concurrency model

Today’s major buckets give you strong read-after-write for new objects and overwrites. You can trust a LIST after a PUT. You cannot trust the bucket to be your lock manager. Patterns we have seen work reliably:

• Single-writer per shard with a lease stored in a fast KV (DynamoDB/etcd). The bucket stores immutable objects; the KV guards the tiny mutable pointer.
• Manifest commits. Write your data to a new content-addressed object, then update a small manifest that points at it. Serialize manifest updates through a single writer or lease.
• Idempotent writes via hashing. Use SHA-256 of the payload as the key to avoid duplicates across retries.

Could you attempt compare-and-swap on a pointer object in the bucket? Not portably. Keep your locking in a KV designed for it, and keep your data immutable in the bucket.

Design patterns that survive production

1) Layout for batching and compaction

• Partition by time and shard: e.g., topic=orders/partition=03/dt=2026-08-25/0001.parquet.
• Write small, compact later: Emit 1–8 MB objects during ingest for latency, then compact hourly into 64–256 MB objects to slash request and listing overhead.
• Use range-friendly formats: Parquet, packfiles, tar-like bundles with an index footer so reads can fetch just the needed ranges.

2) A tiny KV for control plane

• Store pointers, leases, and sequence numbers in a KV store with conditional writes. Keep it to a few hundred writes/sec.
• Everything else is immutable and lives in the bucket. If it moves or mutates, you are doing it wrong.

3) Aggressive caching

• Edge CDN for reads with long TTLs and cache-busting via versioned keys.
• Local NVMe cache in the binary for hot bundles and compaction inputs.
• Presigned URLs so clients can download large blobs directly from the bucket without proxying through your binary.

4) Cost guards

• Budget alarms on request counts, not just transfer and storage.
• Abort incomplete multipart uploads automatically; otherwise you will leak storage and money.
• Rotate prefixes to limit object listing cost in huge directories; avoid listing the world on every request.

5) Security and compliance

• Private buckets with VPC endpoints to avoid public egress and reduce attack surface.
• SSE-KMS for encryption with per-tenant keys if you are multi-tenant.
• Object Lock (governance or compliance mode) for retention-critical datasets.
• Access logs and CloudTrail enabled; keep those logs in a separate, locked bucket.

Use cases that actually work

• Git-like content stores. Put packfiles and refs in the bucket; serialize ref updates in a KV. A small binary translates Git wire protocol to bucket ops. This is exactly what several new lightweight Git servers are doing on HN.
• Durable event streams over HTTP. Messages are stored in time-partitioned bundles; consumers page through bundles via Range requests. Latency is tens to hundreds of milliseconds, but durability is extreme and ops is simple.
• Media, models, and artifacts. Bless immutability. Use content addresses; never overwrite real data, only manifests.
• Feature flag snapshots and config. A single pointer to the active config, served via CDN; config bundles in the bucket; a KV coordinates the pointer swap.

Where this goes wrong in the wild

• Treating buckets like POSIX. No atomic rename semantics, no directory locks, and listing is slow if you dump 100 million objects into a single prefix.
• Small-object storms. If you are below ~1 MB per object on average and pushing hundreds of writes/sec, request costs will eat you.
• Unbounded LISTs in hot paths. Always carry continuation tokens; never LIST more than you need; better yet, read from a manifest or index.
• Cross-region illusions. Replication lag is real. Never build cross-region coordination on top of replication catching up.

Portability and vendor realities

The S3 API is the de facto standard, but semantics differ at the edges:

• Consistency: S3 has strong read-after-write; some providers historically did not for LIST. Verify in staging.
• Egress and request pricing: Cloudflare R2 removed egress tolls, but request classes are priced differently than S3. Do the math before you chase “no egress”.
• On-prem: MinIO is excellent for S3 semantics in your own DC or in-country regions when LGPD or customer contracts require data residency in Brazil.

A decision framework you can apply this week

Score your candidate workload on each axis from 1 (bad fit) to 5 (great fit):

1. Write rate and granularity. Under 10 writes/sec average and batchable to 1–8 MB objects? Score 5. Hundreds of tiny writes/sec you cannot batch? Score 1.
2. Latency tolerance. OK with 50–150 ms reads and occasional 200–400 ms tails? Score 5. Need sub-20 ms p99? Score 1.
3. Immutability first. Is 95% of your data append-only? Score 5. Is most of it hot, mutable rows needing transactions? Score 1.
4. Regulatory retention. Need WORM or provable retention? Score 5. No compliance pressure? Score 3. Need frequent deletes/updates? Score 1.
5. Portability value. Cross-cloud or on-prem hedge matters? Score 5. You are all-in on one managed service anyway? Score 2.

If you score 18–25: Strong candidate for a bucket-backed service. 12–17: Consider a hybrid: immutable data in buckets, pointers/locks in a KV/DB. Below 12: Stick to databases and queues.

A 30–60–90 day pilot plan

30 days: Prove the core

• Pick one workload: artifact store, audit log, or a read-mostly config service.
• Build a single stateless binary in Go or Rust with a minimal API: PUT bundle, GET by key or range, LIST by manifest.
• Back it with an S3-compatible store in staging (AWS S3 or MinIO), plus a tiny KV (DynamoDB/etcd) for leases and pointers.
• Instrument request counts, p50/p95/p99 latencies, and bundle sizes.

60 days: Harden

• Add compaction and bundle indexes to cap request costs.
• Put a CDN in front of GETs; switch to presigned URLs for large downloads.
• Enable versioning and Object Lock in governance mode for audit datasets.
• Run crash-consistency tests: kill the process mid-commit 1000 times; assert no partial state escapes.

90 days: Decide

• Compare observed cost/latency vs. an equivalent DB-backed design.
• Decide scope: expand to more read-heavy services, or keep it for artifacts and logs only.
• Write the runbook: compaction SLOs, bucket lifecycle policies, multipart abort schedules, and budget alarms.

How we’d staff it with a nearshore pod

For US startups, a two to three person nearshore pod out of Brazil can own this end-to-end:

• 1 senior Go or Rust engineer to implement the binary, range I/O, and compaction.
• 1 platform engineer to wire S3 policies, VPC endpoints, KMS, and CDN.
• Optional data engineer if you need Parquet/columnar formats and downstream analytics.

Expect 6–8 hours of overlap with US time zones, and a total build window of 8–12 weeks to get from POC to hardened service with SLOs and cost guardrails.

The bottom line

“One binary, one bucket” is not a stunt. It is an operational wedge: ship simpler systems by leaning into immutability, batching, and the durability semantics object storage gives you for free. Use it where your workload’s physics match the bucket’s physics. When the fit is right, you save 20–50% on infra, cut operational surface area, and buy yourself a real multi-cloud hedge. When the fit is wrong, you build a slow, expensive file system cosplay. Choose wisely.

Key Takeaways

• Object storage-backed services are production-viable now thanks to strong consistency and mature edge/serverless tooling.
• Great fits: immutable data, audit/WORM, artifacts, and low-to-moderate write rates that can be batched.
• Bad fits: hot, transactional state and sub-20 ms tail latency requirements.
• Granularity drives cost—batch into 1–64 MB bundles and compact to avoid request-fee blowups.
• Keep locks and pointers in a real KV; keep data immutable in the bucket.
• Use versioning, Object Lock, VPC endpoints, KMS, and CDN to harden security and performance.
• Run a 30–60–90 day pilot with hard SLOs and budget alarms before you commit.