How Photon works: a photography site with a pipeline behind it

Photography gives me a reason to step away from a computer. Building a photography site apparently gives me a reason to come straight back. Photon brings those interests together: a place for photographs, albums, and public profiles, backed by a processing pipeline that does the less visible work of making an uploaded image useful on the web.

You can see my photos on my Photon profile. The interesting engineering starts before a photo reaches that page: uploading the source file, generating display sizes, preserving useful metadata, deciding what can be public, and keeping optional AI work from holding up the photograph itself.

A small web app with a background pipeline

The interface is Next.js and TypeScript. Supabase provides authentication and a PostgreSQL database for users, albums, photos, tags, and preferences. Go workers handle image processing on Google Cloud Run, with Pub/Sub carrying jobs between stages. Google Cloud Storage holds source files and generated variants; Cloudflare R2 provides another layer for delivering image bytes. The cloud resources are defined with Pulumi in Go.

Browser → authenticated upload API → signed Cloud Storage upload
        → upload confirmation → Pub/Sub → Go variant worker
        → Cloud Storage variants + R2 cache warming
        → optional metadata worker → tags and descriptions

Viewer → photo route → access check → R2 hit
                                     or Cloud Storage fallback

That division gives each component a fairly specific job. The web application handles identity and interaction. Workers do the expensive image work. The database records what exists and which stage it has reached. Object storage holds the bytes.

The browser uploads directly to storage

Uploading starts with an authenticated request to the application. It creates a photo record and returns a signed upload URL. The browser sends the file to Cloud Storage, then confirms completion with the application so processing can be dispatched.

I like this boundary because a large image does not need to travel through the web server on its way to storage. The application still decides who can upload and associates the object with an owner and a photo record. A completed transfer and a completed photo are separate events, which also makes the intermediate state explicit in the interface.

One upload becomes several useful images

The Go worker extracts EXIF metadata, decodes the image, applies its orientation, and generates three JPEG variants. Their maximum dimensions are 400, 1,200, and 2,400 pixels, with the aspect ratio preserved. A thumbnail grid and a large photo viewer can therefore request different versions of the same photograph.

This is where I particularly enjoy Go. The work involves explicit resource lifetimes, storage clients, image buffers, errors, and a little concurrency. The worker releases the reference to the source byte buffer before generating variants and warms the R2 copies concurrently. Those are small implementation choices, but they matter when a decoded photograph is much larger than the compressed file that arrived over the network.

Metadata has its own wrinkles. Orientation must survive reprocessing after EXIF has been removed, so the database can supply a fallback. The JPEG stripping code removes EXIF APP1 segments while preserving other segments. That is a specific operation: it does not imply that every format or every kind of embedded metadata has been scrubbed. Treating metadata as a deliberate part of the pipeline makes these details much easier to reason about.

Storage and delivery have different jobs

Cloud Storage is the durable backing store for the pipeline. R2 holds cached copies used by the image delivery route. The worker proactively uploads the generated sizes to R2; the route can fall back to Cloud Storage when a requested object is missing from that cache.

The extra layer creates operational work: cache misses, stale copies, invalidation, and failures during cache warming all need an answer. In return, delivery and processing can evolve separately. The cache is populated from stored objects, so it does not have to be the only place a generated image exists.

Access is another layer of that decision. Photos can have their own visibility or inherit it through an album and the owner's default. The application resolves that visibility for image requests before fetching the object. HTTP cache policy needs equal care, especially for content whose visibility can change; a database permission and a browser cache lifetime describe different parts of the system.

AI metadata is an optional stage

The optional AI pass produces tags and descriptions. User settings control tagging and whether descriptions use a descriptive or storyteller style. The tag schema is built from the application's known tags, so the worker can resolve model output back to database IDs. That keeps the result usable by the rest of the product.

Variant generation and metadata generation are separate stages. While a photo is marked as enhancing, its display variants already exist and the image route can serve them. If analysis fails, the enhancement worker can still mark the photo ready without AI metadata. A model call should have a bounded place in the lifecycle of a photography application.

Background work needs ownership

A photo moves through states such as pending, processing, enhancing, ready, and failed. Workers claim work with a database token and a timestamp. Duplicate deliveries can be skipped while a live claim exists, expired claims can be taken over, and a worker releases only the token it owns. Processing and enhancement have separate claims.

Those details are a large part of why I find this project interesting. A gallery gives very concrete reasons to think about asynchronous work, retries, memory, permissions, and optional dependencies. The payoff is also easy to see: an uploaded photograph becomes something I can organize, share, and look at. That connection between the pipeline and the thing it enables is a good reason to keep building it.