Notes from building the systems.
We write about architecture, incidents, tradeoffs, and the work behind the products. Each article comes from a real engineering investigation.
We taught Canopy to use your GPU
A technical account of Canopy Host's heterogeneous compute model: native Apple and NVIDIA execution, capability-based placement, Cortex training, artifact verification, real hardware results, and the gaps still open.
When 3,100 files became 300
An engineering investigation into silent Git provider diff limits and how Canopy Source records the difference between a complete comparison and the files a provider happened to return.
A Grove should know what it can run
The execution boundary behind Canopy Host is changing: requirements travel with work, Groves report capabilities, and placement records the decision. Here is what runs, what is gated, and what still lacks hardware or a dispatch path.
When HEAD said 138 bytes and GET returned zero
How registry garbage collection left BuildKit cache blobs visible to metadata checks after their files were gone, and how that poisoned later cache exports in Forest.
How the Canopy Pipeline Graph made our pipeline incremental
How Canopy's Pipeline Graph makes incremental execution possible: execution identity, scoped source inputs, order-only dependencies, verified workspaces, shadow mode, and determinism checks.
The package name was not the network boundary
A Cortex API build failure that looked like flaky package downloads, and turned out to be a CUDA dependency graph, a 2 GiB egress ceiling, an HTTP proxy boundary, and a model distributed across several hosts.
The pipeline is code. The execution is a graph.
Inside canopy.pipeline.ts and @canopy/ci: a synchronous authoring API, the Canopy Pipeline Graph, content identity, planning, and the boundary where execution begins.
Why a WORKDIR took five minutes
A technical account of the BuildKit performance incident in Forest: native snapshotting, a misleading benchmark, configuration precedence, and a separate one-hour PVC retention bug.
Forest, and the problem with pretending one machine is not enough
A technical account of Forest, the runtime behind Canopy Host: its workload model, Talos and Kubernetes layer, capacity mistakes, migration failures, and what production changed.