A laptop became part of the infrastructure because it was already there.
One machine on my desk has Apple Silicon and a Metal GPU. Another has Linux, Docker, and an NVIDIA card. Canopy already had managed machines for builds and deployments. The useful question was not whether we could detect three GPUs. It was whether one platform could accept work, understand what it needed, and send it to a machine that could actually run it.
That turns out to be a scheduling problem, a trust-boundary problem, and a record-keeping problem. The gpu: true flag is the easy part. It is also the part that tells you the least.
This is what we built, what we proved on hardware, and what is still unfinished.
Start with the work, not the machine
Build systems and model training both execute commands, but they tend to grow separate machinery. A build pipeline has a graph, logs, secrets, cancellation, and a history. A training service often grows another worker queue, another machine registry, and another way to report results. Then someone has to keep two schedulers, two accounting models, and two definitions of “finished” in agreement.
We already had an execution platform in Canopy Host. It plans pipeline work, tracks durable runs, places nodes on Groves, and stores what happened. Cortex needed accelerated machines for training. Developers had machines that could help. The architectural bet was that training could describe its execution requirements and use Host’s placement and job lifecycle instead of introducing another fleet scheduler.
The principle is simple: a workload should say what it needs; it should not have to know every machine in the system. That keeps the training code focused on training and gives the platform one place to explain why work ran, waited, or could not run.
One execution model, more than one kind of machine
Canopy Host stores the run and its placement decision. Forest is the execution layer. A managed Grove is a Kubernetes environment operated by Canopy. A native Grove is a customer-connected machine running a Forest agent: today that can be a Mac or a Linux machine with Docker. Both report capacity to Host, but their execution substrates are different.
That distinction matters. Kubernetes remains useful for managed services and container jobs. A native agent can use the machine’s host operating system, Apple frameworks, local Docker daemon, and attached hardware. We did not turn a Mac into a Kubernetes node or make every ordinary Grove GPU-aware. We gave Host a shared job model and let the target substrate do its own work.
flowchart LR cortex["Cortex training"] -->|"compute job: requirements and alternatives"| host["Canopy Host"] host --> placement["Placement and admission"] placement -->|"container job"| managed["Managed Grove<br/>Kubernetes"] placement -->|"native Linux job"| linux["Customer Linux machine<br/>Forest agent + Docker"] placement -->|"native macOS job"| mac["Customer Apple Silicon Mac<br/>Forest agent + Metal / MPS"] managed -->|"status and logs"| host linux -->|"status and logs"| host mac -->|"status and logs"| host host -->|"execution state and result"| cortex
Figure 1. Control and status share Host's job model; the selected Grove executes on its own substrate.
The arrows labelled status and logs are control-plane reporting. Input and output bytes take their own paths, described below. The diagram shows the architecture in code; it does not mean every route is generally available or that every combination has passed hardware acceptance.
Metal is not CUDA
Apple and NVIDIA both sell GPUs, which is about where the similarity becomes less useful.
Apple Silicon uses unified memory: CPU and GPU share physical memory, and the amount a workload can use is constrained by the device’s working-set limit and the operating system. PyTorch exposes Apple’s GPU through MPS. MLX is another Apple-native runtime. Our native Mac agent can report an Apple GPU and its Metal capability; that is not the same as reserving a private block of VRAM as a cloud scheduler might.
An NVIDIA machine usually exposes a discrete GPU with its own VRAM. A Linux container can use CUDA when the host driver, container runtime, image, and libraries agree. A digest-pinned image gives the job a stable software input; it does not install a compatible host driver or make a 6 GiB card hold a model that needs 16 GiB.
So Canopy models an accelerator with more than a Boolean: vendor, count, and optional memory requirement travel alongside platform, CPU, system memory, and runtime. A step can list alternatives: for example, a CUDA container on Linux and a host process on darwin/arm64. Host evaluates the matching alternative against the machine’s substrate and reported inventory. Unsupported hardware is refused or waits according to the request; a GPU-only job does not quietly become CPU work.
Here is a small, verified CLI shape for a CUDA job. The image must be replaced with a real digest-pinned image available to the submitting team.
canopy compute jobs submit \
--team "$CANOPY_TEAM_ID" \
--image 'nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04@sha256:2fcc4280646484290cc50dce5e65f388dd04352b07cbe89a635703bd1f9aedb6' \
--run 'nvidia-smi -L' \
--gpus 1 \
--vendor nvidia \
--gpu-memory-mb 5500 \
--providers own \
--wait 600
The flags are defined by the current Host CLI. This asks for one NVIDIA GPU with at least the requested memory on the team’s own machines and permits a ten-minute wait. It does not pin a particular machine. The digest is the CUDA base image pinned by the Cortex trainer Dockerfile; this command is a capability smoke request, not a training benchmark. It requires valid team credentials and an eligible machine that can pull the image.
For a job that can execute differently on different substrates, the current compute request supports alternatives in a JSON job file:
{
"team_id": "00000000-0000-4000-8000-000000000000",
"placement": { "providers": ["own", "canopy"], "wait_seconds": 86400 },
"steps": [{
"name": "train",
"alternatives": [
{
"platform": "linux/amd64",
"image": "nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04@sha256:2fcc4280646484290cc50dce5e65f388dd04352b07cbe89a635703bd1f9aedb6",
"run": "nvidia-smi -L",
"resources": { "cpus": 4, "memory_mb": 12288,
"accelerator": { "type": "gpu", "vendor": "nvidia", "count": 1, "memory_mb": 5500 } }
},
{
"platform": "darwin/arm64",
"run": "python3 -c 'import torch; print(torch.backends.mps.is_available())'",
"resources": { "cpus": 4, "memory_mb": 12288,
"accelerator": { "type": "gpu", "vendor": "apple", "count": 1 } }
}
]
}]
}
This shape is accepted by the compute request parser and documented in the Compute Jobs guide. Here the commands are GPU visibility smoke checks for their respective runtimes, not equivalent training workloads. The alternatives are complete execution descriptions, not an instruction to translate CUDA code into Metal. The workload owner supplies a valid command and runtime for each path.
Placement is a policy decision
Finding a GPU is not enough. Is the machine eligible for this team? Is it online? Is the requested vendor and memory available? Is the machine opted into unnamed jobs or bound to this service? Is it the customer’s capacity or capacity Canopy has offered? Is the account entitled to use that capacity and within its budget?
Host evaluates those constraints before dispatch. A team can pin a job to its own machine, choose a pool, limit the allowed provider to own or canopy, or give Host a preference while leaving it to choose. Service bindings add a scoped preference: a service identity can ask for jobs on machines bound to its accelerator role, then Host can consider eligible team machines and (if allowed) offered capacity. Blueprint represents those machines as resources, so the binding can be reviewed with the rest of the service configuration.
flowchart TD
request["Workload requirements and placement preferences"] --> candidates["Find eligible machines"]
candidates --> access{"Owner, offer, binding, pool and provider allowed?"}
access -->|"no"| refuse["Refuse candidate"]
access -->|"yes"| compatible{"Substrate, platform and accelerator fit?"}
compatible -->|"no"| refuse
compatible -->|"yes"| available{"Online, capacity and entitlement available?"}
available -->|"yes"| chosen["Persist machine and alternative"]
available -->|"no, may recover"| waiting["Queue with waiting reason"]
available -->|"no eligible path"| impossible["Refuse: no compatible path"]
waiting -->|"capacity changes"| candidates
Figure 2. Compatibility and authorization remove candidates; transient capacity constraints produce a waiting state.
The exact waiting reason is useful operational data: offline, busy, not entitled, over budget, or no matching GPU are different problems. A job that asks specifically for NVIDIA should not be sent to a CPU merely because one is free. A waiting state is honest, observable, and can become runnable when capacity changes. A refusal is appropriate when the request cannot match any allowed machine at all.
Canopy-managed GPU capacity is a separate matter from the placement code. The repository includes policy for offered capacity, but it does not establish a generally available NVIDIA cloud. Capacity is constrained by what has actually been offered, its entitlement, and the tested execution paths.
Your machine stays yours
A native Grove enrolls with Host and maintains outbound communication. Host does not need an inbound SSH port into a developer’s laptop to ask it to do work. The agent polls for assigned work, downloads the required inputs, runs the job under its service account, and reports state. For native services, the agent can also hold a tunnel session; Host authenticates and scopes requests to a bound service endpoint, and the agent forwards an allowlisted HTTP request to the workload on loopback.
sequenceDiagram participant Dev as Developer participant Host as Canopy Host participant Agent as Native Grove agent participant Work as Local workload Dev->>Host: Submit job with requirements Host->>Host: Check policy and persist placement Agent->>Host: Outbound poll / heartbeat Host-->>Agent: Assigned job and scoped transfer references Agent->>Host: Fetch input through authenticated transfer Agent->>Work: Start process on the machine Work-->>Agent: Exit state and output Agent->>Host: Report status and result digest Dev->>Host: Read job state and logs Dev->>Host: Call bound inference endpoint Host->>Agent: Authenticated tunnel request Agent->>Work: Allowlisted loopback HTTP request Work-->>Host: Response through agent tunnel Host-->>Dev: Stream response
Figure 3. The native machine initiates its control connection; Host authorizes work and relays scoped endpoint requests.
The job and service paths have different payloads, but the same ownership boundary: the machine initiates its connection, Host authorizes what that identity can do, and the workload remains local. Signed, short-lived transfer tokens let an enrolled machine move bytes through Host when it cannot reach Canopy’s private object store directly. A single-use upload claim and digest confirmation keep a retry from silently replacing different bytes.
The tunnel is not a general proxy. The current endpoint path accepts a constrained set of HTTP routes, resolves the named workload’s serving port on the agent, and relays bounded request and response bodies. That is enough for a service to reach an inference endpoint without exposing the machine as a public server. It is also a recently implemented path whose existence in code should not be confused with broad operational availability.
Cortex training through Host
Cortex owns training semantics: the run, dataset, model, progress, adapter validation, and the final training result. Host owns compute placement and the machine job. The seam is an execution record with a scoped execution token.
For a Cortex run, the backend derives the resource estimates and the execution alternatives. A Linux alternative uses a digest-pinned trainer image and NVIDIA CUDA. When a compatible Darwin trainer bundle is configured, another alternative can use the Mac’s host runtime and PyTorch MPS. QLoRA is CUDA-only; it is not offered as an Apple alternative. Host selects an eligible machine or reports why the run is waiting. The executor then gets its input from Cortex’s authenticated execution API, not from a shared database connection.
sequenceDiagram participant User as Developer participant Cortex as Cortex API participant Host as Canopy Host participant Agent as Selected machine participant Store as Cortex artifact storage User->>Cortex: Create and launch training run Cortex->>Cortex: Create execution and reserve usage Cortex->>Host: Submit job, alternatives and execution token Host->>Host: Place or queue by machine capabilities and policy Agent->>Host: Poll and receive assigned job Agent->>Cortex: GET execution input and dataset with run token Cortex-->>Agent: Training specification and dataset splits Agent->>Agent: Train with CUDA or MPS runtime Agent->>Cortex: PUT adapter + SHA-256 with run token Cortex->>Store: Persist uploaded bytes and artifact metadata Cortex->>Cortex: Inspect archive and verify adapter files Cortex->>Cortex: Finalize run and settle machine usage Host-->>Cortex: Job state, machine and start/finish times Cortex-->>User: Final run state and verified artifact
Figure 4. Cortex owns run data and artifact checks; Host owns placement and machine execution state.
The separation is deliberate. The machine cannot query Cortex’s database. Its token authorizes a narrow set of operations for one execution: fetch inputs, report progress, and upload named artifacts. The upload includes a SHA-256 digest. Cortex checks the digest, rejects unexpected archive members and unsafe paths, and requires the adapter configuration and safetensors weights before finalization. Usage settlement uses the machine’s recorded start and finish times rather than counting queue time as GPU time.
The current CLI exposes placement preferences directly:
cortex training create \
--name "support-adapter" \
--base-model "Qwen/Qwen2.5-0.5B-Instruct" \
--dataset 42 \
--own-machines-only \
--gpu-vendor apple
This command is implemented by the Cortex CLI; the placement flags are converted into the training request, while the Host backend constructs the machine-specific alternatives. The model and dataset must be allowed and available to the organization. A chosen vendor narrows the execution capability; it does not assert that compatible capacity exists.
There is an important storage caveat. The code inspected here stores uploaded run artifacts through Cortex’s artifact_storage abstraction and records them in cortex_ops.run_artifacts. That abstraction currently uses filesystem-backed storage. We found no cortex_ops.artifact_blobs implementation in this checkout, and therefore cannot describe database-backed blob persistence as completed. Moving durable artifacts to a shared object store remains necessary for reliable cross-pod finalization and deploy survival.
What the hardware proved
The acceptance notes are narrower than the architecture diagram. Here is what they actually record:
| Test | Recorded result | What it proves |
|---|---|---|
| Cortex LoRA on Apple M5 Pro, 10 October | Six steps in 7.6 seconds; loss 5.13 to 1.56; PyTorch reported mps:0 |
An earlier Cortex training path used MPS on a real Mac. It was not the newer Host placement and artifact lifecycle. |
| Host-placed Apple GPU job, 4 November | MLX matrix multiplication on the M5 Pro, 4096 × 4096 in 0.16 seconds; provider recorded as own |
Host placed a standalone native Apple GPU job and the job used the GPU. This is not a benchmark comparison. |
| NVIDIA GTX 1660 SUPER, 6 GiB | Machine inventory and job procedure exist; no successful CUDA container acceptance result appears in the dated record | NVIDIA capability is represented in code; this checkout does not document a completed real container test. |
| Host-based Cortex training with returned adapter | Backend, transfer endpoints and verification code exist; acceptance record says the end-to-end run remained open | The lifecycle is implemented, but the requested newer end-to-end hardware proof is not present here. |
| Native inference tunnel | Tunnel transport and agent endpoint code are present; dated acceptance says the tunnel still needed review and agent release | Code path exists; a completed real endpoint round trip is not established by the records inspected. |
| Canopy-managed CPU job for a team without its own machine | No matching acceptance result found | The scheduler can represent offered capacity, but this outcome remains unverified here. |
The earlier Mac training result is real and useful. It is also not run #11 through the current Host backend, so we should not merge the two into a cleaner story than the evidence allows. Likewise, we did not record a CUDA training result, a performance comparison, or a general GPU service-level guarantee.
The bugs were the architecture
The acceptance work found failures at the seams, which is where distributed systems tend to keep their best material.
An early Mac training command failed before training because the code insisted on CUDA even though the Mac environment described MPS. We added an explicit MPS path for LoRA and full fine-tuning and refused QLoRA with a clear reason. Later, an unpinned dependency update brought a fused loss implementation that crashed on MPS. Pinning the runtime versions made the software environment match the one we had validated. A hardware path is the sum of the driver, framework, libraries, and workload—not a vendor label.
Native jobs also run under a service account with a restricted home directory. The trainer originally assumed it could use the normal Hugging Face and pip caches. It could not. The job now creates a writable temporary home for caches and outputs, then removes it. That solved permission problems, but it also made artifact return essential: a correct adapter in a temporary directory is still a lost adapter.
That led to the execution-token protocol. The runner streams its adapter back with a digest; Cortex verifies the format before accepting the result. We keep that channel separate from database access so the machine has only the authority needed for its own run.
One unresolved reliability issue also deserves a clear line in the sand: the engineering record flags container image pulls that can outlive the requested timeout. A timeout written in a job is not useful if the layer performing the pull does not enforce it. Until that path is closed and verified, the timeout is not a guarantee that every startup failure ends promptly.
What is next
The next steps follow from the gaps in the results table: finish and record a CUDA container acceptance; run the current Cortex-on-Host flow through adapter persistence on real hardware; verify the inference tunnel with a deployed native service; and make artifact storage durable across application instances. Cancellation and timeout behavior need the same end-to-end scrutiny as successful jobs. Machines that sleep or disappear need useful waiting and recovery behavior, not optimistic status labels.
Canopy-managed GPU capacity can grow when there is a concrete need and a machine path worth operating. The placement model already distinguishes owned from offered capacity and accounts for them differently. That is groundwork, not a cloud product announcement.
The interesting outcome is smaller and more practical: when a capable machine is already in the room, the platform can begin to treat it as part of the system without asking every workload to become its own scheduler. There is still a lot to prove. At least the work now has somewhere honest to wait.
Further reading
- Compute Jobs — request shape, placement, and CLI usage.
- Native Groves — joining a customer-owned Mac or Linux machine.
- A Grove should know what it can run — the earlier placement model and its boundaries.
- Forest runtime — why execution lives behind a Grove boundary.