A laptop became part of the infrastructure because it was already there.

One machine on my desk has Apple Silicon and a Metal GPU. Another has Linux, Docker, and an NVIDIA card. Canopy already had managed machines for builds and deployments. The useful question was not whether we could detect three GPUs. It was whether one platform could accept work, understand what it needed, and send it to a machine that could actually run it.

That turns out to be a scheduling problem, a trust-boundary problem, and a record-keeping problem. The gpu: true flag is the easy part. It is also the part that tells you the least.

This is what we built, what we proved on hardware, and what is still unfinished.

Start with the work, not the machine

Build systems and model training both execute commands, but they tend to grow separate machinery. A build pipeline has a graph, logs, secrets, cancellation, and a history. A training service often grows another worker queue, another machine registry, and another way to report results. Then someone has to keep two schedulers, two accounting models, and two definitions of “finished” in agreement.

We already had an execution platform in Canopy Host. It plans pipeline work, tracks durable runs, places nodes on Groves, and stores what happened. Cortex needed accelerated machines for training. Developers had machines that could help. The architectural bet was that training could describe its execution requirements and use Host’s placement and job lifecycle instead of introducing another fleet scheduler.

The principle is simple: a workload should say what it needs; it should not have to know every machine in the system. That keeps the training code focused on training and gives the platform one place to explain why work ran, waited, or could not run.

One execution model, more than one kind of machine

Canopy Host stores the run and its placement decision. Forest is the execution layer. A managed Grove is a Kubernetes environment operated by Canopy. A native Grove is a customer-connected machine running a Forest agent: today that can be a Mac or a Linux machine with Docker. Both report capacity to Host, but their execution substrates are different.

That distinction matters. Kubernetes remains useful for managed services and container jobs. A native agent can use the machine’s host operating system, Apple frameworks, local Docker daemon, and attached hardware. We did not turn a Mac into a Kubernetes node or make every ordinary Grove GPU-aware. We gave Host a shared job model and let the target substrate do its own work.

flowchart LR
  cortex["Cortex training"] -->|"compute job: requirements and alternatives"| host["Canopy Host"]
  host --> placement["Placement and admission"]
  placement -->|"container job"| managed["Managed Grove<br/>Kubernetes"]
  placement -->|"native Linux job"| linux["Customer Linux machine<br/>Forest agent + Docker"]
  placement -->|"native macOS job"| mac["Customer Apple Silicon Mac<br/>Forest agent + Metal / MPS"]
  managed -->|"status and logs"| host
  linux -->|"status and logs"| host
  mac -->|"status and logs"| host
  host -->|"execution state and result"| cortex

Figure 1. Control and status share Host's job model; the selected Grove executes on its own substrate.

The arrows labelled status and logs are control-plane reporting. Input and output bytes take their own paths, described below. The diagram shows the architecture in code; it does not mean every route is generally available or that every combination has passed hardware acceptance.

Metal is not CUDA

Apple and NVIDIA both sell GPUs, which is about where the similarity becomes less useful.

Apple Silicon uses unified memory: CPU and GPU share physical memory, and the amount a workload can use is constrained by the device’s working-set limit and the operating system. PyTorch exposes Apple’s GPU through MPS. MLX is another Apple-native runtime. Our native Mac agent can report an Apple GPU and its Metal capability; that is not the same as reserving a private block of VRAM as a cloud scheduler might.

An NVIDIA machine usually exposes a discrete GPU with its own VRAM. A Linux container can use CUDA when the host driver, container runtime, image, and libraries agree. A digest-pinned image gives the job a stable software input; it does not install a compatible host driver or make a 6 GiB card hold a model that needs 16 GiB.

So Canopy models an accelerator with more than a Boolean: vendor, count, and optional memory requirement travel alongside platform, CPU, system memory, and runtime. A step can list alternatives: for example, a CUDA container on Linux and a host process on darwin/arm64. Host evaluates the matching alternative against the machine’s substrate and reported inventory. Unsupported hardware is refused or waits according to the request; a GPU-only job does not quietly become CPU work.

Here is a small, verified CLI shape for a CUDA job. The image must be replaced with a real digest-pinned image available to the submitting team.

canopy compute jobs submit \
  --team "$CANOPY_TEAM_ID" \
  --image 'nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04@sha256:2fcc4280646484290cc50dce5e65f388dd04352b07cbe89a635703bd1f9aedb6' \
  --run 'nvidia-smi -L' \
  --gpus 1 \
  --vendor nvidia \
  --gpu-memory-mb 5500 \
  --providers own \
  --wait 600

The flags are defined by the current Host CLI. This asks for one NVIDIA GPU with at least the requested memory on the team’s own machines and permits a ten-minute wait. It does not pin a particular machine. The digest is the CUDA base image pinned by the Cortex trainer Dockerfile; this command is a capability smoke request, not a training benchmark. It requires valid team credentials and an eligible machine that can pull the image.

For a job that can execute differently on different substrates, the current compute request supports alternatives in a JSON job file:

{
  "team_id": "00000000-0000-4000-8000-000000000000",
  "placement": { "providers": ["own", "canopy"], "wait_seconds": 86400 },
  "steps": [{
    "name": "train",
    "alternatives": [
      {
        "platform": "linux/amd64",
        "image": "nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04@sha256:2fcc4280646484290cc50dce5e65f388dd04352b07cbe89a635703bd1f9aedb6",
        "run": "nvidia-smi -L",
        "resources": { "cpus": 4, "memory_mb": 12288,
          "accelerator": { "type": "gpu", "vendor": "nvidia", "count": 1, "memory_mb": 5500 } }
      },
      {
        "platform": "darwin/arm64",
        "run": "python3 -c 'import torch; print(torch.backends.mps.is_available())'",
        "resources": { "cpus": 4, "memory_mb": 12288,
          "accelerator": { "type": "gpu", "vendor": "apple", "count": 1 } }
      }
    ]
  }]
}

This shape is accepted by the compute request parser and documented in the Compute Jobs guide. Here the commands are GPU visibility smoke checks for their respective runtimes, not equivalent training workloads. The alternatives are complete execution descriptions, not an instruction to translate CUDA code into Metal. The workload owner supplies a valid command and runtime for each path.

Placement is a policy decision

Finding a GPU is not enough. Is the machine eligible for this team? Is it online? Is the requested vendor and memory available? Is the machine opted into unnamed jobs or bound to this service? Is it the customer’s capacity or capacity Canopy has offered? Is the account entitled to use that capacity and within its budget?

Host evaluates those constraints before dispatch. A team can pin a job to its own machine, choose a pool, limit the allowed provider to own or canopy, or give Host a preference while leaving it to choose. Service bindings add a scoped preference: a service identity can ask for jobs on machines bound to its accelerator role, then Host can consider eligible team machines and (if allowed) offered capacity. Blueprint represents those machines as resources, so the binding can be reviewed with the rest of the service configuration.

flowchart TD
  request["Workload requirements and placement preferences"] --> candidates["Find eligible machines"]
  candidates --> access{"Owner, offer, binding, pool and provider allowed?"}
  access -->|"no"| refuse["Refuse candidate"]
  access -->|"yes"| compatible{"Substrate, platform and accelerator fit?"}
  compatible -->|"no"| refuse
  compatible -->|"yes"| available{"Online, capacity and entitlement available?"}
  available -->|"yes"| chosen["Persist machine and alternative"]
  available -->|"no, may recover"| waiting["Queue with waiting reason"]
  available -->|"no eligible path"| impossible["Refuse: no compatible path"]
  waiting -->|"capacity changes"| candidates

Figure 2. Compatibility and authorization remove candidates; transient capacity constraints produce a waiting state.

The exact waiting reason is useful operational data: offline, busy, not entitled, over budget, or no matching GPU are different problems. A job that asks specifically for NVIDIA should not be sent to a CPU merely because one is free. A waiting state is honest, observable, and can become runnable when capacity changes. A refusal is appropriate when the request cannot match any allowed machine at all.

Canopy-managed GPU capacity is a separate matter from the placement code. The repository includes policy for offered capacity, but it does not establish a generally available NVIDIA cloud. Capacity is constrained by what has actually been offered, its entitlement, and the tested execution paths.

Your machine stays yours

A native Grove enrolls with Host and maintains outbound communication. Host does not need an inbound SSH port into a developer’s laptop to ask it to do work. The agent polls for assigned work, downloads the required inputs, runs the job under its service account, and reports state. For native services, the agent can also hold a tunnel session; Host authenticates and scopes requests to a bound service endpoint, and the agent forwards an allowlisted HTTP request to the workload on loopback.

sequenceDiagram
  participant Dev as Developer
  participant Host as Canopy Host
  participant Agent as Native Grove agent
  participant Work as Local workload
  Dev->>Host: Submit job with requirements
  Host->>Host: Check policy and persist placement
  Agent->>Host: Outbound poll / heartbeat
  Host-->>Agent: Assigned job and scoped transfer references
  Agent->>Host: Fetch input through authenticated transfer
  Agent->>Work: Start process on the machine
  Work-->>Agent: Exit state and output
  Agent->>Host: Report status and result digest
  Dev->>Host: Read job state and logs
  Dev->>Host: Call bound inference endpoint
  Host->>Agent: Authenticated tunnel request
  Agent->>Work: Allowlisted loopback HTTP request
  Work-->>Host: Response through agent tunnel
  Host-->>Dev: Stream response

Figure 3. The native machine initiates its control connection; Host authorizes work and relays scoped endpoint requests.

The job and service paths have different payloads, but the same ownership boundary: the machine initiates its connection, Host authorizes what that identity can do, and the workload remains local. Signed, short-lived transfer tokens let an enrolled machine move bytes through Host when it cannot reach Canopy’s private object store directly. A single-use upload claim and digest confirmation keep a retry from silently replacing different bytes.

The tunnel is not a general proxy. The current endpoint path accepts a constrained set of HTTP routes, resolves the named workload’s serving port on the agent, and relays bounded request and response bodies. That is enough for a service to reach an inference endpoint without exposing the machine as a public server. It is also a recently implemented path whose existence in code should not be confused with broad operational availability.

Cortex training through Host

Cortex owns training semantics: the run, dataset, model, progress, adapter validation, and the final training result. Host owns compute placement and the machine job. The seam is an execution record with a scoped execution token.

For a Cortex run, the backend derives the resource estimates and the execution alternatives. A Linux alternative uses a digest-pinned trainer image and NVIDIA CUDA. When a compatible Darwin trainer bundle is configured, another alternative can use the Mac’s host runtime and PyTorch MPS. QLoRA is CUDA-only; it is not offered as an Apple alternative. Host selects an eligible machine or reports why the run is waiting. The executor then gets its input from Cortex’s authenticated execution API, not from a shared database connection.

sequenceDiagram
  participant User as Developer
  participant Cortex as Cortex API
  participant Host as Canopy Host
  participant Agent as Selected machine
  participant Store as Cortex artifact storage
  User->>Cortex: Create and launch training run
  Cortex->>Cortex: Create execution and reserve usage
  Cortex->>Host: Submit job, alternatives and execution token
  Host->>Host: Place or queue by machine capabilities and policy
  Agent->>Host: Poll and receive assigned job
  Agent->>Cortex: GET execution input and dataset with run token
  Cortex-->>Agent: Training specification and dataset splits
  Agent->>Agent: Train with CUDA or MPS runtime
  Agent->>Cortex: PUT adapter + SHA-256 with run token
  Cortex->>Store: Persist uploaded bytes and artifact metadata
  Cortex->>Cortex: Inspect archive and verify adapter files
  Cortex->>Cortex: Finalize run and settle machine usage
  Host-->>Cortex: Job state, machine and start/finish times
  Cortex-->>User: Final run state and verified artifact

Figure 4. Cortex owns run data and artifact checks; Host owns placement and machine execution state.

The separation is deliberate. The machine cannot query Cortex’s database. Its token authorizes a narrow set of operations for one execution: fetch inputs, report progress, and upload named artifacts. The upload includes a SHA-256 digest. Cortex checks the digest, rejects unexpected archive members and unsafe paths, and requires the adapter configuration and safetensors weights before finalization. Usage settlement uses the machine’s recorded start and finish times rather than counting queue time as GPU time.

The current CLI exposes placement preferences directly:

cortex training create \
  --name "support-adapter" \
  --base-model "Qwen/Qwen2.5-0.5B-Instruct" \
  --dataset 42 \
  --own-machines-only \
  --gpu-vendor apple

This command is implemented by the Cortex CLI; the placement flags are converted into the training request, while the Host backend constructs the machine-specific alternatives. The model and dataset must be allowed and available to the organization. A chosen vendor narrows the execution capability; it does not assert that compatible capacity exists.

There is an important storage caveat. The code inspected here stores uploaded run artifacts through Cortex’s artifact_storage abstraction and records them in cortex_ops.run_artifacts. That abstraction currently uses filesystem-backed storage. We found no cortex_ops.artifact_blobs implementation in this checkout, and therefore cannot describe database-backed blob persistence as completed. Moving durable artifacts to a shared object store remains necessary for reliable cross-pod finalization and deploy survival.

What the hardware proved

The acceptance notes are narrower than the architecture diagram. Here is what they actually record:

Test Recorded result What it proves
Cortex LoRA on Apple M5 Pro, 10 October Six steps in 7.6 seconds; loss 5.13 to 1.56; PyTorch reported mps:0 An earlier Cortex training path used MPS on a real Mac. It was not the newer Host placement and artifact lifecycle.
Host-placed Apple GPU job, 4 November MLX matrix multiplication on the M5 Pro, 4096 × 4096 in 0.16 seconds; provider recorded as own Host placed a standalone native Apple GPU job and the job used the GPU. This is not a benchmark comparison.
NVIDIA GTX 1660 SUPER, 6 GiB Machine inventory and job procedure exist; no successful CUDA container acceptance result appears in the dated record NVIDIA capability is represented in code; this checkout does not document a completed real container test.
Host-based Cortex training with returned adapter Backend, transfer endpoints and verification code exist; acceptance record says the end-to-end run remained open The lifecycle is implemented, but the requested newer end-to-end hardware proof is not present here.
Native inference tunnel Tunnel transport and agent endpoint code are present; dated acceptance says the tunnel still needed review and agent release Code path exists; a completed real endpoint round trip is not established by the records inspected.
Canopy-managed CPU job for a team without its own machine No matching acceptance result found The scheduler can represent offered capacity, but this outcome remains unverified here.

The earlier Mac training result is real and useful. It is also not run #11 through the current Host backend, so we should not merge the two into a cleaner story than the evidence allows. Likewise, we did not record a CUDA training result, a performance comparison, or a general GPU service-level guarantee.

The bugs were the architecture

The acceptance work found failures at the seams, which is where distributed systems tend to keep their best material.

An early Mac training command failed before training because the code insisted on CUDA even though the Mac environment described MPS. We added an explicit MPS path for LoRA and full fine-tuning and refused QLoRA with a clear reason. Later, an unpinned dependency update brought a fused loss implementation that crashed on MPS. Pinning the runtime versions made the software environment match the one we had validated. A hardware path is the sum of the driver, framework, libraries, and workload—not a vendor label.

Native jobs also run under a service account with a restricted home directory. The trainer originally assumed it could use the normal Hugging Face and pip caches. It could not. The job now creates a writable temporary home for caches and outputs, then removes it. That solved permission problems, but it also made artifact return essential: a correct adapter in a temporary directory is still a lost adapter.

That led to the execution-token protocol. The runner streams its adapter back with a digest; Cortex verifies the format before accepting the result. We keep that channel separate from database access so the machine has only the authority needed for its own run.

One unresolved reliability issue also deserves a clear line in the sand: the engineering record flags container image pulls that can outlive the requested timeout. A timeout written in a job is not useful if the layer performing the pull does not enforce it. Until that path is closed and verified, the timeout is not a guarantee that every startup failure ends promptly.

What is next

The next steps follow from the gaps in the results table: finish and record a CUDA container acceptance; run the current Cortex-on-Host flow through adapter persistence on real hardware; verify the inference tunnel with a deployed native service; and make artifact storage durable across application instances. Cancellation and timeout behavior need the same end-to-end scrutiny as successful jobs. Machines that sleep or disappear need useful waiting and recovery behavior, not optimistic status labels.

Canopy-managed GPU capacity can grow when there is a concrete need and a machine path worth operating. The placement model already distinguishes owned from offered capacity and accounts for them differently. That is groundwork, not a cloud product announcement.

The interesting outcome is smaller and more practical: when a capable machine is already in the room, the platform can begin to treat it as part of the system without asking every workload to become its own scheduler. There is still a lot to prove. At least the work now has somewhere honest to wait.

Further reading