We added 3,100 small files in one commit, pushed the same Git history to GitHub and GitLab, and asked each provider what changed.
GitHub’s comparison API returned 300 files. The request succeeded. The response had no file total and no flag saying the list was incomplete. Requesting page two did not produce the other 2,800 files; that page had no files field at all.
The tested GitLab instance returned 1,000 diffs and compare_timeout: true. At first that sounded like a slow comparison. Its configured diff_max_files was also 1,000. A smaller 400-file comparison returned all its diffs with compare_timeout: false.
There was no failed HTTP request to handle in either case. The failure available to a consumer was more dangerous: a plausible, successful answer that did not describe the whole change. A source browser that renders those arrays without qualification can make the missing files look as though they never changed.
Why we needed the comparison
Canopy Source is the preview source workspace inside a Canopy Host environment. It reads attached repositories so a developer can browse a revision, inspect a commit, compare two commits, or ask what changed between a deployed release and today’s branch heads. The environment can contain more than one repository, and its services can be built from different source paths.
That last distinction makes a file list operationally meaningful. If a comparison omits the files under a service’s source path, a delivery view might say that service was unaffected. A code review view could hide changes. An investigation into a deployment could miss the source change that explains a new artifact. Those are possible consequences of an incomplete comparison, not incidents we observed in production.
The straightforward implementation would ask the provider for a comparison, follow its pagination, and show the returned files. That is sound only if the provider’s pagination covers the file list and its response tells us when it does not. The 3,100-file fixture was designed to test that assumption directly.
One history, two APIs
The fixture builder creates a deterministic local repository. It fixes commit dates and identity, then pushes the same commits to a private repository on GitHub and to our self-managed GitLab 19.4.1 instance. The commit that adds the 3,100 files is therefore the same Git object on both sides. Comparing results does not require guessing whether two independently prepared repositories have the same content.
The captured responses are sanitised and checked into the test fixtures. The provider contract suite can replay them without provider credentials, and it can run against the live fixture repositories when explicitly enabled. The October 18 live contract run recorded 100 passing tests across the two providers. That is stronger evidence about these endpoints than a mock alone; it is still evidence about the accounts, versions, settings, and fixture we actually exercised.
| Request on the 3,100-file commit | What we observed | What the response did not establish |
|---|---|---|
| GitHub comparison | 300 files, HTTP 200, no truncation flag or file total | Whether those 300 were all the changed files |
| GitHub commit detail, paged at 100 files | 30 full pages, then an empty page 31 | The remaining 100 files, despite paging |
| GitLab comparison on our instance | 1,000 diffs, HTTP 200, compare_timeout: true |
Whether the flag meant elapsed time, a file limit, or both |
| GitLab commit diff on our instance | Ten pages of 100, x-total: 1000, then no next page |
The other 2,100 files known to exist in Git |
These are two distinct GitHub ceilings. The comparison endpoint exposes at most 300 files in this test. The commit endpoint exposes 3,000 files. Taking 3,000 from the latter does not repair a 300-file comparison, and paging a comparison’s commits does not page its files. We verified the distinction by requesting later comparison pages: GitHub still returned commit metadata, but omitted the files key.
The GitLab number belongs to the tested instance. Its recorded application settings were diff_max_files: 1000, diff_max_patch_bytes: 204800, and diff_max_lines: 50000. We have not measured the equivalent limits on gitlab.com. The GitLab adapter uses 1,000 as the threshold for this tested configuration; another instance with different settings needs its own evidence.
A length is not a proof
It is tempting to treat a full page as a truncation signal. A list of exactly 300 files could mean GitHub cut a larger comparison down to 300. It could also mean the comparison really changed exactly 300 files. The response’s length cannot distinguish them.
For GitHub, we therefore use the boundary conservatively. When a comparison returns 300 files, Canopy Source reads the trees at the relevant commits and compares their entries. A tree gives us paths, object IDs, and modes. It is enough to determine which paths were added, deleted, modified, or changed mode, provided both trees are complete. The reconstructed file set can then be paged by Canopy. In the 3,100-file case, that route gives the caller the full path list and labels the strategy tree_diff.
It does not magically recreate the provider’s patch text or rename heuristic. Canopy initially marks reconstructed patches as lazy and can fetch a selected file’s old and new content to build a patch. Exact object identity can establish a rename when an old path disappears and a new path has the same object ID. A rename with edits has different IDs and may appear as a deletion plus an addition. The response says renameDetection: 'exact_only', and the interface explains that limit. A comparison reconstructed from trees also leaves additions and deletions totals unknown rather than turning absent line counts into zero.
The tree strategy itself has a failure condition. If GitHub says a recursive tree is truncated, the two trees cannot prove the complete file set. When the original provider list exists, the adapter returns those known files with truncated completeness and an unknown total. If even that list is unavailable, it returns a truncated tree result rather than a fabricated empty comparison. We have a unit test for the fallback; we did not exercise GitHub’s tree recursion ceiling against the live fixture.
GitLab does not currently take the same tree fallback on its comparison path. When compare_timeout is true and the response contains at least 1,000 diffs, the adapter marks the file list truncated with reason provider_file_limit. When the flag is true below that threshold, it marks the list partial with reason provider_timeout. That is an interpretation based on the tested instance’s settings and recorded response, not a claim that the field has one universal meaning across GitLab installations.
The distinction matters because a timeout is not necessarily the reason an answer stopped. If we had treated compare_timeout as only a clock event, we would have described a configured file-count cut as a transient performance problem. If we had treated x-total: 1000 as the real number of changed files, we would have repeated the provider’s limited view as a repository fact.
For the 3,100-file fixture, the resulting paths through the adapters are different:
flowchart TD
fixture["One commit adds 3,100 files"] --> github["GitHub compare returns 300 paths"]
fixture --> gitlab["Tested GitLab compare returns 1,000 diffs"]
github --> trees{"Are both Git trees complete?"}
trees -->|yes| rebuilt["Reconstruct all changed paths from trees<br/>Patches lazy; renames exact only"]
trees -->|no| githubCut["Keep known paths<br/>Mark truncated; total unknown"]
gitlab --> gitlabCut["compare_timeout is true<br/>Mark truncated; total unknown"]
Paths, patches, and direction
Even a complete path list is not the same as complete patches. The fixtures include a text patch of about 273 KB whose GitLab diff is marked too_large with empty patch text. That exceeds the tested instance’s configured 200 KB patch limit. The instance also has a line limit; we did not isolate that limit in this example. Other GitLab diffs can be collapsed: the path and modes remain visible, but the patch is withheld until another request. Canopy represents those cases separately as a file with a too_large or lazy patch. It does not discard the file merely because the patch body is empty.
Large blobs present another boundary. The fixture has a 5 MB text file and an 11.88 MB file. Canopy’s own read cap can return metadata or an explicit first-lines prefix without claiming to have returned the whole blob. A Git LFS pointer is the small file stored in Git, not automatically the large object referenced by it. GitHub’s contents endpoint even reported the LFS object’s size beside pointer content in our fixture. For source identity, the tree mode and Git blob are more reliable than inferring type and size from that one contents response.
Modes matter too. GitHub reported our mode-only change as modified, with zero additions and deletions and no old or new mode fields in the commit file entry. A file changed into a symlink appeared as a removed entry and an added entry at the same path. A near-pure rename was labelled renamed, while a rename with a larger edit was reported as remove plus add. The adapter combines same-path remove and add into a type change and uses tree modes where it needs to know whether an entry is executable, a symlink, or a submodule. A zero-line patch is not proof that nothing relevant changed.
Comparison direction is another way an apparently valid diff can answer the wrong question. GitHub’s comparison endpoint uses base...head. On a diverged pair it reports changes on the head side from the merge base, which is useful for a review but different from the direct tree change from base to head. If base is ahead of head, the ordinary comparison can contain no files; Canopy requests the reverse comparison and inverts the result for a direct comparison. For large or diverged direct comparisons, it uses trees at the requested endpoints.
GitLab uses a from and to query and the straight switch: straight=true asks for a direct comparison, while straight=false uses merge-base semantics. The fixture deliberately changes CANOPY_REMOVED to CANOPY_ADDED; the captured comparisons check that the added and removed lines swap when the direction swaps. This test catches a mistake that a file count could never reveal.
Unrelated histories are also provider specific. GitHub’s comparison rejects a pair with no common ancestor; Canopy can still compare the endpoint trees in direct mode. Our GitLab instance returned a successful comparison of unrelated histories containing a whole-tree-looking file list. The separate merge-base endpoint returned 404 Merge Base Not Found. Canopy asks for that relation explicitly and reports unrelated; a merge-base comparison is not meaningful for the pair.
The contract carries uncertainty
The central change in Canopy Source was to put completeness in the model instead of trying to infer it from an empty array, a next-page link, or a UI convention. The actual TypeScript contract has four states:
type GitCompleteness =
| { state: 'complete' }
| { state: 'partial'; reason: GitIncompleteReason; loaded: number; total: number | null }
| { state: 'truncated'; reason: GitIncompleteReason; loaded: number; total: number | null }
| { state: 'unavailable'; reason: GitIncompleteReason };
partial means we have some evidence but could not finish the operation, as with a provider timeout. truncated means a known cap cut the result, as with a provider file limit. unavailable means the requested information could not be read. The reason travels with the state, and total stays null when no trustworthy total exists. A list of 1,000 files with total: null communicates something very different from “1,000 files changed.”
Completeness is attached to the paged files and commits independently and to the comparison as a whole. Individual files also carry patch state: included, lazy, too large, binary, empty, or unavailable. That is why a list of known paths can still be useful when their patches need separate work. The API’s caller does not have to reverse-engineer this distinction from provider JSON.
The interface renders completeness notices beside affected lists. The cross-repository Changes view preserves unavailable repositories as rows with reasons and possible fixes. When it uses changed paths to decide whether a service’s source_path was affected, it leaves affected unknown if the file set was incomplete and no matching file was seen. A missing match in a partial list is not evidence of no match in the repository. Client paging has its own canopy_page_cap notice, separate from provider limits.
This is a narrower guarantee than “all diffs are complete.” GitHub’s tree fallback can reconstruct paths when its trees are complete; it cannot always reconstruct rename intent or an oversized patch. GitLab’s comparison can remain truncated. The contract makes those limitations visible to the next layer instead of laundering them into an authoritative looking empty state.
Source is part of the delivery record
A branch head answers what the provider points to now. It does not identify the source from which an older deployment was built. A push, a force push, a release, or a rollback can make those answers diverge without changing the bytes of the running artifact.
Canopy Host records source pins with its build and service-build data. A snapshot can represent deployed, a build, a release, a preview, or current branch heads. The first four read recorded or inferred history; heads resolves moving provider refs. Each pin includes a repository identity, commit SHA when known, provenance, and services that used it. Provenance distinguishes a commit recorded at build time from one inferred from older build data, one resolved now, and one that was never recorded. A missing historical SHA is reported as missing, not replaced with today’s branch head.
This becomes important in a multi-service release. One service may be built from a new commit while another carries an earlier image forward. The release and deployment concern the actual service artifacts and their recorded source pins, not one global branch tip. Canopy’s commit relations can connect a SHA to builds, releases, previews, and pipeline runs, with bounded results marked when cut. Its deployment comparison reads release service records when available, follows a rollback’s source release when necessary, and can fall back to service-build image references with configuration changes marked unknown. If there are no usable service records, deployment changes are unavailable; they are not an empty list presented as “nothing changed.”
The path from commit to build to artifact to release to deployment is only useful if each edge says what it actually knows. A complete source comparison cannot prove that a deployment ran it. A recorded deployment cannot fill in files omitted by a provider diff. Keeping the SHA and the completeness of the comparison separate makes both questions answerable without confusing them.
The remaining boundary
Canopy Source is a preview capability. We tested the provider contract live on GitHub and on one self-managed GitLab 19.4.1 instance, then exercised source reads and comparisons through Canopy’s API and CLI in a separate October 19 acceptance run. The replay fixtures and unit tests cover more branches than those live runs, including tree truncation and timeout classification. They are tests of our handling, not fresh evidence that every provider configuration behaves the same way.
Bitbucket Source browsing is disabled pending its own fixture and provider verification. We have not verified gitlab.com limits, a provider 429 on either fixture account, or a live GitHub tree recursion cut. GitHub’s .diff media response returned successfully for this fixture; we did not reproduce its documented timeout case. These gaps matter because a provider-side hard limit can leave information that Canopy cannot retrieve through the tested endpoint.
The original 3,100 files still exist in Git. The surprising part was that a successful comparison response could show 300 of them without saying, in a machine-readable way, that the answer was incomplete. The engineering fix starts by refusing to let the status code speak for the evidence. When we can reconstruct the file set, we say how. When we cannot, the result says how much is known and where it stopped.