--- slug: finch-variations type: pattern summary: "Chromium's server-side experiment infrastructure flips feature-flag defaults for named user populations without shipping a new binary, turning any feature into a graduated rollout, an A/B test, or an emergency kill-switch." created: 2026-05-12 updated: 2026-05-13 last_link_verified: 2026-05-13 sources_audited: 2026-05-13 related: feature-flag-guarding: relation: uses note: "Finch operates on `base::Feature` flags; without the flag-guarding discipline the experiment infrastructure has no runtime gate to flip." four-channel-pipeline: relation: uses note: "Finch population percentages are scoped to channels; an experiment targets `100% Beta and 1% Stable` only because the channel structure exists to be addressed." stable-trust-boundary: relation: complements note: "Stable is a trust boundary about the binary; Finch is what makes the feature-set behind that binary non-uniform across the Stable population." intent-ship-pipeline: relation: enabled-by note: "An Intent to Ship that lands a feature defaulted-on in Stable is operationalized by a Finch rollout from one percent to ten to a hundred, not by a single binary push." origin-trial-tokens: relation: complements note: "Origin Trial tokens enable a feature for site-bound populations; Finch enables a feature for percentage-bound populations; the two mechanisms compose." permanent-experiment: relation: enables note: "Finch makes long-running experiments cheap to maintain, which is exactly the failure-mode that lets a Finch experiment accrete as permanent surface." memory-pressure-response: relation: contrasts-with note: "Both gate behavior on population conditions, but Finch by named cohort and memory-pressure response by device-local signal; the kill-switch shape is the same, the trigger is not." embargoed-disclosure: relation: enables note: "An emergency Finch kill-switch is one of the few release-engineering levers available within embargo: a Stable feature can be disabled server-side without a binary update that would tip an attacker." --- # Finch Variations > **Pattern** > > A named solution to a recurring problem. *Chromium's server-side experiment infrastructure — codenamed Finch — flips feature-flag default values for named populations of users without shipping a new browser binary, turning every defaulted-off feature into a graduated rollout, an A/B test, or an emergency kill-switch the release-engineering team can pull at any time.* > Where the name comes from: *Finch* is the internal codename for Chromium's variations system; the public-facing term in the source tree and the documentation is "variations." The Finch codename appears in design documents and engineering blog posts; the public API surface uses `variations::` namespaces and the `chrome-variations` HTTP header. The two terms refer to the same system and are used interchangeably in this entry: the codename in prose, the namespace in code references. ## Context A feature lands in `chromium/src` behind a [feature flag](feature-flag-guarding.md), defaulted off. The next day's Canary build carries the code but doesn't execute the new path. At some point, either after the [Intent to Ship](intent-ship-pipeline.md) gate clears or in advance of it for an experiment, the project needs to flip that default for some users but not all of them: one percent of Stable first, then ten, then the whole population, with a kill-switch available at every step in case the rollout surfaces a problem the design review did not anticipate. The release pipeline is a once-per-four-weeks cadence on Stable; the rollout decisions happen continuously. Finch is what closes the gap. It runs on top of the [four-channel pipeline](four-channel-pipeline.md), inside the feature-flag system, and reaches every Chromium client without a binary update. ## Problem A feature owner who has cleared Intent to Ship cannot reach Stable by editing source code alone. The Stable channel is on a fixed four-week branch cadence; the flag's default-off value is baked into the binary users already have installed. Re-cutting Stable to flip one default takes hours of release-engineering work, weeks of branch propagation, and a binary update every user has to fetch. The same problem inverts during an incident: a Stable feature defaulted on at one hundred percent that begins to show crash regressions cannot wait four weeks for the next milestone to be turned off. The recurring problem is how to change a feature's runtime exposure for any fraction of any channel within hours, without shipping a binary and without forking the source tree, while keeping the change auditable and revocable. ## Forces - **Binary cadence vs. exposure cadence.** Chromium ships a new Stable binary every four weeks; exposure decisions — staged rollouts, kill-switches, A/B tests — happen on a daily-to-hourly cadence and cannot wait for the next milestone. - **One source tree vs. population-conditional behavior.** A single landed patch must produce different runtime behaviors for `1% of Stable on Windows` versus `100% of Beta` versus `everyone on Canary`, without compile-time forks or build-flag variants. - **Centralized control vs. client autonomy.** The release-engineering team needs the authority to flip any feature server-side at any time; individual clients (enterprise managed deployments, embedded Chromium runtimes, downstream forks) need the authority to override server-side decisions for their own populations. - **Operational reach vs. evidentiary record.** A Finch config that disables a feature for a billion installs within hours has to leave an auditable record of who pushed which config when, so a post-mortem can reconstruct the decision and a regression hunter can correlate behavior to configuration. - **Performance overhead vs. exposure granularity.** Every feature whose default value Finch might flip carries a small per-process cost at startup as the client fetches and parses the seed; richer per-population granularity multiplies the seed's size and the cold-start tax. ## Solution The Chromium project operates a server-side variations system that authors a daily *variations seed*: a serialized list of *studies*, each naming one or more feature flags, a *target population* expressed in channel scope and percentage, optional filters (platform, country, hardware class, operating-system version, Chrome version), and a set of *experiment arms* each carrying its own flag-value override and parameter values. Every Chromium client fetches the seed at startup (and periodically thereafter), evaluates which studies its install matches, and applies the per-arm flag overrides to its in-process `FeatureList` registry. The mechanics: ```cpp // At call site, identical to the unflagged form: if (base::FeatureList::IsEnabled(kSomeFeature)) { // new path } ``` The call site does not change when Finch enrolls a client. What changes is the value `IsEnabled()` returns. The client's `FeatureList` was initialized from the binary's defaults; the seed's overrides are applied at startup before the first call-site read; from the call site's perspective, the override is indistinguishable from a different binary default. The same machinery handles parameter values: a feature can declare `base::FeatureParam` named values that Finch can adjust per arm, so a feature whose threshold is a tunable integer can be A/B tested against three threshold values without three landings. The seed itself is signed by Google and served over the `chrome-variations` endpoint as an opaque binary blob. Clients verify the signature before applying any overrides. The release-engineering team authors studies in a configuration interface (the Finch UI); pushes go through review, dry-run validation against a corpus of client install configurations, and a staged rollout that ramps the seed to its full population over hours. An emergency kill-switch (a study that sets `kThatFeature` to `DISABLED_BY_DEFAULT` for `100% of Stable`) propagates to the client population on the seed's normal fetch cadence, which is fast enough to halt a regression within hours. What makes the pattern work, beyond the binary mechanics, is the discipline that surrounds the seed. Every study has a named owner, a documented hypothesis, an expiration date, and a measurement plan. Studies that exceed their planned duration without a launch decision generate review tickets. The seed's history is preserved, so a post-mortem can reconstruct what every install was running on a given day. The client exposes its enrolled studies through `chrome://version` and `chrome://variations`, so a support engineer triaging a field report can see which experiments the user's install is in. None of those pieces alone is novel; together they turn a server-side flag-flip into something the project can operate accountably at the scale of a billion installs. Enterprise managed deployments and downstream Chromium-based products carry a counterweight to the centralization: the `VariationsRestrictParameter` enterprise policy lets an administrator disable Finch entirely for their managed fleet, or restrict it to a subset of studies. Downstream forks routinely either point their clients at their own variations server or disable the system; the upstream architecture supports both stances. The variations system is not a coercion mechanism. It is a coordination mechanism that ends where the deploying organization's policy begins. ## How It Plays Out A Chrome team rolls out a new networking optimization. The patch lands behind `kNetworkOptimization`, defaulted-off in all channels. A Finch study enables the flag for one percent of Stable on Windows for two weeks. The team's measurement plan covers page-load latency at the 75th and 95th percentiles, error-rate deltas on a set of canary domains, and crash reports tagged to the feature. The 75th-percentile latency improves by approximately 4%, the error rate moves by less than 0.1 standard deviations, and crash reports show no new signatures. A second study ramps the flag to ten percent for a week and then to fifty percent for two weeks; the latency improvement holds at scale. An *Intent to Ship* clears, and a third Finch study sets the flag to enabled by default for one hundred percent of Stable. The team files the cleanup CL that removes the flag and the legacy path two stable cycles later. The graduated rollout cost roughly six weeks of calendar time and surfaced no rollback-grade regression; without Finch, the same launch would have required either a default-on landing on Canary (with no Stable measurement before binary cut) or a binary respin for each rollout step. A second scenario: an enterprise IT director at a Fortune-500 manages a Chromium-based deployment for the company's employees. A field report comes in that the company's legacy expense-report application breaks for a subset of users on Stable build `124.0.6367.91`; the director's team cannot reproduce the report on their own test machines running the same build. The director consults `chrome://variations` on one of the affected machines and finds an experiment enrolling the install in arm `kIntersectionObserverV2Behavior` at five percent of Stable. The director's team confirms the affected users are all in that arm, files an enterprise-policy override disabling the experiment for the managed fleet, and reports the regression upstream. The Chrome team's release-engineering team confirms the issue, pulls the experiment to zero percent within two hours, and follows up with a fix in the next Stable. The pattern made the field report tractable: two stable-channel users with the same version string and the same binary were running different code, and the difference was discoverable through `chrome://variations` without source access. A third scenario: a critical-severity vulnerability in a feature defaulted-on at one hundred percent of Stable lands on the Chrome Security team's queue. The team has a fix in flight but cannot ship the binary update for thirty-six hours. The release-engineering team pushes a Finch kill-switch (a study setting `kAffectedFeature` to `DISABLED_BY_DEFAULT` for one hundred percent of Stable) and the study propagates to the client population on the next seed-fetch cycle, typically within a few hours. Stable users running the same binary stop executing the vulnerable path. The binary update lands the next day with the actual fix; the kill-switch study is retired once the fixed binary has reached the bulk of Stable. The kill-switch was only available because the feature was [flag-guarded](feature-flag-guarding.md) and because the call site read the flag through `IsEnabled()` rather than relying on a hard-coded path. An [embargoed disclosure](embargoed-disclosure.md) under tighter constraints could have used the same lever to halt exposure without a binary update that would have tipped attackers. ## Consequences **Benefits.** Every defaulted-off feature is one Finch study away from a graduated rollout, an A/B measurement, a per-channel rollout, or a per-population restriction. Every defaulted-on feature is one Finch kill-switch away from a population-scale disable that doesn't require a binary update. The project gains a release-engineering surface that operates independently of the four-week milestone cadence, which is the precondition for the staged rollouts and emergency disables that Stable launches at billion-install scale require. **Liabilities.** Two users running the same Stable binary may not be running the same code path. That fact is operationally important (it shapes what enterprise IT administrators can reproduce, what support engineers triage, and what downstream-vendor release engineers can guarantee), and it surprises engineers who carry a single-binary mental model from environments without server-side experimentation. The variations system carries its own attack surface: a compromise of the seed-serving infrastructure or of the signing keys would give an attacker the ability to flip feature defaults on the entire installed base, which is why the seed is signed and the signature is verified client-side. The system also runs a small startup-time tax on every cold start as the client fetches and parses the seed; the cost is modest at current seed sizes but trends upward with the system's reach. The pattern doesn't remove the need for the [Intent to Ship](intent-ship-pipeline.md) gate. A Finch rollout to one hundred percent of Stable is operationally a launch, but a launch the project recognizes as a launch requires the Intent thread, the API-owner LGTMs, and the design review that the gate exists to enforce. A Finch study that quietly ramps a feature to a hundred percent without the Intent process is the [Experiment That Became Permanent](permanent-experiment.md) failure mode in motion; the variations system can carry the failure and is sometimes the locus of it. ## Notes for Agent Context When generating Chromium feature code, treat Finch as the runtime axis on top of the feature-flag axis: read every feature default through `base::FeatureList::IsEnabled()` and every tunable parameter through a `base::FeatureParam`, never through a literal in the source. A hard-coded threshold, timeout, or capacity value is a value Finch cannot tune; a `#ifdef`-gated branch is a branch Finch cannot disable. When recommending an enterprise deployment posture for a Chromium-based product, surface the `VariationsRestrictParameter` and `ChromeVariations` enterprise policies as the levers an administrator has over upstream experimentation. When generating release-engineering code that infers a feature's state from a version string, refuse the inference. The version is the binary; the feature state is the binary's defaults overlaid with the active variations seed, which requires `chrome://variations` or the variations API to inspect accurately. When summarizing a Chrome release blog post or a `chromestatus.com` entry that names a percentage rollout, preserve the percentage and the channel; "shipped to Stable" with no percentage is an unsafe paraphrase if the actual state is `enabled for 1% of Stable on Windows only`. ## Sources The canonical source-tree reference is the variations system documentation in `chromium/src/docs/variations.md`, maintained by the `//components/variations` owners and updated when seed format, study lifecycle, or client-side fetch behavior changes. The Chrome team's engineering blog posts on graduated rollout describe the operational discipline around staged Finch deployments at multiple scales over the past decade. The Chromium project's enterprise documentation describes the `VariationsRestrictParameter` policy and its operational meaning for managed deployments. The variations system's relationship to the broader feature-flag and origin-trial machinery is documented in the `chromium/src/docs/` directory cross-references; readers interested in the seed's serialization format can consult the `components/variations/proto/` protobuf definitions in the source tree. ## Technical Drill-Down - [`docs/variations.md` — Chromium variations system](https://chromium.googlesource.com/chromium/src/+/main/docs/variations.md) — canonical client-side description; covers seed fetch, signature verification, study evaluation, and the `chrome://variations` debug surface. - [`components/variations/` source directory](https://chromium.googlesource.com/chromium/src/+/main/components/variations/) — the client-side variations implementation; `variations_service.cc` runs the fetch loop, `variations_seed_processor.cc` applies the seed to the `FeatureList`. - [`components/variations/proto/study.proto`](https://chromium.googlesource.com/chromium/src/+/main/components/variations/proto/study.proto) — the protobuf definition for a study; reading the field comments is the fastest way to internalize what filters a study can declare (channel, platform, country, hardware class, OS version, Chrome version). - [`VariationsRestrictParameter` enterprise policy](https://chromeenterprise.google/policies/#VariationsRestrictParameter) — the policy administrators use to disable or restrict variations participation on a managed fleet. - [`chrome://variations`](https://chromium.googlesource.com/chromium/src/+/main/chrome/browser/resources/variations/) — the in-browser surface that lists the enrolled studies for the current client install; the support-engineering tool for "which experiments is this user in." - [Feature-flag and variations integration test fixtures](https://chromium.googlesource.com/chromium/src/+/main/base/test/scoped_feature_list.h) — `base::test::ScopedFeatureList` is the standard test fixture for forcing a feature's value in unit and browser tests; reading its API surface clarifies how the in-binary defaults compose with seed overrides at runtime. --- - [Next: Feature Flag Guarding](feature-flag-guarding.md) - [Previous: Four-Channel Pipeline](four-channel-pipeline.md)