CLI command
experiments factors
A multivariate experiment's factor effects: each factor's levels pooled across the variants that show them, and the leading level once it clears every interval.
Invocation#
echo-cli experiments factors --experiment <experiment-id> --factors <name,name> --sigmas <n>Inputs and output#
Writes experiment_id, sigmas and factors, one per factor named: factor, levels (level, the variants showing it, exposures, conversions and rate, best first), comparisons (the leading level against every other: difference of rates, low and high at --sigmas standard errors) and leading (the leading level, or null while an interval still reaches 0).
State effect#
The command calls Echo Web's authenticated /api/experiments/factors with ECHO_INTERNAL_BEARER at ECHO_WEB_BASE_URL; the Multivariate factors section of the tracking screen and of Echo Desktop's GEO screen call the same operation, and echo-mcp reads it as echo_experiment_factors. A multivariate experiment's variants are combinations whose configs state a level of every factor; the effects pool, per level, the exposures and conversions experiments outcomes counts for each variant from stored rows. It stores nothing.
Refusals#
- A wrong invocation is refused before anything is sent: "experiments factors needs <flag>" and "--sigmas must be a number of standard errors; got <value>".
- Echo Web answers a bad request with "experiment_id is required: …", "factors is required: the names the variants' configs give the factors, such as headline,button", "sigmas must be a positive number: how many standard errors each interval spans", "variant <key>'s config states no level of <factor>; a multivariate variant states every factor" or the reward refusals of experiments outcomes; and not found with "no adaptive experiment has id <id>; echo-cli experiments lists them" or "experiment <id> has assigned no subject yet, so there is nothing to compare". The command prints Echo Web's sentence and exits 1.