Ctrlk
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Analytics & Detail Page

Once a Split URL test is created, opening it takes you to its detail page: a header with test info and controls, an outcome summary, side-by-side variant comparison cards, and a tabbed set of analytics views.

This page covers what is shown on the detail page and exactly how every number is calculated. For the creation wizard, see Split URL Test.

  • Test name, status badge (Draft / Preview / Active / Paused / Completed / Archived), and a purple Cross-store badge when cross-store tracking is enabled.

  • Quick facts row: test type, traffic split per variation (e.g. 50% / 50%), the traffic limit %, and either Created: <date> (running tests) or Duration: <start> – <end> (paused/completed tests).

  • A date-range picker for the analytics window, defaulting to the test's creation date through today (or through the test's end date once paused/completed).

Plan gating: on the Free plan, the date picker is visible but disabled (clicking it opens an upgrade prompt) — Free-plan analytics always show the full lifetime range.

Status controls (shown conditionally by test status)

Status
Available actions

Draft / Preview / Paused

Start Test

Active

Pause, Stop

Paused / Preview

Stop

Completed

Archive

Archived

Unarchive

Active

Settings (edit name/goal/variations/traffic limit without leaving the page)

Not Active, not Completed

Edit (reopens the full creation wizard)

Preview / Active

Preview (opens the preview modal — see below)

A Move to preview control is always available to pull a live test back into preview mode.

If the Shopify app embed isn't installed when you try to start/resume a test, a modal explains this and offers to open the theme editor; the page polls in the background and automatically resumes the test once the embed is detected as installed.

Outcome Summary (Metrics tab only)

A sticky bar summarizing the test's current read, shown only while viewing the Metrics tab (and hidden while the test is still collecting its first orders):

  • Status: Collecting data (fewer than 5 total orders) / Running / Paused / Ended.

  • Leading: the name of whichever variation is currently ahead on the test's configured goal metric (or the control's name if no variant beats it).

  • Expected uplift: the leader's % improvement over control on the goal metric.

  • Metric tracked: the goal set when the test was created — Conversion Rate, Revenue Per Visitor, Avg. Order Value, Add to Cart Rate, or Profit Per Visitor (defaults to Avg. Order Value).

  • Confidence label: e.g. "87% probability to win · Significant".

  • Time remaining: days left until the test's end date, or "Running" if untimed, or "Ended".

  • Sessions and Orders counted so far.

  • A View details link that jumps to the Statistical Significance tab (or prompts an upgrade if that tab is plan-locked).

A footer note states which metric is being tracked as the primary goal and that changing it requires editing the test.

Formulas:

  • Leading variation = whichever variant has the highest value for the goal metric; control is "leading" only if the best variant's improvement is negative.

  • Displayed confidence % = the leader's win probability (see Statistical Significance below), flipped to represent the leader's own chance of winning when control is ahead, capped at 99%.

  • Confidence tiers: ≥ 95% → Highly Significant · ≥ 80% → Significant · below that → Less Significant.

Variant Comparison Cards

Directly below the header, one card per variation (Control first, then each variant), each showing:

  • A live screenshot of that variation's destination page (with a manual refresh button).

  • The variation's name and letter (A, B, C…), color-coded consistently with the rest of the page.

  • Visitors and their share of total test traffic.

  • Orders, Revenue, Revenue Per Visitor (RPV), and — for variants — the % lift in RPV vs. control.

  • A win probability — the % chance this variation is genuinely better than control on RPV.

  • A kebab menu with a Preview link that opens that exact variation on the live storefront.

Formulas:

  • Visitors % = Variation Sessions ÷ Total Sessions (across all variations) × 100

  • RPV = Revenue ÷ Visitors

  • RPV lift (variant) = ((Variant RPV − Control RPV) ÷ Control RPV) × 100

  • Variant win probability = the RPV metric's win probability from the significance engine (see below), capped at 99%.

  • Control's win probability ≈ the probability control beats every variant at once, approximated as Π (1 − variant's RPV win probability) across all variants (assumes each pairwise comparison is independent), capped at 99%.

Analytics Tabs

Fourteen tabs sit below the comparison cards. On the Free plan, everything except Metrics is locked (blurred, with an upgrade prompt on click). Attribution/Traffic Source additionally requires attribution to be enabled on the plan; Subscriptions requires the subscriptions add-on tier.

Metrics tab

  • A "What to look at" selector (Conversion Rate / Revenue / AOV / Orders) that drives a daily trend chart below it, plotted per variation.

  • A full metrics table, one column per variation, one row per metric, each with a lift badge vs. control:

Metric
Formula

Sessions

raw count

Link Clicked

raw count

Has Scrolled

raw count

Collection Viewed

raw count

Product Viewed

raw count

Add To Cart

raw count

Checkout Started

raw count

Contact/Address/Shipping/Payment Info Submitted

raw count (per checkout step)

Orders

raw count

Revenue

sum of order totals

AOV

Revenue ÷ Orders

RPV

Revenue ÷ Sessions

Gross Profit

see Gross Profit formula below

PPV

Gross Profit ÷ Sessions

ATC Rate

Add to Cart ÷ Sessions × 100

Checkout Started Rate

Checkout Started ÷ Sessions × 100

Conversion Rate

Orders ÷ Sessions × 100

Click Through Rate

Link Clicked ÷ Sessions × 100

Leave Without Action Rate

(Sessions − Link Clicked) ÷ Sessions × 100

If store cost data isn't configured, Gross Profit and PPV cells show a link to set it up instead of a number.

Orders tab

The raw list of orders attributed to this test, filterable/searchable by product, with per-order revenue-distribution and unit-quantity charts and an orders-over-time chart.

Traffic Source tab

Breaks down sessions and orders by acquisition channel or referring site, with a per-variation funnel (View Product / Add to Cart / Begin Checkout / Abandoned Cart rates) at each source.

Abandoned Carts tab

Checkouts that were started but never completed, attributed to a variation and broken down as a comparison ("Page impact on abandonment") between variants, using each abandoned checkout's line items, subtotal, and product images.

Product Revenue tab

Revenue and order counts broken down by individual product, for the orders attributed to this test.

Revenue tab

A full revenue/profit breakdown per variation:

Field
Formula

Gross Revenue

sum of line-item original price × quantity

Net Revenue

sum of order totalPriceSet (what the customer actually paid)

Discounts

sum of order-level discount totals

Refunds

sum of refund totals per order

Refunded Merchandise

sum of refunded line-item subtotals

Net Sales

Gross Revenue − Discounts − Refunded Merchandise

Gross Profit

see Gross Profit formula below

Gross Profit Margin

Gross Profit ÷ Net Sales × 100

AOV

Net Revenue ÷ Orders

Avg Discount

Total Discounts ÷ Orders

Avg Shipping

Total Shipping ÷ Orders

Avg Tax

Total Tax ÷ Orders

A daily revenue/orders time-series chart is included per variation.

Time Series tab

Four charts (toggle between line/bar), one metric each, plotted daily per variation: Visitors, Conversions, Add to Cart, Revenue.

Conversion Funnel tab

A line chart with one line per variation across four steps — Visitors (100%) → Add to Cart → Begin Checkout → Order Complete — plus a table of the same rates with each variant's % improvement vs. control shown inline, a "Key Insights" panel (biggest drop-off step; best-performing variant), and a performance summary (step count, variation count).

Formulas:

  • Each step's rate (except Visitors) = step event count ÷ that variation's sessions × 100.

  • Improvement vs. control (per step, per variant) = ((Variant rate − Control rate) ÷ Control rate) × 100.

Statistical Significance tab

A confidence-level selector (90/95/99%), summary cards (control sample size, variants tested, count of significant results, the Bonferroni-adjusted threshold in use), a grouped bar chart comparing control vs. every variant across Conversion Rate, Add-to-Cart Rate, AOV, and RPV, and a full per-variant table (Control value, Variant value, Improvement %, Win Probability, Credible Interval, Risk, Significant Yes/No) for every tracked metric, plus Profit Per Visitor when cost data is configured.

This is the statistical engine behind every "win probability" and "lift" figure elsewhere on the page. Full formulas are in the Statistical Significance Formulas section below.

Subscriptions tab

Breaks orders into All / One-time / Subscription, per variation, with a Take rate column, a pie chart of subscription orders by variation, and summary cards (total subscriptions, subscription revenue, subscription take rate, average subscription value).

Formula:

  • Take rate = Subscription orders for a variation ÷ all orders for that variation × 100

Visitor Type tab

The same per-variation breakdown table (Visitors, Orders, Revenue, Conversion Rate, RPV, Gross Profit per Visitor, and Lift vs. control), split into All / New / Returning visitors, plus a pie chart of orders by visitor type. New vs. returning is read from an order-level tag set at checkout, falling back to the customer's historical order count.

Device Type tab

The same breakdown table and pie chart, split by All / Desktop / Mobile / Tablet instead of visitor type.

Shipping tab

Per-shipping-method metrics (order count, total and average shipping revenue collected, % of orders with free shipping) plus a per-variation comparison and a metric-selector time series:

Field
Formula

Avg Shipping Revenue

Total Shipping Revenue ÷ Orders

Free Shipping %

Orders with $0 net shipping ÷ Orders × 100

Shipping Revenue per Order (per variation)

Total Shipping Revenue ÷ Orders for that variation

Discounts tab

Per-variation discount usage:

Field
Formula

% of Orders with Discount

Discounted Orders ÷ Orders × 100

Avg Discount

Total Discount ÷ Discounted Orders

Discount % of Gross Revenue

Total Discount ÷ Gross Revenue × 100

Discount % of Gross Profit

Total Discount ÷ Gross Profit × 100

Preview Modal

Opened from the Preview button in the header (only available once a test has reached preview or active status). Shows the trigger condition (matching type + base URL) and, for every variation, its trigger URL, its redirect destination (or "Stays on (Control)" for the control), a preview-mode link, and a button to open that variation in a new tab.

Shared Formulas Reference

Gross Profit

This deliberately stops at gross profit — ad spend and other operating costs aren't tracked, and total revenue (including tax) is never used as the base, since that would inflate the margin.

Lift / Improvement (used throughout)

Statistical Significance Formulas

Two different engines are used, depending on the metric type:

1. Bayesian Beta-Binomial model — used for rate/proportion metrics (Conversion Rate, Add-to-Cart Rate):

2. Welch's t-test — used for continuous $ metrics (AOV, RPV, PPV), which don't fit a binomial model:

RPV and PPV pad every non-purchasing session in as a $0 observation, so the mean is per-visitor, not per-order.

Multiple-comparison correction (Bonferroni):

This tightens the bar as more variants are added, so testing more variants at once doesn't inflate the chance of falsely declaring a winner.

Win Probability = the probability a variant is genuinely better than control. Risk of Launch = the probability that launching the variant would actually hurt your metrics (1 − Win Probability). Every variant is compared independently, pairwise, against control — variants are never merged together.

PreviousHow to Setup
NextPrice Test Getting Started

Last updated