A/B experiments

Bind variant assignments to sessions, then every conversion, funnel step, replay, and log in that session joins the right variant at query time. Sumidata is the analytics side of A/B — your feature-flag service still decides who sees what.

01Mental model

Sumidata is a tracker, not a flag service. You tell it which variant you chose; it joins that to everything else.

An experiment assignment in Sumidata is a tiny row that says: "In session X, experiment Y was running, and the user saw variant Z." The assignment lives in a dedicated table (session_experiments), keyed by (sessionId, experimentId). Reports and the AI Analyst join it to events, conversions, and replays on sessionId — no variant field is needed on downstream events.

02Sticky by session, not by user

First assignment per (session, experimentId) wins. A later call with a different variant is a no-op.

Sumidata dedups on (sessionId, experimentId) — the first assignment the server sees for a session+experiment pair is the one that sticks. A second call with a different variant for the same experiment in the same session returns { status: "skip" } and writes nothing. This protects analysis from:

• Flicker-reassignment when a flag evaluator returns different values on re-mount.
• Double-fires when the SDK re-runs on route change.
• Race conditions between client-side and server-side assignment.

03From the browser

One push call. The SDK fills in deviceId and sessionId automatically.

Call identifyExperiment as soon as your flag framework has resolved assignments for the current user — ideally before the first product event fires. An experiment registered after the first conversion in a session will still work for anysubsequent events in that session, but the earlier events stay unattributed (the join matches on the stored assignment row; rows don't retroactively inject into past events).

// Called once per session, as soon as your A/B framework resolves assignments
Sumidata.push('identifyExperiment', [[
  { id: 'pricing-v3', variant: 'control' },
  { id: 'onboarding-flow', variant: 'short' },
  { id: 'hero-copy', variant: 'v2' },
]])

04From the server

Post the same shape from your backend when assignments are resolved server-side.

Useful when your flag framework runs in SSR, edge workers, or an internal feature-flag service, and you want to register variants without waiting for the client. You need the browser's sessionId and deviceId — get them from the SDK on the client first (see Linking browser session to server), store them on the user row, then call this endpoint.

curl "https://api.sumidata.io/sdk/session-experiments" \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"projectId":"…","deviceId":"…","sessionId":"…","experiments":[{"id":"pricing-v3","variant":"control"}]}'

await axios.post('https://api.sumidata.io/sdk/session-experiments', {
  projectId: process.env.SUMIDATA_PROJECT_ID,
  deviceId: user.sumidataDeviceId,
  sessionId: user.sumidataSessionId,
  experiments: flags.resolveForUser(user.id),
})

{ "status": "ok" }     // new assignments inserted
{ "status": "skip" }   // every (sessionId, experimentId) was already registered

05Integration recipes

A one-call recipe for each of the common flag frameworks.

The pattern is always the same: resolve the variant with your flag service, then hand the result to Sumidata. Examples in the browser SDK; swap to the server endpoint if the decision happens on your backend.

GrowthBook

import { GrowthBook } from '@growthbook/growthbook'

const gb = new GrowthBook({ /* … */ })

gb.setTrackingCallback((experiment, result) => {
  Sumidata.push('identifyExperiment', [[
    { id: experiment.key, variant: result.key }
  ]])
})

Statsig

import { StatsigClient } from '@statsig/js-client'

const statsig = new StatsigClient({ /* … */ })
await statsig.initializeAsync()

const pricing = statsig.getExperiment('pricing-v3')
Sumidata.push('identifyExperiment', [[
  { id: 'pricing-v3', variant: pricing.get('variant', 'control') }
]])

LaunchDarkly

import * as LDClient from 'launchdarkly-js-client-sdk'

const ld = LDClient.initialize(clientId, { kind: 'user', key: user.id })
await ld.waitForInitialization()

const variant = ld.variation('pricing-v3', 'control')
Sumidata.push('identifyExperiment', [[
  { id: 'pricing-v3', variant }
]])

Optimizely

import { createInstance } from '@optimizely/optimizely-sdk'

const optimizely = createInstance({ sdkKey: '…' })
await optimizely.onReady()

const decision = optimizely.createUserContext(user.id).decide('pricing-v3')
Sumidata.push('identifyExperiment', [[
  { id: decision.flagKey, variant: decision.variationKey }
]])

06Reading the results back

Ask the AI Analyst in plain English, or write SQL against session_experiments.

Sumidata intentionally does not expose a read endpoint for assignments — you almost never want the raw rows, you want the analysis built on top. Two ways to get that:

AI Analyst

Plain English. Works out of the box once any assignments are registered.

revenue and conversion rate by variant for pricing-v3, last 14 days

which onboarding-flow variant has the lowest drop-off on step 2?

Experiments dashboard

Under Analytics → Experiments you get a list of every experiment in your window ranked by sales contrast (the spread between your best- and worst-selling variant), a per-variant detail view (devices, sessions, purchases, revenue, and conversion with a 95% confidence interval per variant), and an events-by-variant breakdown showing how each tracked event fires per device across variants. No setup — it populates as soon as assignments are registered.

Replay filtering

In the Sessions and Replays dashboards, the Experiment and Variant selects scope the session list to a single experiment and variant (their options come straight from your data). Each variant row in the experiment detail deep-links here via ?experimentId=&variant=, which pre-fills both selects — great for qualitative review, watching five "treatment" sessions against five "control" back-to-back.

07Limitations

What Sumidata won't do for you — fill these in with your own tooling.

• No read endpoint for assignments. There is no GET /sdk/session-experiments. If you need the raw variant for a given session, ask the AI Analyst for the session_experiments rows.
• Sample sizes and confidence intervals, not verdicts. The Experiments dashboard shows per-variant conversion with 95% confidence intervals and the sample sizes behind every number. It does not issue significance verdicts — no "winner", no "significant". Whether a difference is real enough to act on stays with you and your decision tooling. Sumidata is the assignment logger, not the decision engine.
• No webhooks on new assignments. The registration is synchronous and returns a status, but downstream systems aren't notified.
• Session-scoped stickiness. If you need lifetime stickiness, resolve the variant in your flag service on every session and re-register — the first per-session assignment will match what you chose.
• Plan-governed retention. Assignments are retained under your plan's data-retention policy, the same as the rest of your analytics data; older data is purged on that schedule.

08AI-agent quick reference

Drop this into an agent prompt when automating experiment setup and analysis.

concept: Sumidata records which variant was shown; your flag service decides who sees what.

register:
  browser: Sumidata.push('identifyExperiment', [[{ id, variant }, …]])
  server: POST /sdk/session-experiments { projectId, deviceId, sessionId, experiments: [{ id, variant }] }
  response: { status: "ok" } (inserted) | { status: "skip" } (already registered)

stickiness:
  dedup_key: (sessionId, experimentId) — first-wins
  changing_variant: requires a new session (logout / reset / 30 min inactivity)
  scope: session-bound, not user-bound

timing:
  register_before: the first event you want attributed — earlier events stay unattributed
  retroactive: no — assignment rows don't alter past events

analysis:
  join: events JOIN session_experiments ON sessionId (+ projectId)
  no_variant_field_on_events: true — never stamp variant on event payloads; the join handles it
  dashboard: Analytics → Experiments — list ranked by sales contrast, per-variant detail (conversion + 95% CI), events-by-variant
  session_filter: Sessions/Replays accept Experiment + Variant; deep link ?experimentId=&variant=
  ai_analyst: "revenue by variant for pricing-v3 last 14 days"

limits:
  - no GET endpoint for raw rows — inspect via the AI Analyst
  - CI + sample sizes shown, but no significance verdict — decision stays with you
  - no webhooks on new assignments
  - retention governed by your plan's data-retention policy