A/B experiments

Bind variant assignments to sessions, then every conversion, funnel step, replay, and log in that session joins the right variant at query time. Sumidata is the analytics side of A/B — your feature-flag service still decides who sees what.

01Mental model

Sumidata is a tracker, not a flag service. You tell it which variant you chose; it joins that to everything else.

An experiment assignment in Sumidata is a tiny row that says: "In session X, experiment Y was running, and the user saw variant Z." The assignment lives in a dedicated table (session_experiments), keyed by (sessionId, experimentId). Reports, the AI Analyst, and SQL Explorer join it to events, conversions, and replays on sessionId — no variant field is needed on downstream events.

02Sticky by session, not by user

First assignment per (session, experimentId) wins. A later call with a different variant is a no-op.

Sumidata dedups on (sessionId, experimentId) — the first assignment the server sees for a session+experiment pair is the one that sticks. A second call with a different variant for the same experiment in the same session returns { status: "skip" } and writes nothing. This protects analysis from:

• Flicker-reassignment when a flag evaluator returns different values on re-mount.
• Double-fires when the SDK re-runs on route change.
• Race conditions between client-side and server-side assignment.

03From the browser

One push call. The SDK fills in deviceId and sessionId automatically.

Call identifyExperiment as soon as your flag framework has resolved assignments for the current user — ideally before the first product event fires. An experiment registered after the first conversion in a session will still work for anysubsequent events in that session, but the earlier events stay unattributed (the join matches on the stored assignment row; rows don't retroactively inject into past events).

// Called once per session, as soon as your A/B framework resolves assignments
Sumidata.push('identifyExperiment', [[
  { id: 'pricing-v3', variant: 'control' },
  { id: 'onboarding-flow', variant: 'short' },
  { id: 'hero-copy', variant: 'v2' },
]])

04From the server

Post the same shape from your backend when assignments are resolved server-side.

Useful when your flag framework runs in SSR, edge workers, or an internal feature-flag service, and you want to register variants without waiting for the client. You need the browser's sessionId and deviceId — get them from the SDK on the client first (see Linking browser session to server), store them on the user row, then call this endpoint.

curl "https://api.sumidata.io/sdk/session-experiments" \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{"projectId":"…","deviceId":"…","sessionId":"…","experiments":[{"id":"pricing-v3","variant":"control"}]}'

await axios.post('https://api.sumidata.io/sdk/session-experiments', {
  projectId: process.env.SUMIDATA_PROJECT_ID,
  deviceId: user.sumidataDeviceId,
  sessionId: user.sumidataSessionId,
  experiments: flags.resolveForUser(user.id),
})

{ "status": "ok" }     // new assignments inserted
{ "status": "skip" }   // every (sessionId, experimentId) was already registered

05Integration recipes

A one-call recipe for each of the common flag frameworks.

The pattern is always the same: resolve the variant with your flag service, then hand the result to Sumidata. Examples in the browser SDK; swap to the server endpoint if the decision happens on your backend.

GrowthBook

import { GrowthBook } from '@growthbook/growthbook'

const gb = new GrowthBook({ /* … */ })

gb.setTrackingCallback((experiment, result) => {
  Sumidata.push('identifyExperiment', [[
    { id: experiment.key, variant: result.key }
  ]])
})

Statsig

import { StatsigClient } from '@statsig/js-client'

const statsig = new StatsigClient({ /* … */ })
await statsig.initializeAsync()

const pricing = statsig.getExperiment('pricing-v3')
Sumidata.push('identifyExperiment', [[
  { id: 'pricing-v3', variant: pricing.get('variant', 'control') }
]])

LaunchDarkly

import * as LDClient from 'launchdarkly-js-client-sdk'

const ld = LDClient.initialize(clientId, { kind: 'user', key: user.id })
await ld.waitForInitialization()

const variant = ld.variation('pricing-v3', 'control')
Sumidata.push('identifyExperiment', [[
  { id: 'pricing-v3', variant }
]])

Optimizely

import { createInstance } from '@optimizely/optimizely-sdk'

const optimizely = createInstance({ sdkKey: '…' })
await optimizely.onReady()

const decision = optimizely.createUserContext(user.id).decide('pricing-v3')
Sumidata.push('identifyExperiment', [[
  { id: decision.flagKey, variant: decision.variationKey }
]])

06Reading the results back

Ask the AI Analyst in plain English, or write SQL against session_experiments.

Sumidata intentionally does not expose a read endpoint for assignments — you almost never want the raw rows, you want the analysis built on top. Three ways to get that:

AI Analyst

Plain English. Works out of the box once any assignments are registered.

revenue and conversion rate by variant for pricing-v3, last 14 days

which onboarding-flow variant has the lowest drop-off on step 2?

SQL Explorer

The join is always on sessionId. A couple of useful templates:

Revenue and conversion rate by variant

SELECT
  se.variant,
  uniq(e.sessionId)                                 AS sessions,
  uniq(if(e.orderId != '', e.sessionId, NULL)) AS converting_sessions,
  sum(e.totalAmount)                                AS revenue
FROM events e
JOIN session_experiments se
  ON se.sessionId = e.sessionId
 AND se.projectId = e.projectId
WHERE se.experimentId = 'pricing-v3'
GROUP BY se.variant
ORDER BY se.variant

Funnel drop-off by variant

SELECT
  se.variant,
  e.name,
  uniq(e.sessionId) AS sessions
FROM events e
JOIN session_experiments se
  ON se.sessionId = e.sessionId AND se.projectId = e.projectId
WHERE se.experimentId = 'onboarding-flow'
  AND e.name IN ('onboard_step_1', 'onboard_step_2', 'onboard_done')
GROUP BY se.variant, e.name
ORDER BY se.variant, e.name

Replay filtering

In the Replays dashboard, filter by Experiment to scope the session list to a single variant. Great for qualitative review — watch five "treatment" sessions against five "control" back-to-back.

07Limitations

What Sumidata won't do for you — fill these in with your own tooling.

• No read endpoint for assignments. There is no GET /sdk/session-experiments. If you need the raw variant for a given session, query session_experiments directly via SQL Explorer.
• No built-in statistical significance. Sumidata stores assignments and joins them to outcomes; p-values, confidence intervals, power analysis, sequential testing — all out of scope. Export to your stats tool of choice, or ask the AI Analyst for a frequentist split by variant.
• No webhooks on new assignments. The registration is synchronous and returns a status, but downstream systems aren't notified.
• Session-scoped stickiness. If you need lifetime stickiness, resolve the variant in your flag service on every session and re-register — the first per-session assignment will match what you chose.
• 2-year retention. The session_experiments table has a 2-year TTL. Older assignments fall off automatically.

08AI-agent quick reference

Drop this into an agent prompt when automating experiment setup and analysis.

concept: Sumidata records which variant was shown; your flag service decides who sees what.

register:
  browser: Sumidata.push('identifyExperiment', [[{ id, variant }, …]])
  server: POST /sdk/session-experiments { projectId, deviceId, sessionId, experiments: [{ id, variant }] }
  response: { status: "ok" } (inserted) | { status: "skip" } (already registered)

stickiness:
  dedup_key: (sessionId, experimentId) — first-wins
  changing_variant: requires a new session (logout / reset / 30 min inactivity)
  scope: session-bound, not user-bound

timing:
  register_before: the first event you want attributed — earlier events stay unattributed
  retroactive: no — assignment rows don't alter past events

analysis:
  join: events JOIN session_experiments ON sessionId (+ projectId)
  no_variant_field_on_events: true — never stamp variant on event payloads; the join handles it
  ai_analyst: "revenue by variant for pricing-v3 last 14 days"

limits:
  - no GET endpoint — inspect via SQL Explorer
  - no p-value / CI — export to your stats tool
  - no webhooks on new assignments
  - 2-year retention (ClickHouse TTL)