Advanced Experimentation

Start learning
All categoriesDataExperimentation & A/B Testing

Master advanced experimentation

Basic A/B works for big effects and you need to detect smaller ones. Master sequential testing, switchback, CUPED, and holdouts, so you can run experiments most teams cannot.

Start learningSee the chapters

Overview

Basic A/B works for big effects and you need to detect smaller ones. Master sequential testing, switchback, CUPED, and holdouts, so you can run experiments most teams cannot. Octo builds this course around your role, your experience, and what you already know, so the version you get isn't the same one a beginner across the hall is reading.

What you'll learn

By the end, you'll be able to do these, not just have read about them.

  • Apply sequential testing and switchback designs correctly

  • Run holdouts and long-term tests for durable lift

  • Diagnose novelty effects, peeking, and selection bias

  • Operate an experimentation platform with rigor and trust

Who this is for

  • You're an analyst, PM, or operator who wants to stop waiting on the data team.

  • You're an engineer picking up data skills as part of a broader role.

  • You're a data professional sharpening a specific specialty.

Prerequisites

  • Solid fluency with the fundamentals, you've shipped or studied this seriously.

  • You're looking to push past intermediate, not refresh basics.

Suggested chapters

This is the typical chapter list. Your version is generated against your background and adapts as you go. It may compress, expand, or reorder these.

  1. 01

    Foundations of Advanced Experimentation

    The mental model and shared vocabulary you'll lean on for the rest of the course.

  2. 02

    Core building blocks

    The handful of moves that show up everywhere, drilled until they feel obvious.

  3. 03

    Working through real examples

    Applied patterns on examples close to the kind of work you actually do.

  4. 04

    Edge cases & failure modes

    Where the simple version breaks, and how to recognize it before it bites you.

  5. 05

    Putting it together

    Combining what you've learned into something end-to-end and defensible.

  6. 06

    Capstone

    A small project tied to your real work that proves you can use the material, not just recall it.

Real-world projects

  • 01Apply advanced experimentation to a small problem from your actual work or studies.
  • 02Produce one written or built artifact you can put on your resume, portfolio, or in a review packet.
  • 03Run a self-graded capstone against an Octo-provided rubric.

Tools & concepts

Real tools and ideas covered. Octo brings them in when they fit your stack.

  • SQL
  • Python
  • Dashboards
  • A/B testing
  • Cohorts & funnels
  • Statistical reasoning

Where this leads

  • 01

    Analyst- and DS-grade fluency

  • 02

    Self-service for product, marketing, and ops decisions

  • 03

    Foundation for advanced data specialties

Common questions

  • Is this a fixed course, or is it built for me?

    Built for you. The chapter list below is a typical outline. Your actual course is generated against your role, experience, and what you already know, then adapts as you go.

  • How long does it take?

    Most learners finish in 2–6 weeks at a normal pace, depending on the topic. Octo compresses where you're strong and slows down where you're weak.

  • Is there a fixed schedule or cohort?

    No. You start when you start. There's no live session, no calendar, no deadline.

  • Can I ask questions while I'm learning?

    Yes, every module has an AI Sidekick in the margin. Ask for a different example, push back, or get a clarifying analogy without leaving the page.

  • What do I get at the end?

    A verifiable, HMAC-signed certificate with a public verify page. It records the modules passed, scores, and capstone, not just attendance.

  • How much does it cost?

    Octo is in research preview, courses are open. We'll be transparent before pricing changes.

More in Experimentation & A/B Testing

Start learning