Engineer / Scientist — Experimentation & Measurement

← All open roles

About Optimly

Optimly is where AI models go to learn about brands and where brands go to learn about AI models.

Optimly measures how AI models talk about brands. We ask grounded, web-search-backed questions to a panel of frontier models, score the answers against the brand's own first-party description of itself, and generate recommendations for what the brand should change.

In Optimly, the experimentation loop is a core differentiator. This role owns it.

The role

You'd own the experimentation and measurement loop inside the app — the science layer, and the engineering that makes it run. This includes metric design, attribution and causal inference, agent and judge evaluation, and the experiment surface.

What we're looking for

This role is a hybrid scientist-engineer. We're looking for:

• Strong grounding in statistics and causal inference.
• Python, SQL, data pipelines, and comfort in a production codebase.
• Direct experience evaluating LLM systems.

Nice to have: a research background (academic or industry), prior work on search or recommender measurement, published evaluation methodology.

How we work

We use AI heavily and expect you to, as well. Day-to-day work happens in agentic coding tools. We want someone who can hold both ideas at once: using these systems fluently while remaining genuinely skeptical of what they output, including the output you built the harness for.

We're small. You'll see the whole system and talk to customers.