Applied Research Engineer, Evaluation

About Black Forest Labs

We're the team behind Latent Diffusion, Stable Diffusion, and FLUX — foundational technologies that changed how the world creates images and videos. Our FLUX models are among the most advanced in the world, and we're just getting started.

Headquartered in Freiburg, Germany with a growing presence in San Francisco, we're scaling fast while staying true to what makes us different: research excellence, open science, and building technology that expands human creativity.

Why This Role

Frontier image and video models are now good enough that the hard question is no longer whether they work, but whether a new one is actually better, and how you'd know.

Aesthetics, prompt following, editing fidelity, text rendering, character consistency: none of these collapse into a single number, and public benchmarks have fallen well behind the models they're meant to measure.

You'll work on the system BFL uses to answer that question.

Every FLUX checkpoint, every finetuning and RL run, and every competitor release goes through our evaluation stack, and you'll help build and sharpen it.

Product Research relies on what it shows to decide what to fix next, and to confirm the fix worked.

It gets harder from here. FLUX 3 is our first multimodal model, spanning image, video, audio and action, and each new modality needs its own way of being measured.

What You'll Work On

What you focus on will depend on where you're strongest and what the team needs most when you join.

In scope:

What We're Looking For

You have made meaningful contributions to the evaluation of generative models already.

You understand that a benchmark figure without careful consideration of sample size and judge agreement with humans is at best uninformed optimism and at worst dangerously misleading, and you have been the person that found that inconvenient regression that nobody wanted to talk about.

You likely have:

We'd be especially excited if you:

How We Work Together

We're a distributed team with real offices that people actually use.

This role is based in Freiburg, where you'll join us at least 2 days a week (or one full week every other week).

We'll cover reasonable travel costs to make this possible.

We think in-person time matters, and we've structured things to make it accessible to all.

We'll discuss what this will look like during our interview process.

Everything we do is grounded in four values:

If this sounds like work you'd enjoy, we'd love to hear from you.

Compensation : €150K-€220K base + equity depending on experience