How we test

Same input, same settings, every model, and a published rubric. Here is exactly how a result on this site is made.

One job at a time

We don't publish general rankings. A model that is best for realistic people can be poor at Chinese text on a poster, so every result belongs to one job, such as “posters with Chinese text” or “PPT decks”. We pick jobs from what people actually search for and struggle with, and we start with the ones where a wrong choice costs the most time or money.

How a test round works

Fixed briefs. Each job has a set of briefs (10 to 20 for video, 10 to 15 for images, 3 to 10 for writing and office work). Every model gets the same brief, the same input files and the closest equivalent settings.
Blind grading. Outputs are shuffled and labelled A, B, C. A panel of three graders scores them against the job's rubric without knowing which model made which.
Hard checks where possible. Chinese text is checked character by character. Citations are opened and verified one by one. Résumés go through an applicant-tracking-style parse check. Websites are deployed and edited.
Cost per usable result. We record every attempt, including retries, so we can report what a usable result actually costs, not just the list price.

The Reference Kit

Image-to-video and editing tests start from the same pixels for every model. Without a shared reference, a test mostly measures the prompt, not the model. The kit has:

6 synthetic characters (no real person's likeness), each with front, side and back views and 4 expressions, across ages and ethnicities, including East Asian faces.
4 products: a bottle, a phone, a shoe and packaged food, each on a clean white background.
3 locations: a café interior, a city street at night and a living room from an architectural rendering.
2 art styles: anime and 3D-cartoon versions of two characters, for comic-drama tests.
Start frames that place kit characters in kit locations, for image-to-video tests. First and last frame pairs come next.

Every image is synthetic and free to download, so anyone can reproduce a result. Made on 31 Oct 2026.

Reference Kit: Character 1 — **Character 1**East Asian woman, about 28 · Full size

Reference Kit: Character 2 — **Character 2**East Asian man, about 45 · Full size

Reference Kit: Character 3 — **Character 3**Black woman, about 34 · Full size

Reference Kit: Character 4 — **Character 4**Latino man, about 24 · Full size

Reference Kit: Character 5 — **Character 5**White woman, about 63 · Full size

Reference Kit: Character 6 — **Character 6**South Asian man, about 52 · Full size

Reference Kit: Character 1, anime — **Character 1, anime**For comic-drama tests · Full size

Reference Kit: Character 1, 3D cartoon — **Character 1, 3D cartoon**For comic-drama tests · Full size

Reference Kit: Character 4, anime — **Character 4, anime**For comic-drama tests · Full size

Reference Kit: Character 4, 3D cartoon — **Character 4, 3D cartoon**For comic-drama tests · Full size

Reference Kit: Product: facial oil — **Product: facial oil**Printed label · Full size

Reference Kit: Product: smartphone — **Product: smartphone**Wordmark and camera module · Full size

Reference Kit: Product: running shoe — **Product: running shoe**Logo on the heel · Full size

Reference Kit: Product: snack bag — **Product: snack bag**Chinese and English text · Full size

Reference Kit: Location: café — **Location: café**Morning light · Full size

Reference Kit: Location: night street — **Location: night street**Wet, neon · Full size

Reference Kit: Location: living room — **Location: living room**Architectural rendering · Full size

Reference Kit: Start frame: café — **Start frame: café**Character 1 in the café · Full size

Reference Kit: Start frame: street — **Start frame: street**Character 2 on the night street · Full size

The BiBi Score

Each job page gives every model a BiBi Score from 0 to 100, built from four parts that are always shown separately too:

Quality, 50%: the blind panel's average on this job's rubric.
Reliability, 20%: the share of attempts that were usable without retries.
Value, 20%: cost per usable result, scaled against the cheapest model that passed.
Ease, 10%: the steps, settings and limits a non-expert runs into.

The weights are published and versioned (this is BiBi Score v1.0). They never change for a sponsor. A model that can't be used in a region is labelled, not scored down.

Prompt packs and test logs

With each graded round we publish the full prompt pack and a test log with every output, so you can check our work. Test logs are snapshots and keep their date; job pages are updated in place when we re-test.

Health, study and money topics

Health pages are reviewed by a named, licensed doctor before they are published, use only de-identified sample reports, and are not medical advice.
Study and research tests are about finding real papers, checking citations and summarising. We report each tool's fake-citation rate and don't publish “write my paper” or “avoid AI detection” content.
Sponsors never see scores before publication and can't change them. Losing results are published too.

Corrections

When we get something wrong we fix it and note the change and the date on the page. Spotted a mistake? Tell us.