Methodology

How TrackSOTA collects, normalizes, and publishes leaderboards.

Where do the numbers come from?

Top3D (top3d.ai) — direct structured data or llms.txt for Textured; other modes (Geometry, Low-poly, Segmentation, Image-to-3D) are periodically bootstrapped from public leaderboards.
LM Arena (arena.ai) — text-to-image, text-to-video, image-to-video, image-edit, video-edit. We take public leaderboard tables as snapshots because structured APIs are often not available.

How often is it updated?

Top3D textured is near-live when possible. Arena boards are snapshotted when live scraping is feasible. Full rebuilds are triggered manually or via scripts. The dataset carries the exact updated_at timestamp.

Are scores transformed?

No. We preserve original scores and rank order from the source. The only normalization is consistent column naming and addition of domain / source metadata.

What does “Approximate ELO” mean?

Some Top3D modes use bootstrap estimates because the live arena HTML is blocked or not machine-readable at scrape time. These are clearly marked in ingest metadata as bootstrap. They are useful for relative ordering but have higher uncertainty than live ELO.

Can I suggest a correction?

Yes. Use the Suggest edit button — it goes directly to the Hugging Face dataset discussions. All changes are reviewed and the canonical Parquet is updated.

What is considered “open”?

A model is marked “Open” when weights and/or code are publicly available under a permissive or research-friendly license (Apache 2.0, MIT, etc.). This flag comes from the source or public model cards.

Snapshots vs live data

When structured access is unavailable, we periodically snapshot the public HTML tables. This is explicitly noted per task (“snapshot (arena.ai)”). Live sources are preferred when possible.

Dataset is the source of truth: DenisKochetov/tracksota