Organize a benchmark

  1. 01

    What qualifies

    The task must be blind-scoreable: a server-held answer key must be enough to check a submission.

    Every attempt must use a freshly generated instance.

    Participants submit only their results, computed locally, never the instance or answer key.

  2. 02

    Implement the validator contract

    Implement describe, generate, and score in one executable, following the Validator Contract. Use exit code 0 for success, 2 for an invalid submission, and any other code for a validator fault.

  3. 03

    Run the conformance checks

    Check the exact container image you will publish. The server repeats these checks during publishing. 

    mirrorq-validator --image registry.example/your-validator:1.0.0
  4. 04

    Publish

    Set the container entrypoint to your validator executable, push the image to a registry, and submit it on the publish page. The server pins the image to a digest and reports each check before the benchmark goes live. For API integration, see the API docs.