Browse benchmarks →

Submit a benchmark result

Platform

  1. Install MirrorQ

    Install from GitHub with access to the private repository.

    git clone https://github.com/QudeLeap/mirrorq-cli.git
cd mirrorq-cli
cargo install --path . --locked --force
mirrorq --version

    This guide uses mirrorq 0.1.4-rc.3.

  2. Sign in

    Approve the sign-in request in your browser.

    mirrorq login

  3. Choose a task

    Use the full VALIDATOR/TASK_ID from the list. 

    mirrorq list

  4. Prepare your program

    Follow the format guide in Tasks & Evaluation. Your program must read the dataset path and write its result to the output path.

    your-program <dataset-directory> <output-file>

  5. Check with public sample data

    Use the sample and checker for your input view in Tasks & Evaluation, under Task resources, when provided.

  6. Submit privately

    Set VALIDATOR/TASK_ID and VIEW from step 03, and replace python solver.py with your program.

    Omit --view for tasks with only one input view.

  7. Publish a completed result

    In My Submissions, choose Publish on a completed run, or use its submission ID below. Results stay private until published. 

    mirrorq publish --submission-id SUBMISSION_ID

  8. Compare published results

    Compare scores for the same task and review its ranking rules.

Troubleshooting

  1. Not signed in

    mirrorq login

  2. Output rejected

    Check the error in My Submissions and correct your output using the task's format guide.

  3. Check a run

    Check its status using the submission ID from My Submissions.

    mirrorq status --submission-id SUBMISSION_ID

  4. Stop an unfinished run

    Use Cancel in My Submissions, or run:

    mirrorq cancel --submission-id SUBMISSION_ID

How scoring works

A score calculated on your computer from public sample data is a local score. It is not uploaded automatically and cannot be imported as a verified score.

  • mirrorq submit downloads a freshly generated blinded dataset.
  • It runs your program on your machine and automatically uploads its output.
  • The server scores that output. The answer key never leaves the server.
The server sends blinded data to a developer and scores the returned results. The answer key stays on the server.

Citation

If MirrorQ results appear in your work, please cite:

@misc{mirrorq,
  title  = {MirrorQ},
  author = {{QudeLeap}},
  year   = {2026},
  url    = {https://bench.qudeleap.com}
}