Issues

• [Bug] Native ai2d benchmark shows letter-masked diagrams on questions that need the label text

  modelscope/evalscope#1847 · Filter by author surgan12 opened on Oct 12, 2026

• [Bug] Terminal-Bench 使用 Codex 智能体时，Harbor 0.24.0 因不支持 max_turns 参数导致评测失败

  modelscope/evalscope#1836 · Filter by author zhawlll opened on Oct 10, 2026
  1

• 任务是否可以获取当前正在进行的任务？

  enhancement
  modelscope/evalscope#1781 · Filter by author baigongli opened on Sep 26, 2026

• 增加模型评估哪些数据集是必做的

  enhancement
  modelscope/evalscope#1779 · Filter by author baigongli opened on Sep 25, 2026

• 是否接入jev类选择模型

  enhancement
  modelscope/evalscope#1762 · Filter by author zhawlll opened on Sep 22, 2026
  1

• mv_bench 评测报错，fine_grained_pose 子数据集截断

  modelscope/evalscope#1708 · Filter by author Sunr1seee opened on Sep 7, 2026
  1

• [bug] omni_doc_bench_v1_6，GLM-OCR两阶段模型，测评精度数值异常

  modelscope/evalscope#1682 · Filter by author simplew2011 opened on Aug 31, 2026
  1

• perf(multi-turn): warmup still drains before the measured window, and benchmark conversations pay a cold first turn

  modelscope/evalscope#1648 · Filter by author Yunnglin opened on Aug 27, 2026
  1

• Followup to #1629: CICD Improvements

  modelscope/evalscope#1631 · Filter by author Moenupa opened on Aug 25, 2026
  1

• ifbench评测集指标问题

  modelscope/evalscope#1606 · Filter by author joan126 opened on Aug 20, 2026
  7

• Cross-project discussion: reproducibility, uncertainty and benchmark provenance

  modelscope/evalscope#1582 · Filter by author KeilerHirsch opened on Aug 14, 2026
  1

• [Feature] Framing-robustness benchmark: does a judge's moral verdict survive the act being reworded?

  modelscope/evalscope#1540 · Filter by author LarytheLord opened on Aug 4, 2026
  2

• [需求]EvalScope 模型推理性能评测支持 Multi-LoRA 服务的性能评测

  modelscope/evalscope#1493 · Filter by author UMR19 opened on Jul 16, 2026
  2

• [Feature ] video generate datasets support

  enhancement
  modelscope/evalscope#1452 · Filter by author simplew2011 opened on Jul 2, 2026
  1

• Feature proposal: Benchmark pruning via variance-weighted stratified sampling

  enhancement
  modelscope/evalscope#1393 · Filter by author Shashank-mankala1 opened on Jun 2, 2026

• 希望支持tts模型的基准，例如：Seed-TTS-Eval，CPP / g2pM

  enhancement
  modelscope/evalscope#1360 · Filter by author AuFlow opened on May 22, 2026
  1

• 最近很多新模型在toolcall / 代码生成等任务上表现优异，希望evalscope能集成更多相关数据集的支持

  enhancement
  modelscope/evalscope#1278 · Filter by author fankexinxin opened on Apr 8, 2026
  2

• Qwen3_VL_Embedding/Reranker支持

  enhancement
  modelscope/evalscope#1151 · Filter by author simplew2011 opened on Jan 16, 2026

• Audio ASR benchmarks (LibriSpeech / TORGO) fail with 400 Bad Request when using sglang server mode

  modelscope/evalscope#1108 · Filter by author penguinwang96825 opened on Dec 23, 2025
  1

• BFCL-v4评测复现失败

  modelscope/evalscope#1100 · Filter by author Yjonben opened on Dec 22, 2025
  6

• 文生图模型无法调用本地部署的服务

  enhancement
  modelscope/evalscope#1040 · Filter by author lht947590837 opened on Dec 3, 2025
  1

• 增加对qwen-image模型的性能测试

  modelscope/evalscope#979 · Filter by author allenhula opened on Nov 17, 2025

• [🎯 Roadmap] EvalScope Roadmap

  modelscope/evalscope#951 · Filter by author Yunnglin opened on Nov 4, 2025
  3

• Question: standard vs CoT prompting in API calls

  enhancement
  modelscope/evalscope#930 · Filter by author penguinwang96825 opened on Oct 29, 2025
  5

• Bringing Back Stages for Infer and Predict

  enhancement
  modelscope/evalscope#842 · Filter by author sophies-cerebras opened on Sep 18, 2025
  1