Issues: open-compass/opencompass
Issues list
[Bug] livecodebench evaluator may need to check the memory footprint
#2055
opened Apr 27, 2025 by
astly123456
2 tasks done
[Bug] ImportError: cannot import name 'handle_file' from 'gradio_client' (/opt/conda/lib/python3.10/site-packages/gradio_client/__init__.py)
#2044
opened Apr 25, 2025 by
zgydbc
2 tasks done
[Bug] I get aime2024 score 10 points on qwen 2.5 32b instruct
#2042
opened Apr 24, 2025 by
lebronjamesking
2 tasks done
[Bug] The test results using lmdeploy are not reproducible, while using vllm is reproducible but yields low scores.
#2038
opened Apr 23, 2025 by
SefaZeng
2 tasks done
[Bug] setup.py中python版本>=3.8，但代码使用3.10+才有的的 union types as X | Y，导致3.8，3.9版本运行报错
#2032
opened Apr 22, 2025 by
vra
2 tasks done
[Bug] model-kwargs参数和generation-kwargs参数无法通过CLI命令行指定
#2027
opened Apr 18, 2025 by
XIAOHUIL1
2 tasks done
[Bug] The AIME2024 data provided by OpenCompass is inconsistent with Hugging Face
#2026
opened Apr 18, 2025 by
c-box
[Bug] FileNotFoundError: [Errno 2] No such file or directory: 'opencompass/ChemBench/dev/Name_Conversion_benchmark.json'
#2014
opened Apr 10, 2025 by
baoyihe
2 tasks done
[Bug] on agieval_gen_617738, pyarrow.lib.ArrowInvalid: Could not convert '$5$;$10$' with type str: tried to convert to int64
#2004
opened Apr 5, 2025 by
Youth-49
2 tasks done
[Bug] Ceval_gen not compare for modelscope
#2003
opened Apr 4, 2025 by
zhangtianhong-1998
2 tasks done
[Bug] OpenAISDK类的定义没有加@MODELS.register_module()的修饰符
#1994
opened Apr 1, 2025 by
StyleAIPro
2 tasks done
[Bug] The ifeval evaluation on 541 cases returns unexpected low score.
#1983
opened Mar 29, 2025 by
lebronjamesking
2 tasks done
[Feature] How about add bfcl evaluation to the repo
#1981
opened Mar 28, 2025 by
lebronjamesking
1 task
[Bug] [Bug] LiveStemBench数据集评测时，推理结果正确。但是启动打分程序时报错 mmengine KeyError: '
cfg or default_args must contain the key "type", but got {}\nNone'
#1980
opened Mar 28, 2025 by
BigFishLi
2 tasks done
[Bug] extract_non_reasoning_content error when predict is null
#1964
opened Mar 20, 2025 by
simplew2011
2 tasks done
opencompass/tasks/outer_eval/alpacaeval.py中 example.json 内容是什么格式？框架中没有看到样例
#1948
opened Mar 16, 2025 by
liuchunming033
ProTip!
What’s not been updated in a month: updated:<2025-03-27.