Will a open source pure Mamaba LLM surpass 82 MMLU on MMLU (5-shot) before end of year 2024?
Basic
3
Ṁ75Jan 1
25%
chance
1D
1W
1M
ALL
Will any company release a pure Mamaba LLM that scores over 82 points on the standard full MMLU (5-shot) using https://github.com/EleutherAI/lm-evaluation-harness with open source weights?
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Related questions
Related questions
Will openAI have the most accurate LLM across most benchmarks by EOY 2024?
37% chance
🐕 OpenLLMs: Will Any Open Source LLM on the HuggingFace OpenLLM Leaderboard Significantly Gain in Avg Score by YE 2024?
49% chance
Will a Mamba-based LLM of GPT 3.5 quality or greater be open sourced in 2024?
79% chance
Will there be an open source LLM as good as GPT4 by the end of 2024?
68% chance
What will be the maximum context length of any OpenAI LLM by EOY 2024?
Will any LLM outrank GPT-4 by 150 Elo in LMSYS chatbot arena before 2025?
20% chance
Will an open-source LLM beat or match GPT-4 by the end of 2024?
85% chance
Will any open source LLM with <20 billion parameters outperform GPT-4 on most language benchmarks by the end of 2024?
13% chance
What will be true of OpenAI's best LLM by EOY 2025?
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance