Will an LLM be able to solve Raven's Progressive Matrices from an image in 2025? | Manifold

Will an LLM be able to solve Raven's Progressive Matrices from an image in 2025?

Basic

6

Ṁ117

Dec 31

58%

chance

1D

1W

1M

ALL

The prompt will be "Solve this. Explain your answer" with an RPM attached as an image. For example:

The AI must be able to solve 8 out of 10 puzzles of my choosing. I will only choose puzzles that I can solve.

If there's a consensus that chatbots can/can't do this I may not bother doing the test myself.

As of market creation the best commercially available LLMs fail embarrassingly:
Chatgpt 4o:
https://www.perplexity.ai/search/solve-this-explain-your-answer-ohIVE8CaQ3OIu9ODzaEXcg

Claude Sonnet 1.5:
https://www.perplexity.ai/search/solve-this-explain-your-answer-Ay.Kpyc9Tfm6uKBT3KUylQ

Rules:
- Must be an general purpose ai, can't be something made specifically to solve certain kinds of problems.
- I will not bet.

This question is managed and resolved by Manifold.

#Technical AI Timelines

Get

1,000

and

3.00

Sort by:

I'm a "it's just predicting the next word" guy and if LLMs start doing this I will change my mind.

@Shai I will note they have no problem "seeing" what's in the image. They can describe any shape when asked.

Related questions

Will LLM hallucinations be a fixed problem by the end of 2025?

Will an LLM consistently create 5x5 word squares by 2026?

Will the most interesting AI in 2027 be a LLM?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

Will LLMs such as GPT4 be considered a solution to Moravec’s paradox by 2030?

Will an LLM be able to solve the Self-Referential Aptitude Test before 2027?

Will the best public LLM at the end of 2025 solve more than 5 of the first 10 Project Euler problems published in 2026?

In 2025, will I be able to play Civ against an LLM?

Will there be any simple text-based task that most humans can solve, but top LLMs can't? By the end of 2026

By 2028 will we be able to identify distinct submodules/algorithms within LLMs?

Related questions

Will LLM hallucinations be a fixed problem by the end of 2025?

Will an LLM be able to solve the Self-Referential Aptitude Test before 2027?

Will an LLM consistently create 5x5 word squares by 2026?

Will the best public LLM at the end of 2025 solve more than 5 of the first 10 Project Euler problems published in 2026?

Will the most interesting AI in 2027 be a LLM?

In 2025, will I be able to play Civ against an LLM?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

Will there be any simple text-based task that most humans can solve, but top LLMs can't? By the end of 2026

Will LLMs such as GPT4 be considered a solution to Moravec’s paradox by 2030?

By 2028 will we be able to identify distinct submodules/algorithms within LLMs?

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules