I'm currently working on a project that involves some lesser-known hardware, mostly in embedded systems. It's really intriguing to see how well AI models can perform, but I'm also often shocked by how poorly they handle even simple tasks. They tend to shine with coding that has a lot of human-written examples, but I've been curious about their reasoning capabilities. I created a logical reasoning question that should be solvable by someone familiar with basic Arithmetic and Geometric Progressions in just a few minutes. The question goes as follows: fill in the values in place of '?':
CE- B, 11, 28, 69
EJ- S, 105, 495, 2405
BG- P, 39, 78, 149
IF- N, ?, ?, ?
I've tested this with several AI models, including Gemini 2.5 Pro and Claude 3.7, and none were able to solve it. They often claim to handle advanced mathematics, but in my experience, they struggle with even this level of reasoning, which seems like basic high school math. I'm beginning to wonder how accurate their claims really are. What is it going to take for these models to truly get better?
4 Answers
It sounds like a tough problem! Have you tried getting opinions from actual people on it? Just because it's not a textbook problem doesn't mean it's unsolvable. I think your question might require familiarity with a specific format that AIs aren’t used to, which is why they’re struggling with it. As someone who tackled it, I'd say while it feels increasingly like a logical reasoning question, it's also a bit convoluted for someone unfamiliar with this type of problem.
I was curious too and managed to crack it! The answer is: IF- N, 132, 1146, 10224. The logic lies in interpreting the letters as numbers and understanding the sequence based on that, which might not be clear for the AI models.
Gotta say, I laughed at that part where you said to not let AI search the web while testing—they might just find the solution! Still, it's funny that they can't handle some basic reasoning tasks, definitely puts their abilities into perspective.
For sure! It’s amazing how these models can sometimes seem more like fancy search engines instead of problem solvers.
You know, I think the confusion stems from the way the question is formatted, which might not lead the AI to the right logic path. It feels more like a sequence guess than pure reasoning. I managed to solve it as well, but I could see how it would trip up someone handling this for the first time.
That makes sense! Some of these reasoning questions can seem straightforward to us if we’ve practiced them, but can be a bit of a curveball for AIs.