Test-Time Training & the ARC Challenge

Hello guys,

So my title was volontarily a bit clck-baity but not so much. Here is the paper :

The Surprising Effectiveness of Test-Time Training for Abstract Reasoning

I stumbled on this video from Matthew Berman, who is I think one of the higher end content creator on Youtube for AI stuff :

Q-Star 2.0 – AI Breakthrough Unlocks New Scaling Law (New Strawberry) (his title is very much click-baity I admit)

So in this paper, they say that ensembling their method (test-time training) with recent program generation approaches, they get SoTA public validation accuracy of 61.9%, matching the average human score.

What do you think? Is it a real breakthrough? A scam? Somewhere in between?