Set criteria:
At least 1
Draft criteria:
Can't have been led in any way by a human once trained, except for being run
Can have been trained on some set of problems but shouldn't be trained using semi solved problems which then count as the breakthrough
Inspired by the below
Link: https://twitter.com/RichardMCNgo/status/1640568776495353860?s=20
1,000
3.00think I just made a bad bet, this is a super ambiguous market and i have no idea how significant the breakthrough has to be. Even worse, Ngo's tweet has 'novel theorems' as a subclass of 'scientific breakthroughs', but 'scientific brekathrough or novel theorem' in the title doesn't share that relationship, and novel theorems are much easier than breakthroughs.
@osmarks Yeah. I guess the major question here is mostly "what's the threshold for novel?".
For example, has it already happened with sort algorithms? This result is novel and comes from a neural network:
https://www.nature.com/articles/s41586-023-06004-9
@EvanDaniel I guess I don't find this breakthrough that impressive. If some non AI process had done this I wouldn't be surprised.
@NathanpmYoung It looks that way in some ways, but also it's been a fairly long time since a human made an improvement there despite attention and effort. And the sorting network used to do it is novel.
I think it makes sense not to count it; it isn't terribly dramatic. But if we're not counting it, I think that makes the threshold fairly murky and prone to goalpost moving. "AI is all the things computers can't do yet" and all that.
(I'm avoiding this market until I have a clearer sense of the resolution criteria.)
@NathanpmYoung yeah I think this should have some concrete criteria that excludes the sort improvement, (though I don't really think should be excluded) otherwise this just seems like goalpost moving. The improvement to sort would've already been made if it was possible via non ML methods, and is already impactful as it's being widely used in computing.
@brubsby It feels like the sort of improvement that's entirely possible with non-ML methods. Having done a little bit of assembly code golfing (not well) ages ago, the resulting code was entirely clear and reasonable. The sorting network analysis made sense. I highly doubt I'm the only one thinking "I could have thought of that if I'd stared at it" who also... didn't in fact think of it. But it's not weird or alien, just a little harder to spot than it looks. (It does not look easy to spot, even in retrospect, it just doesn't look impossible either.)
And it's since been reproduced by non-ML automated code generation methods that were apparently just a little bit of tuning away from being able to find it.
@EvanDaniel All that's well and good, but the novel breakthrough was made with neural nets instead of not neural nets, which is the subject of this market.