Unfortunately many skeptical takes on #Mythos / #Glasswing remind me of Gell-Mann amnesia:

https://en.wiktionary.org/wiki/Gell-Mann_Amnesia_effect

Every #LLM company and claim deserves ruthless skepticism, but arguments based on falsehoods and misunderstandings don't lead us forward.

Watching this talk is recommended:

https://www.youtube.com/watch?v=1sd26pWhfmg

@buherator It's also worth noting that this guy works for Anthropic. There are several publications that replicated what Mythos did, with far less resources.

The Gap Between “Thousands of Vulnerabilities” and Reality:
https://medium.com/@manikndn369/the-gap-between-thousands-of-vulnerabilities-and-reality-ccf2ff5d7f8f

The Jagged Frontier:
https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jagged-frontier

@obivan There is a lot of nuance to consider with these results, incl. feasibility with other models (I'm willing to believe it's doable). But e.g. the FP problem doesn't seem at all that limiting as some suggest and some of the results (with Mythos&others) are very hard to ignore as a vulnerability researcher.

@buherator agreed, it's not all black&white. We will have to wait and see if they publish more details and a fair, transparent analysis.

If this is 100% true I would expect more CVE's and PoC's in the following months. And perhaps a different benchmark for exploitation, not those CTFy CyberGym types.

I do believe that their new model is powerful, but can't ignore the fact that they replicated OpenAI's marketing tactic from few years ago, also considering their IPO in October this year.