HuggingFace incident report:

https://huggingface.co/blog/agent-intrusion-technical-timeline

The report itself reeks of LLM slop with gems like the "kill chain", consisting of phases like recon, exfil, c2...and k8s :) Nuances are overemphasized (like how code execution was used to execute code) while important steps are blurry (e.g. they had some kind of "allowlist" in the dataset processor, that allowed everything which didn't look like a URL?).

I feel sorry for blue teams not because they'll have to respond to more incidents but because they'll have to wade through reports like this...

@buherator Sam is all over it!

@buherator why do they not sue OpenAI?
It is a cyberattack….

@buherator these sad folks eat, drink, breath, see, hear, smell, poop and piss AGI.

@buherator what really bothers me is that – apart from a very few – no one pointed out how atrocious OpenAI's sandboxing was. Based on the reports it seems they used JFrog on the very same box to limit the model's access and roughly that's about it. On the top of it they haven't noticed their model was hacking another company for two days.
That sounds more like OpenAI's absolute stupidity than their model's incredible capability.
"Ah you know, our dog escaped containment not because we left the door wide open but because it's super intelligent."

This diagram is also bad, but at least it shows the high-level flows.

Notice that "Third party code sandbox" is marked as "Compromised", while the post text explicitly states that "Modal’s infrastructure was not compromised in any way". Also, how do you even compromise a *sandbox*? WTF is Lateralize?!

This other diagram is supposed show the time distribution of events. What are considered as "events"? What is the vertical scale of this diagram? What is 461? Why do stripes contain different colors sometimes?

@ozu

Yes: https://infosec.place/notice/B8dh429Gf6p490VuJU

And HF also didn't notice someone is messing with their k8s in the noisiest way possible for days.

@buherator probably they have an AI SOC.

@buherator It may well be LLM-generated, but also, I regret to report, that's just how some security teams talk. They do seem to attract a bunch of folks who would have been happier LARPing a battle somewhere.

@buherator Pew pew...

@buherator the entire story seems contrived to convince us how "powerful" LLM are.

@buherator It appears to be shooting for a Star Trek LCARS vibe but with more muted tones and no Helvetica ultra compressed (I had to look that up)

@buherator It was only a month ago that the newest AI was so dangerous, it was a military secret. And then released to the public a couple of weeks later. Where it proved to be a marginal improvement on the last model. We've had several years of badly disguised press releases about how the AI is sentient or passed the genAI threshold.

The sky is falling in... again. This is just another PR stunt to pump stonks.

@buherator Yet another thing that lines up with the "fake incident for marketing purposes, both parties were in on it" conspiracy hypothesis. I'm tired, boss...

@landelare It doesn't look fake at all, but the whole story draws a really bad picture of both model reliability and company competence. OAI+HF noticing the behavior and deciding not to intervene (or even support the actions) would be the most generous interpretation for the companies IMO.

@buherator Oh it's already happening, I wish whoever found the Linux kernel vulns in May and decided to slop out the reports to step on 100 lego bricks 🙃