Conversation

@davidgerard

Prompt injection works because it’s one stream of data mixed with instructions. If you separate it with tags, the user can fake the tags.

If only someone had told people that in-band signalling is a bad idea before LLMs came along.

3
2
0

@david_chisnall @davidgerard you can't expect them to have noticed yet, it's only been going on for decades with phone phreaking.

1
0
0

@dysfun @davidgerard

I suspect that's the root problem. The problems were largely discovered in the '70s, and tech bros don't believe they need to know about anything that happened more than ten years ago.

0
0
1

@david_chisnall @davidgerard @buherator in the last 50+ years, in-band signaling has been the root cause and main enabler of blue boxing, stack-based and heap-based buffer overflows (well any NUL-terminated string based vulnerability really), format string bugs, (SQL) injection… I’m sure I’m missing some vulnerability classes.

1
0
1
@raptor @david_chisnall @davidgerard I think an often overlooked but important example is path traversal
1
1
2