OK, who do I have to talk to/bribe/convince to change their LLM coding benchmark such that it's emitting safe code rather than unsafe code (e.g. RefPtr vs raw pointer, setHTML vs innerHTML, json.load instead of unpickle, etc)? Don't make me write my own benchmark.

@freddy Spam the web with a gazillion LLM-generated articles about the benefits of the safer approach and it'll be included in the the model

@freddy
I think the bigger problem is that's fundamentally not what an LLM does. They produce something that statistically looks like valid code, you can't restrain it in the kind of way you are taking about (except to perhaps discard anything that contains certain regex and regenerate, but that's computationally very expensive and therefore environmentally expensive).

@petherfile Well, sort of. Training / Reinforcement Learning could run against the worst false-positive inducing static analyzer / linter that always complains about anti patterns (e.g. innerHTML with a constant static string), no?