CVSS is dead to us

https://daniel.haxx.se/blog/2025/01/23/cvss-is-dead-to-us/

#curl

@bagder At the risk of sounding stupid - could you associate your severity Levels with a fixed CVSS score and publish that? In my mind that would stop-gap "helpful" completion attempts.

@cpy I write about exactly that in the post as well

@bagder
> Every CVE filed to MITRE is supposed to have a CVSS score set.

That's odd. I requested CVEs through them (https://cveform.mitre.org/), there is no field to provide a CVSS score.
Not sure if that makes it better or worse. Someone else without a clue of the system rates it, or the researcher initially rates it a higher to make it "more important"? Both terrible choices.

@bagder Ah. I had actually read it before responding. Somehow I didn't process "Can we avoid this?". My bad :).

@bagder To me it seems like CVSS is trying to do a dozen things at the same time, and is being potentially provided by multiple groups with conflicting ideas on what it should be based on what thing they're trying to make it do, and their own personal goals (explicit or implicit). There's no way that could ever work.

Heck, even simple risk assessment is two-dimensional - how likely is the risk, and how big is the impact when it happens. This is trying to flatten that, plus many more properties, into a single axis, and of course it's useless.

It would be much better if the CVSS was just dropped wholesale, and replaced with scores that have a relevant target. At the very least we can have the two axes that normal risk assessment puts on it, which would solve most of the CVSS sillyness already. Add a third axis / score for user involvement, too, since a bug that can be triggered without users acting is much worse than one a user has to choose to run.

For example, your 9.8 "integer overflow could be abused" from a year ago would be marked as "impact minimal", "user invoked", "likelyhood certain". Maybe even give a somewhat formatted field to indicate which platforms are affected, so things like vulnerability scanners can check if their platform is even listed in the first place. Critical Windows vulnerability that does not exist on Macs should not give rise to a forced update.

@bagder One approach you could take, which doesn't involve fictional CVSS scores but does avoid the risk of CISA (or others) causing you this kind of problem, would be to have a policy of using the mid-point of the severity range for each of your 4 severity bandings as the score.

That gives people a good idea of what the severity is, and avoids the problem you had here.

@raesene with a CVSS we must provide the calculator string showing how we came to that score. That would be the part where we would have to "lie" to get the score we want.

@bagder A rating is required for prioritization. Until you convince MITRE/CISA to abandon "CVSS rating", it would be helpful to put a sane value in the "CVSS box". All of us who get the flow of CVE alerts would thank you if you put your rating (set the vectors randomly as long as it gets your ratings!).

@bagder Ahh I see :( Back when I was a pentester, luckily we didn't need to provide the calculations, so we regularly back-ported a CVSS score, if required, to the severity we thought the issue had...

@bagder thank you so much for writing this up, and illustrating the problems with CVSS’s fake precision. Your post will be a helpful example next time I try convincing someone about the utility of CVSS scoring.

@bagder very interesting read! I personally found CVSS as a good weapon to use against managers to convince them about severity of fixing stuff, as they like metrics. I wonder what could I use instead. High/critical doesn't have the same gravity as IMO 😀
Other than that, for me, it doesn't really have much difference without it, and I know better now not to focus on the score

@bagder The sad thing is: it could be a good system if used properly. It captures a lot of useful properties in the vector, especially when you go beyond the base score and include temporal and environmental score.

In my fever dreams, I'm imagining a system that captures my overall system architecture including security requirements, security boundaries, etc., so when a CVE is coming in, it automatically calculates an environmental score, based e.g. on "that's a local vuln, hard ro exploit, on an appliance that doesn't really have any internal security boundaries anyways, on a network physically protected and behind seven firewalls, get lost" or "unauthenticated remote code execution in all configurations on a service handling valuable data, actually exposed to the Internet, actively exploited, all hands on deck NOW".

So from a vulnerability information consumer point of view, data points like exploitability, authenticated vs. unauthenticated, local or remote, actually confirmed by a person with an understanding of the code,make a huge difference, and it would be good to have them in machine readable format. Of course, a 9.8 slapped on by some analyst who doesn't even bother to look at the code doesn't help anyone.

@bagder your next article should be titled "Make CVSS Great Again"

@jimfuller is that even possible?

on hacker news: https://news.ycombinator.com/item?id=42802736

@bagder
Should we participate in CVE production at all?

@levitte then we just leave it to someone else to do it and I think that is even worse...

@bagder Really good article. My experience with "security experts" is that most actually have very limited knowledge in the field. And lack critical thinking. This leads to an almost blind trust in these tools that spit out reports on CVSS scores that can easily be exported to nice looking spreadsheets.

Unfortunately, those tend to be taken as gospel by management. Because management never have a clue about anything.

@bagder
Uhmmmm... so, make yourself a CVE authority on your stuff [effectively blocking anyone else from publishing anything on your stuff), and do nothing else (i.e. no CVE published).

Of course, it's a bit late for #curl, but you know, other projects?

@levitte there's an appeal process to prevent that kind of action from CNAs

@faker @bagder strictly speaking, CVE does not require CVSS. CVSS and CWE have been more heavily encouraged since early 2024, and CISA Vulnrichment will add them if they are missing. Also Vulnrichment is responsive to issues and PRs, we do not get every CVSS or CWE right the first time.

@bagder how is it fake in case curl maps their 4 levels to some 4 values in cvss and at least sets it.
This allows arguing for a better system while defending users from misinformation?

@lig because we then need to provide a string showing how we filled in the answers to get the score, which would be the real lie

Then suddenly my previous PR to CISA for fixing the wrong curl CVE (mentioned in the blog post) gets response!

also now written about in places like: https://socket.dev/blog/curl-project-and-go-security-teams-reject-cvss-as-broken

@bagder they‘ll make a shortlist of projects to score more carefully? as the usual treatment might cause unwanted attention?

@icing does not instill confidence in the system, nope

@zmanion more heavily encouraged by whom?
I reported CVEs but only few, I'm not part of a CNA or any organization that deals a lot with CVEs.
I've never heard of this.

The MITRE submission form does not include any mention of CVSS.
Their FAQ mentions CVSS once, but not in the CVE request section (https://www.cve.org/ResourcesSupport/FAQs#pc_cve_list_basicscve_list_serverity_ratings).
Does MITRE agree with this?

@faker Here's one example: https://www.cve.org/Media/News/item/blog/2024/09/10/CNA-Enrichment-Recognition-List

@faker Perhaps the FAQ needs to be updatred. cveform.mitre.org, including the path to request CVD IDs, is up for review/revision. So it may make sense to have CVSS and CWE options in a new form.