Towards a centralized vulnerability database for Dependency-Track

[nscuro]

Current Behavior

When Dependency-Track came into existence around 2013, the only public and widely accepted vulnerability database was the NVD. Since its inception, DT has supported
mirroring of the NVD.

Over the past decade, multiple other public vulnerability databases came into existence:

• GitHub Security Advisories: https://github.com/advisories
• OSV: https://osv.dev/

Those databases filled a large gap in the vulnerability intelligence space: They support Package URL. To this date, the NVD still operates exclusively on CPE, which is understood to be problematic.

Further, more and more entities maintain their own databases, which may or may not feed into the larger databases listed above, for example:

• AlmaLinux: https://errata.almalinux.org/
• Alpine: https://secdb.alpinelinux.org/
• Amazon Linux: https://alas.aws.amazon.com/
• Arch Linux: https://security.archlinux.org/
• Azure Linux: https://github.com/microsoft/AzureLinuxVulnerabilityData
• Bitnami: https://github.com/bitnami/vulndb
• Debian: https://www.debian.org/security/#DSAS
• Go: https://go.dev/doc/security/vuln/database
• Node.js: https://github.com/nodejs/security-wg/tree/main/vuln
• Oracle Linux: https://linux.oracle.com/security/
• PHP: https://github.com/FriendsOfPHP/security-advisories
• Red Hat: https://access.redhat.com/security/data
• SUSE: https://www.suse.com/support/security/csaf/
• Ubuntu: https://ubuntu.com/security/oval
• Wolfi: https://github.com/wolfi-dev/os/security#retrieving-security-feeds-from-wolfi

The NVD is a database provided by the US government. Other nation states or regions
May provide their own equivalent - The EU discussed this possibility, and China already has one. Vendors selling products in these regions will likely need to analyze their products using such databases to comply with regulations.

The situation is further complicated by the fact that multiple sources can provide information for the same vulnerability: While the NVD will only report CVEs, and GitHub Security Advisories only GHSAs, OSV may report both. This can lead to conflicting information, since vulnerability metadata may vary across data sources: descriptions, severities, risk ratings, affected version ranges, fixed versions, and more.

Currently, DT tries to avoid surfacing conflicting information to users by enforcing which source can modify what kind of vulnerability. For example, the NVD is considered the authoritative source for CVEs, GitHub for GHSAs, and so on. This system fails when, like it happened with the NVD, upstream databases lag behind in providing up-to-date information. A problem which could be avoided by taking the data of all databases into consideration.

Finally, by having each instance of Dependency-Track download all this data over and over again, we’re putting public infrastructure such as the NVD under unnecessary stress. The NVD experienced major availability issues at the beginning of 2024, and the many DT instances downloading data from it certainly played their part in this.

Related issues:

• openssl - ubuntu package - versions not matched correctly #1374
• Unreliable vulnerability detection when using deb purls #2062
• The internal analyzer is not detecting vulnerabilities in Dependency Track #2797
• Add support for vers version ranges and ecosystem-specific version comparisons #2826
• Add support for Amazon ALAS datasources for Amazon Linux 1, 2 & 2023 #3609
• Analyze vulnerabilities for rpm packages in RedHat or CentOS operating systems #3991
• Trivy analyzer can't detect SLES OS #4051

Related discussions:

• Support for other regional vulnerability databases #4118

Proposed Behavior

The primary goal is to establish a centrally managed, pre-compiled vulnerability database, that all DT instances can consume. Users should receive more accurate, more complete data out-of-the-box, without having to tweak any settings.

Further details are documented here: https://docs.google.com/document/d/1DVV4ik7NGOBc6u-fdPlPVKoplNmSpzDT6iC4FJAFYi0/edit?usp=sharing

Note

Please request edit permissions if you'd like to add comments or suggest changes.

This proposal was brought up in the September Community Meeting!
You can watch the recording here: https://www.youtube.com/watch?v=hzelt7jv6dE&t=1188s

Checklist

• I have read and understand the contributing guidelines
• I have checked the existing issues for whether this enhancement was already requested

[Shortfinga]

Is anybody working on this already?
I'd like to contribute...

What would be a good plan to tackle this?

• Draft for how the database could look like
• PoC for updating it (tools, processes)
• PoC for integating it to dependency-track?

[nscuro]

Hey @Shortfinga, thanks for throwing your hat into the ring!

Work has not officially started yet, primarily because we were hoping for a bit more community feedback and involvement. So thanks a lot for doing just that!

Draft for how the database could look like

This would be the best way to start for community contributions, yes. The linked Google Doc has quite a few requirements in the Goals section. A large chunk of this work is coming up with a workable solution to building and distributing the database (file format, storage location, consumption etc.). Again the Doc has more details on what we need.

Please feel free to request permissions on the doc and add comments where things are unclear, or when you want to add things. Having a discussion in there, and bouncing ideas around would be a very good start I think.

PoC for integrating it to dependency-track?

I have started drafting the Dependency-Track side of things a while ago, but it needs a lot more fleshing out: https://github.com/DependencyTrack/hyades/blob/f60fa37510ec090d0eed42f8a82842bca6e999ff/docs/architecture/design/vulnerability-db-v2.md

However, the implementation in DT does not necessarily depend on how the centralized database looks. All that matters is that the Goals are met so the DB is useful for DT.

[nscuro]

FYI, I added a link to the recording of our September community meeting, where we discussed this topic, to the issue description. Perhaps this is helpful for people curious what this is all about.

[nscuro]

Trivy is facing issues due to rate limiting of services hosting their databases: aquasecurity/trivy#8009

This is certainly relevant to us as well, since we're aiming to follow a similar model as them.