Fixed issues in CVEfixes CVE collection pipeline

[saadullah01]

Hi,

First of all I want to thank you for your developing this amazing tool, I recently used it to collect latest CVEs data for my project.

While using it I ran into the following issues and created a fix for them:

1. Before ordering df_cve in cve_importer.py - preprocess_jsons(df_in) the code does not check if all columns exist, which leads to keyerrors
2. In case you are reusing CWE XML file the code prepends DATA_PATH again to the cwe_doc in extract_cwe_record.py - extract_cwe(), which creates a wrong path leading to a path error

One more thing I wanted to mention is that this issue mentions two hacks to deal with CWE-1026 issue, "hack" to get things running is to manually update your https://github.com/advisories/GHSA-9895-g6x5-xwcp to use CWE-1027, the first subtype in the view, or to add 1026 as a member of itself in cwec_v4.10.xml under View ID="1026".. For these two hacks the second one, about adding 1026 as a member to itself, does not work and gave the same error, but the first one solved the issue.

[leonmoonen]

Dear Saad, Thanks so much for your contribution to CVEfixes 🙏 My colleague Linas will be looking into this, and we will incorporate your fix when all looks good on our side as well. cheers, Leon

…

On Fri, Apr 12, 2024 at 10:00 PM Saad Ullah ***@***.***> wrote: Hi, First of all I want to thank you for your developing this amazing tool, I recently used it to collect latest CVEs data for my project. While using it I ran into the following issues and created a fix for them: 1. Before ordering df_cve in cve_importer.py - preprocess_jsons(df_in) the code does not check if all columns exist, which leads to keyerrors 2. In case you are reusing CWE XML file the code prepends DATA_PATH again to the cwe_doc in extract_cwe_record.py - extract_cwe(), which creates a wrong path leading to a path error One more thing I wanted to mention is that this issue <#11 (comment)> mentions two hacks to deal with CWE-1026 issue, "hack" to get things running is to manually update your https://github.com/advisories/GHSA-9895-g6x5-xwcp to use CWE-1027, the first subtype in the view, or to add 1026 as a member of itself in cwec_v4.10.xml under View ID="1026".. For these two hacks the second one, about adding 1026 as a member to itself, does not work and gave the same error, but the first one solved the issue. ------------------------------ You can view, comment on, or merge this pull request online at: #19 Commit Summary - 3b6533e <3b6533e> FIXED ISSUE - cve_importer.py preprocess_jsons(df_in) - before ordering df_cve the code does not check if all columns exist, which leads to keyerrors - df99131 <df99131> FIXED ISSUE - extract_cwe_record.py extract_cwe() - while reusingthe CWE XML file the code prepends DATA_PATH again to the cwe_doc File Changes (2 files <https://github.com/secureIT-project/CVEfixes/pull/19/files>) - *M* Code/cve_importer.py <https://github.com/secureIT-project/CVEfixes/pull/19/files#diff-e7e63733e762ca77372b7edad288f330d5c612dc088e2c922e3a8b89893bcf30> (5) - *M* Code/extract_cwe_record.py <https://github.com/secureIT-project/CVEfixes/pull/19/files#diff-48d6d4de70dc8ff960dfa61d48d685e12daae7944d0ca793be5810dd96c04692> (2) Patch Links: - https://github.com/secureIT-project/CVEfixes/pull/19.patch - https://github.com/secureIT-project/CVEfixes/pull/19.diff — Reply to this email directly, view it on GitHub <#19>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASMRV3X37YRQL2V2VULQ33Y5A4MLAVCNFSM6AAAAABGEWR7WGVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI2DAOBRGE3DSNQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- Leon Moonen, Prof., PhD. Head of Department, Data-Driven Software Engineering, Simula Research Laboratory, Oslo, Norway & Department of Data Science and Analytics, BI Norwegian Business School, Oslo, Norway ***@***.*** / https://leonmoonen.com