Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Implement RDMS backend (postgres/mysql/sqlite) (guacsec#910)
* Init Artifact and Ent backend Signed-off-by: Ivan Vanderbyl <[email protected]> * Configure backend Signed-off-by: Ivan Vanderbyl <[email protected]> * Change ID type to Int Signed-off-by: Ivan Vanderbyl <[email protected]> * Correct model transform for ID Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement BuilderNode Signed-off-by: Ivan Vanderbyl <[email protected]> * Fix transaction implementation to actually use tx Signed-off-by: Ivan Vanderbyl <[email protected]> * Add package (node) schema Signed-off-by: Ivan Vanderbyl <[email protected]> * Add PackageNamespace schema Signed-off-by: Ivan Vanderbyl <[email protected]> * Add PackageName schema Signed-off-by: Ivan Vanderbyl <[email protected]> * Add PackageVersions Signed-off-by: Ivan Vanderbyl <[email protected]> * Document how to generate ent nodes Signed-off-by: Ivan Vanderbyl <[email protected]> * Add test suite helper for working with sql tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Add IngestPackage and tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Generate Ent nodes for Package, NS, Names, and Versions Signed-off-by: Ivan Vanderbyl <[email protected]> * Add upsert support for software tree Signed-off-by: Ivan Vanderbyl <[email protected]> * Ensure only two versions were inserted Signed-off-by: Ivan Vanderbyl <[email protected]> * Order tree asc Signed-off-by: Ivan Vanderbyl <[email protected]> * Cleanup Signed-off-by: Ivan Vanderbyl <[email protected]> * Implementing unique versions (WIP) Signed-off-by: Ivan Vanderbyl <[email protected]> * Configure postgres in docker Signed-off-by: Ivan Vanderbyl <[email protected]> * Make ent backend configurable Signed-off-by: Ivan Vanderbyl <[email protected]> * Add Packages query Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement preloads for Packages Signed-off-by: Ivan Vanderbyl <[email protected]> * Ent isOccurrence progress WIP. Signed-off-by: Jeff Mendoza <[email protected]> * Improve build performance in Docker Signed-off-by: Ivan Vanderbyl <[email protected]> * Enable global IDs Signed-off-by: Ivan Vanderbyl <[email protected]> * Incorporate Jeff's work Signed-off-by: Ivan Vanderbyl <[email protected]> * Adds Index annotation, but we probably need something different Signed-off-by: Ivan Vanderbyl <[email protected]> * Move to helpers Signed-off-by: Ivan Vanderbyl <[email protected]> * Bridge networking so we can poke at the db from host Signed-off-by: Ivan Vanderbyl <[email protected]> * Migrate to backend package to separate generated files Signed-off-by: Ivan Vanderbyl <[email protected]> * Finish IsOccurrence ingest. Also fix Package ingest to only return the ingested package. Signed-off-by: Jeff Mendoza <[email protected]> * Added IsDependency ingest and query to ent backend. Signed-off-by: Jeff Mendoza <[email protected]> * Refactor backend into separate package, adds Source, and tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Use new ent backend package Signed-off-by: Ivan Vanderbyl <[email protected]> * Cache build step Signed-off-by: Ivan Vanderbyl <[email protected]> * Ignore some files for docker Signed-off-by: Ivan Vanderbyl <[email protected]> * Integrate Jeff's work Signed-off-by: Ivan Vanderbyl <[email protected]> * Add cleanup command to delete all generated ent code Signed-off-by: Ivan Vanderbyl <[email protected]> * Rename IsDependency to Dependency Signed-off-by: Ivan Vanderbyl <[email protected]> * Rename IsOccurrence to Occurrence Signed-off-by: Ivan Vanderbyl <[email protected]> * Rename ent.IsOccurrence Signed-off-by: Ivan Vanderbyl <[email protected]> * Improve cleanup logic Signed-off-by: Ivan Vanderbyl <[email protected]> * Experimenting Signed-off-by: Ivan Vanderbyl <[email protected]> * Add tests for IsDependency Signed-off-by: Ivan Vanderbyl <[email protected]> * Make tests easier to read Signed-off-by: Ivan Vanderbyl <[email protected]> * Cleanup tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Update readme on how to run tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Add note about tx Signed-off-by: Ivan Vanderbyl <[email protected]> * Try jeff's impl of ingest occurrence Signed-off-by: Ivan Vanderbyl <[email protected]> * Add func to ignore empty slices in cmp Signed-off-by: Ivan Vanderbyl <[email protected]> * Filter dep tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Attempting to get tests passing Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement custom predicates for json qualifiers Signed-off-by: Ivan Vanderbyl <[email protected]> * Change PackageVersion schema so that we can query qualifiers Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement version qualfier queries Signed-off-by: Ivan Vanderbyl <[email protected]> * Remove debug statement Signed-off-by: Ivan Vanderbyl <[email protected]> * Test no qualifiers Signed-off-by: Ivan Vanderbyl <[email protected]> * Small refactoring Signed-off-by: Ivan Vanderbyl <[email protected]> * Optimise pkgName query Signed-off-by: Ivan Vanderbyl <[email protected]> * Refactor Occurrences to use Subject edge Signed-off-by: Ivan Vanderbyl <[email protected]> * Simplify package version query Signed-off-by: Ivan Vanderbyl <[email protected]> * Cleanup Signed-off-by: Ivan Vanderbyl <[email protected]> * Add more tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Integrate more of Jeff's work Signed-off-by: Ivan Vanderbyl <[email protected]> * IngestOccurrence with OccurrenceSubject Signed-off-by: mrizzi <[email protected]> * Improve some package queries Signed-off-by: Ivan Vanderbyl <[email protected]> * Fixing more tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Remove OccurrenceSubject and get one test to pass Signed-off-by: Ivan Vanderbyl <[email protected]> * Make all Occurrence tests pass Signed-off-by: Ivan Vanderbyl <[email protected]> * Rename PackageNode to PackageType Signed-off-by: Ivan Vanderbyl <[email protected]> * Rename Source to SourceType Signed-off-by: Ivan Vanderbyl <[email protected]> * Improve package queries Signed-off-by: Ivan Vanderbyl <[email protected]> * Fix test Signed-off-by: Ivan Vanderbyl <[email protected]> * Cleanup package transforms Signed-off-by: Ivan Vanderbyl <[email protected]> * Fix our happy path test Signed-off-by: Ivan Vanderbyl <[email protected]> * Import more of Jeff's work Signed-off-by: Ivan Vanderbyl <[email protected]> * Expose Errorf as a global that we can replace later Signed-off-by: Ivan Vanderbyl <[email protected]> * Always query empty package version Signed-off-by: Ivan Vanderbyl <[email protected]> * Tighter constraints on versions Signed-off-by: Ivan Vanderbyl <[email protected]> * IngestHasSbom implementation Signed-off-by: mrizzi <[email protected]> * Add batch ingest for Artifacts Signed-off-by: Ivan Vanderbyl <[email protected]> * Add IngestPackages Signed-off-by: Ivan Vanderbyl <[email protected]> * Stub IngestOccurrences Signed-off-by: Ivan Vanderbyl <[email protected]> * Add HasSBOM tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement FindSoftware Signed-off-by: Ivan Vanderbyl <[email protected]> * Limit results in search set Signed-off-by: Ivan Vanderbyl <[email protected]> * Rename SBOM to BillOfMaterials and add SLSAAttestation Signed-off-by: Ivan Vanderbyl <[email protected]> * Rename BuilderNode to Builder now that Ent supports it Signed-off-by: Ivan Vanderbyl <[email protected]> * Rename some nodes Signed-off-by: Ivan Vanderbyl <[email protected]> * Use global IDs on tests so that we break any hard coded deps Signed-off-by: Ivan Vanderbyl <[email protected]> * Fix dependency tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement HasSBOM and fix all tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Mod tidy Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement GHSA Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement CVEs Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement OSV and refatcor advisory upserts Signed-off-by: Ivan Vanderbyl <[email protected]> * Change dependency type to enum in pg Signed-off-by: Ivan Vanderbyl <[email protected]> * Add note to readme Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement IsVulnerability Signed-off-by: Ivan Vanderbyl <[email protected]> * Slightly reduce number of fields selected Signed-off-by: Ivan Vanderbyl <[email protected]> * WIP Implementing CertifyVuln Signed-off-by: Ivan Vanderbyl <[email protected]> * Rename Vulnerability to CertifyVuln Signed-off-by: Ivan Vanderbyl <[email protected]> * Rename Signed-off-by: Ivan Vanderbyl <[email protected]> * Implements CertifyVuln and IngestVulnerability + Tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement HashEqual Ingest and Query Signed-off-by: Ivan Vanderbyl <[email protected]> * Improve Package tests and API Signed-off-by: Ivan Vanderbyl <[email protected]> * Stub SLSA Signed-off-by: Ivan Vanderbyl <[email protected]> * Implementing PkgEqual (WIP) Signed-off-by: Ivan Vanderbyl <[email protected]> * Add SLSA Ingest Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement HasSLSA and IngestSLSA Signed-off-by: Ivan Vanderbyl <[email protected]> * Make artifact query consistent Signed-off-by: Ivan Vanderbyl <[email protected]> * Fix SLSA ingest tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Add IngestMaterials Signed-off-by: Ivan Vanderbyl <[email protected]> * Add IngestDependencies Signed-off-by: Ivan Vanderbyl <[email protected]> * Add Sources query and sources tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement HasSourceAt Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement CertifyBad and IngestCertifyBad + tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement CertifyGood and IngestCertifyGood Signed-off-by: Ivan Vanderbyl <[email protected]> * Implementing Node interface Signed-off-by: Ivan Vanderbyl <[email protected]> * Fixing PkgEquals design and tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Small cleanup for consistency Signed-off-by: Ivan Vanderbyl <[email protected]> * Ensure arm compilation doesn't complain Signed-off-by: Ivan Vanderbyl <[email protected]> * Improve the consistency of package queries and transactions Signed-off-by: Ivan Vanderbyl <[email protected]> * Improve code reuse around package queries Signed-off-by: Ivan Vanderbyl <[email protected]> * Possibly break everything Signed-off-by: Ivan Vanderbyl <[email protected]> * Remove Annotations from Ent SBOM schema Signed-off-by: Ivan Vanderbyl <[email protected]> * Run generators Signed-off-by: Ivan Vanderbyl <[email protected]> * Fix my merge mistakes Signed-off-by: Ivan Vanderbyl <[email protected]> * Rename ents Signed-off-by: Ivan Vanderbyl <[email protected]> * Fix certify bad query Signed-off-by: Ivan Vanderbyl <[email protected]> * Fix dependency tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Add correct behaviour for match only empty Signed-off-by: Ivan Vanderbyl <[email protected]> * Add ent/contrib gql support Signed-off-by: Ivan Vanderbyl <[email protected]> * Fix pkgversion tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Generate ent code with GQL Support enabled Signed-off-by: Ivan Vanderbyl <[email protected]> * Schema version of pkgequal that has a & b branches instead of M2M, reverted Signed-off-by: Ivan Vanderbyl <[email protected]> * Add ptrWithDefault helper Signed-off-by: Ivan Vanderbyl <[email protected]> * More package tree tests Signed-off-by: Ivan Vanderbyl <[email protected]> * Add IngestSources Signed-off-by: Ivan Vanderbyl <[email protected]> * Update qualifiers match helper Signed-off-by: Ivan Vanderbyl <[email protected]> * More gql code gen for ent Signed-off-by: Ivan Vanderbyl <[email protected]> * All implemented features passing Signed-off-by: Ivan Vanderbyl <[email protected]> * Implement Scorecards Signed-off-by: Ivan Vanderbyl <[email protected]> * Disable 32bit builds since they have issues with int() Signed-off-by: Ivan Vanderbyl <[email protected]> * package: added pkgSpec entities filtering Signed-off-by: mrizzi <[email protected]> * vulnerability: added CertifyVulnSpec entities filtering Signed-off-by: mrizzi <[email protected]> * Cleanup Signed-off-by: Ivan Vanderbyl <[email protected]> * Use no-op for upserts Signed-off-by: Ivan Vanderbyl <[email protected]> * Update .gitignore Co-authored-by: Mihai Maruseac <[email protected]> * Add neptude backend to validation Signed-off-by: Ivan Vanderbyl <[email protected]> * Cleanup Signed-off-by: Ivan Vanderbyl <[email protected]> * Tidy Signed-off-by: Ivan Vanderbyl <[email protected]> * Rename SecurityAdvisory to Vulnerability Signed-off-by: Ivan Vanderbyl <[email protected]> * Update package spec Signed-off-by: Ivan Vanderbyl <[email protected]> * Mark ent as experimental Signed-off-by: Ivan Vanderbyl <[email protected]> * Update vuln nodes Signed-off-by: Ivan Vanderbyl <[email protected]> * Refactoring backend so that vulns work Signed-off-by: Ivan Vanderbyl <[email protected]> * Comment out vuln and dep code that needs to be updated Signed-off-by: Ivan Vanderbyl <[email protected]> * Enable go arm arch again Signed-off-by: Ivan Vanderbyl <[email protected]> * Use inmem backend by default Signed-off-by: Ivan Vanderbyl <[email protected]> * Add CertifyVEXStatement and IngestVEXStatement Signed-off-by: mrizzi <[email protected]> * CertifyVex snake_case fields Signed-off-by: mrizzi <[email protected]> * IngestVEXStatement: managed DB insert conflict with Ignore() Signed-off-by: mrizzi <[email protected]> * Comment out Vex Signed-off-by: Ivan Vanderbyl <[email protected]> * Enhance 'Node' and add 'Nodes' endpoints Signed-off-by: mrizzi <[email protected]> * Node: Added SourceType, Builder, SecurityAdvisory and refactored TestNode Signed-off-by: mrizzi <[email protected]> * Restricted some queries and proposed new tests approach Signed-off-by: mrizzi <[email protected]> * Refactored SecurityAdvisory to VulnerabilityType Signed-off-by: mrizzi <[email protected]> * Completed test refactoring Signed-off-by: mrizzi <[email protected]> * Fix Static Analysis Signed-off-by: mrizzi <[email protected]> * Fix Lint checks Signed-off-by: mrizzi <[email protected]> * Fix Static Analysis - copyright notice Signed-off-by: mrizzi <[email protected]> * Enable postgres GH service Signed-off-by: mrizzi <[email protected]> * Fix Unit tests: TestEntBackendSuite/TestCertifyBad/HappyPath Signed-off-by: mrizzi <[email protected]> * Scorecards: fix source namespace query Signed-off-by: mrizzi <[email protected]> * Commented TestVulnerability: endpoint commented Signed-off-by: mrizzi <[email protected]> * Test suite util Signed-off-by: mrizzi <[email protected]> * CI for integration tests: enable postgres GH service Signed-off-by: mrizzi <[email protected]> --------- Signed-off-by: Ivan Vanderbyl <[email protected]> Signed-off-by: Jeff Mendoza <[email protected]> Signed-off-by: mrizzi <[email protected]> Co-authored-by: Jeff Mendoza <[email protected]> Co-authored-by: mrizzi <[email protected]> Co-authored-by: Mihai Maruseac <[email protected]>
- Loading branch information
4 people
authored
Aug 29, 2023
1 parent
1bba6a4
commit 1c0a63f