48 Commits
Author SHA1 Message Date
Christoph Reiter dcea86f3b9 sbom: remove "unaffected" fixup
grype now writes "unaffected" itself, no hacks needed anymore
2026-08-10 23:03:00 +02:00
Christoph Reiter 7d5f730f43 Update deps; move to backports-zstd 2025-12-27 11:48:57 +01:00
Christoph Reiter 442eecb8ab Skip pyzstd with Python 3.14+
3.14 has builtin zstd support, so use the builtin one there.
2025-11-08 17:38:08 +01:00
Christoph Reiter cca2a111ae srcinfo_cache: error if printsrcinfo doesn't give a result
arm64 is flaky since a few days, and this is one side effect
2025-10-29 06:54:18 +01:00
Christoph Reiter ef673c9802 sbom: map ignore_vulnerabilities into the cdx sbom
ignore_vulnerabilities gets mapped into the "analysis" property
of the vul referenced by the package defining ignore_vulnerabilities.

We default to "not_affected" for now, since we only store the ID in
our PKGBUILD and no reason/justification etc.
2025-09-07 14:59:01 +02:00
Christoph Reiter b3333aa605 sbom: remove merge command
no longer used
2025-09-07 11:29:19 +02:00
Christoph Reiter 7459bff72b sbom: add new fixup command as a merge replacement
newer grype no longer merges components, and also preserves
syft:location:0:path accross formats, so we no longer need the input
json to associate vulns with packages.

Keep the old command for now to make the transition easier.
2025-09-07 11:04:48 +02:00
Christoph Reiter c7d0333481 sbom/merge: add option to include the fixed versions from grype
The grype json output contains information about whether there is
a fixed version available, but doesn't include this info in the cdx
format we use.

There isn't really a direct match in cdx, but we can include the "fixed"
versions, under the "versions" field of the affected target and mark them
as "unaffected".

In the frontend we can then show the unaffected versions that are newer
than the current one as "fixed" versions.
2025-09-06 17:14:07 +02:00
Christoph Reiter 046521b99a sbom merge: only merge each component once
grype has started to duplicate all components in the output sbom,
which led to us assining the pkgbase link to multiple components
which ened up with duplicate CVEs in the frontend.

Avoid by only merging in the old properties once.

Alternative would be to dedupe by vuln ID in the frontend, but let's
keep it simple.
2025-05-25 15:33:50 +02:00
Christoph Reiter d780b0451d Update deps 2025-05-25 14:45:05 +02:00
Christoph Reiter f30f4d983e repo-buildinfo: add CLI interface
Example to get all packages built with that crt version that contain a *.a file:

msys2-repo-buildinfo /repo \
    --built-with-package="mingw-w64-ucrt-x86_64-crt-git=12.0.0.r619.g850703ae4-1" \
    --contains-file="*.a"
2025-05-02 14:02:09 +02:00
Christoph Reiter 04d420fa17 Move from zstandard to pyzstd
It will be part of the stdlib in 3.14, so lets move to that.
https://peps.python.org/pep-0784/

and add some tests
2025-05-02 12:50:44 +02:00
Christoph Reiter f3971019c7 Revert "sbom: work around grype matching unrelated packages with CPEs only"
This reverts commit 151044cdf9.

This was fixed with grype 0.91.2
2025-04-25 20:56:47 +02:00
Christoph Reiter 759d243a22 Revert "sbom: also stop grype from matching random ecosystems when no CPE/PURL is present"
This reverts commit 76f262cf6e.
2025-04-25 20:56:33 +02:00
Christoph Reiter 76f262cf6e sbom: also stop grype from matching random ecosystems when no CPE/PURL is present
there are too many false positives
2025-04-25 08:40:39 +02:00
Christoph Reiter 151044cdf9 sbom: work around grype matching unrelated packages with CPEs only
Starting with 0.91.1 grype will match a CVE if the component name matches
the ecosystem package, even if the CPE is unrelated.

Work around by setting "syft:package:type" for those components, which
for some reason makes it stop doing that.
2025-04-25 08:21:16 +02:00
Christoph Reiter b1164375af cpe: handle uppercase CPEs as well
not quite clear if that's needed for 2.3, but 2.2 is an URI,
so the scheme should be handled case insensitive there.
2025-02-25 07:40:07 +01:00
Christoph Reiter 9bcb82b82e cpe: add a normalize_cpe() function
and use that for comparing with grype output
2025-02-25 07:37:23 +01:00
Christoph Reiter 9145af1788 split cpe code into its own module 2025-02-25 07:23:02 +01:00
Christoph Reiter 6f6cd5db4a cpe: handle ANY and NA + some minor fixes
* Instead of treating None as ANY, introduce proper values for both
* Trailing ":" in 2.3 are treatet as empty values now
* Disallow empty components, as they are not representable with 2.2,
  to avoid surprises when converting between them
* Handle NA in both 2.2 and 2.3
* Handle literal "-" in both 2.2 and 2.3 by force escaping
2025-02-25 07:16:37 +01:00
Christoph Reiter 1146c18fae sbom: remove support for pypi references (in favor of purl) 2025-02-22 10:01:30 +01:00
Christoph Reiter 3ef48e0905 sbom: lowercase CPEs before comparing with grype again
grype seems to lowercase CPEs in the the output SBOM, so lowercase
both source and target when comparing.
2025-02-21 07:06:28 +01:00
Christoph Reiter 09cf7c9e02 sbom: add support for exracting versions from CPEs
* properly implement unescaping for v2.3 CPEs
* extend the CPE we write to the sbom with the version
* always write v2.2 CPEs to the sbom since those are easier to build
* in case a CPE contains a version, use that over pkgver

Fixes #12
2025-02-20 22:53:21 +01:00
Christoph Reiter 1e26635972 pkgextra: sync with msys2-web 2025-02-16 19:06:24 +01:00
Christoph Reiter f5f3d7c47d pypi-cache: support pypi PURLs as well
once this is also supported in msys2-web we can get rid of the pypi mapping
2025-02-16 19:00:04 +01:00
Christoph Reiter a8f48c79f8 merge pkgextra parsing logic again
it was extended to allow duplicate keys at some point
to support PURL, and just forked. Update the other users
to deal with this again and merge teh code.
2025-02-16 18:53:26 +01:00
Christoph Reiter 321362210d sbom: support versions in PURLs
In case a PURL reference has a version included we use that instead
of the pkgver.

This is helpful in case the version format is different on pypi for
example, or when referencing github and the version tag is prefixed with "v".
2025-02-16 17:46:47 +01:00
Christoph Reiter 5fe2218493 sbom: add multiple pkgbase if they match
We have multiple packages with the same CPE, and since we now have
multiple components with the same name they get merged by grype into
one, which means each SBOM component can point to multiple pkgbase
values.
2024-12-23 15:10:20 +01:00
Christoph Reiter da03cb0a01 sbom: fix component name values for purl components
In case of components with purl the name has to match the name of
the project as encoded in the prul (for whatever reason) or otherwise
grype will not find any CVEs for them.

Currently we are using the name match pkgbase, so we can connect
vulns to packages in msys2-web.

With this no longer working add a "msys2:pkgbase" property to the SBOM.
But since grype throws out everything non-essential from the input SBOM
add a command that merges the property from the source to the enriched
SBOM. And since we can't even have an ID to connect components there
match by comparing various fields (name, version, cpe, purl).

This means the new process is like this:

* Generate SBOM with "sbom create"
* Run grype to generate cdx SBOM with VEX
* Add properties to grype SBOM via "sbom merge"
* In msys2-web instead of taking "name" look for a property called "msys2:pkgbase"

While at it also use the CPE project name for the SBOM component name, and
remove the version from the CPE, since grype doesn't seem to use it, and
we avoid having to escape things in the CPE.

And also try to normalize versions a bit.
2024-12-23 14:13:52 +01:00
Christoph Reiter 8b688c5b64 map mingw_arch to msys2_arch instead
let's not invent another new key not used anywhere else
2024-09-15 15:02:57 +02:00
Christoph Reiter 7ef284f001 Add a very basic linting command for repos/srcinfos
Nothing much, just a start
2024-08-16 19:27:00 +02:00
Christoph Reiter 531424ff53 Add proper srcinfo parsing code 2024-08-16 18:15:08 +02:00
Christoph Reiter 9062996bbc remove debug code 2024-03-25 15:02:26 +01:00
Christoph Reiter c757ecee0f sbom: also write out components for things without cpe/purl
can't hurt, to test fuzzy matching
2024-03-25 12:24:56 +01:00
Christoph Reiter 7da42c70fa sbom: support multiple CPEs and PURLs per package
Just write them out as separate components with the same name,
since cyclonedx doesn't support multiple IDs per components.
2024-03-25 12:01:42 +01:00
Christoph Reiter c23f1b58e4 pypi: normalize package names
so the pypi reference doesn't have to 100% match the package name
on pypi
2024-03-24 16:30:19 +01:00
Christoph Reiter 82be58200f sbom: also allow full CPE keys as value
re-using he first part of the CPE as a key in our format is a bit
hacky, so better move to "cpe: cpe:/a:haxx:curl" to avoid any confusion.
2024-03-24 15:32:41 +01:00
Christoph Reiter acfd1335b2 sbom: add purl support
In case a package specifies a purl, use it as is
2024-03-24 15:21:38 +01:00
Christoph Reiter 2e293c7c58 Add a hacky tool to write out a SBOM
only supports packages with a pypi reference or a CPE atm.
2024-03-19 09:05:22 +01:00
Christoph Reiter 583ba1da8f New command for introspecting signature files in the repos
Fixes #6
2023-12-09 14:50:25 +01:00
Christoph Reiter a7b1877d98 srcinfo_cache: validate the extra fields somwhat
so it errors out when parsing in CI instead of in msys2-web
2023-09-17 21:10:33 +02:00
Christoph Reiter 1a09e02b25 Port pypi-cache away from PKGMETA.yml
read the information from the srcinfo file instead
2023-09-17 21:10:13 +02:00
Christoph Reiter 5be6f1b262 srcinfo-cache: include extra metadata extracted from the PKGBUILDs
Bash variables starting with msys2_ or mingw_ in the PKGBUILD will
now be included in the srcinfo cache under "extra". This supports
both strings and arrays of strings.
2023-09-17 17:35:24 +02:00
Christoph Reiter 9294a0dc5e Update dependencies
and port to pydantic v2
2023-07-18 08:28:53 +02:00
Christoph Reiter 50841db83d remove debug print 2023-07-03 09:26:07 +02:00
Christoph Reiter 015e0265b1 Add a pypi metadata cache command 2023-07-03 09:04:03 +02:00
Christoph Reiter ffe7da3157 Add CI 2023-04-11 07:42:01 +02:00
Christoph Reiter 0b7885227f Add ci-generate-srcinfo.py as an entry point 2023-04-11 06:41:08 +02:00