Skip to content

Commit

Permalink
package/pkg-download: lookup hash files in global-patch-dir
Browse files Browse the repository at this point in the history
Currently, we expect and only use hash files that lie within the package
directory, alongside the .mk file. Those hash files are thus bundled
with Buildroot.

This implies that only what's known to Buildroot can ever get into those
hash files. For packages where the version is fixed (or a static
choice), then we can carry hashes for those known versions.

However, we do have a few packages for which the version is a free-form
entry, where the user can provide a custom location and/or version.  like
a custom VCS tree and revision, or a custom tarball URL. This means that
Buildroot has no way to be able to cary hashes for such custom versions.

This means that there is no integrity check that what was downloaded is
what was expected. For a sha1 in a git tree, this is a minor issue,
because the sha1 by itself is already a hash of the expected content.
But for custom tarballs URLs, or for a tag in a VCS, there is indeed no
integrity check.

Buildroot can't provide such hashes, but interested users may want to
provide those, and currently there is no (easy) way to do so.

We leverage the existing global-patch-dir mechanism to look for extra
hash files. We use the same heuristic that is used for bundled hash
files, and for each global patch directory <dir>, we use the first file
to exist among:
 1. look into <dir>/<package>/<version>/<package>.hash
 2. look into <dir>/<package>/<package>.hash

Reported-by: "Martin Zeiser (mzeiser)" <[email protected]>
Signed-off-by: Yann E. MORIN <[email protected]>
Signed-off-by: Peter Korsgaard <[email protected]>
  • Loading branch information
yann-morin-1998 authored and jacmet committed Nov 7, 2023
1 parent f91e89b commit 5d36710
Show file tree
Hide file tree
Showing 6 changed files with 46 additions and 12 deletions.
10 changes: 6 additions & 4 deletions Config.in
Original file line number Diff line number Diff line change
Expand Up @@ -674,12 +674,12 @@ config BR2_PACKAGE_OVERRIDE_FILE
documentation for more details on this feature.

config BR2_GLOBAL_PATCH_DIR
string "global patch directories"
string "global patch and hash directories"
help
You may specify a space separated list of one or more
directories containing global package patches. For a specific
version <packageversion> of a specific package <packagename>,
patches are applied as follows:
directories containing global package patches and/or hashes.
For a specific version <packageversion> of a specific package
<packagename>, patches are looked up as follows:

First, the default Buildroot patch set for the package is
applied from the package's directory in Buildroot.
Expand All @@ -693,6 +693,8 @@ config BR2_GLOBAL_PATCH_DIR
exists, then all *.patch files in the directory will be
applied.

The hash files are looked up similarly to the patches.

menu "Advanced"

config BR2_FORCE_HOST_BUILD
Expand Down
6 changes: 6 additions & 0 deletions docs/manual/adding-packages-directory.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -540,6 +540,12 @@ mercurial) because Buildroot currently does not generate reproducible
tarballs when source code is fetched from such version control
systems.

Additionally, for packages for which it is possible to specify a custom
version (e.g. a custom version string, a remote tarball URL, or a VCS
repository location and changeset), Buildroot can't carry hashes for
those. It is however possible to xref:customize-hashes[provide a list of
extra hashes] that can cover such cases.

Hashes should only be added in +.hash+ files for files that are
guaranteed to be stable. For example, patches auto-generated by Github
are not guaranteed to be stable, and therefore their hashes can change
Expand Down
24 changes: 23 additions & 1 deletion docs/manual/customize-patches.adoc
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
// -*- mode:doc -*- ;
// vim: set syntax=asciidoc:

=== Adding project-specific patches and hashes

[[customize-patches]]
=== Adding project-specific patches
==== Providing extra patches

It is sometimes useful to apply 'extra' patches to packages - on top of
those provided in Buildroot. This might be used to support custom
Expand Down Expand Up @@ -57,3 +59,23 @@ are available at a URL. *Note:* +BR2_LINUX_KERNEL_PATCH+ specifies kernel
patches that are applied after patches available in +BR2_GLOBAL_PATCH_DIR+,
as it is done from a post-patch hook of the Linux package.


[[customize-hashes]]
==== Providing extra hashes

Buildroot bundles a xref:adding-packages-hash[list of hashes] against
which it checks the integrity of the downloaded archives, or of those
it generates locally from VCS checkouts. However, it can only do so
for the known versions; for packages where it is possible to specify
a custom version (e.g. a custom version string, a remote tarball URL,
or a VCS repository location and changeset), Buildroot can't carry
hashes for those.

For users concerned with the integrity of such downloads, it is possible
to provide a list of hashes that Buildroot can use to check arbitrary
downloaded files. Those extra hashes are looked up similarly to the
extra patches (above); for each directory in +BR2_GLOBAL_PATCH_DIR+,
the first file to exist is used to check a package download:

* +<global-patch-dir>/<packagename>/<packageversion>/<packagename>.hash+
* +<global-patch-dir>/<packagename>/<packagename>.hash+
2 changes: 1 addition & 1 deletion package/pkg-download.mk
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ define DOWNLOAD
-d '$($(2)_DL_DIR)' \
-D '$(DL_DIR)' \
-f '$(notdir $(1))' \
-H '$($(2)_HASH_FILE)' \
$(foreach f,$($(2)_HASH_FILES),-H '$(f)') \
-n '$($(2)_BASENAME_RAW)' \
-N '$($(2)_RAWNAME)' \
-o '$($(2)_DL_DIR)/$(notdir $(1))' \
Expand Down
14 changes: 9 additions & 5 deletions package/pkg-generic.mk
Original file line number Diff line number Diff line change
Expand Up @@ -509,11 +509,15 @@ else
endif
$(2)_VERSION := $$(call sanitize,$$($(2)_DL_VERSION))

$(2)_HASH_FILE = \
$(2)_HASH_FILES = \
$$(strip \
$$(if $$(wildcard $$($(2)_PKGDIR)/$$($(2)_VERSION)/$$($(2)_RAWNAME).hash),\
$$($(2)_PKGDIR)/$$($(2)_VERSION)/$$($(2)_RAWNAME).hash,\
$$($(2)_PKGDIR)/$$($(2)_RAWNAME).hash))
$$(foreach d, $$($(2)_PKGDIR) $$(addsuffix /$$($(2)_RAWNAME), $$(call qstrip,$$(BR2_GLOBAL_PATCH_DIR))),\
$$(if $$(wildcard $$(d)/$$($(2)_VERSION)/$$($(2)_RAWNAME).hash),\
$$(d)/$$($(2)_VERSION)/$$($(2)_RAWNAME).hash,\
$$(d)/$$($(2)_RAWNAME).hash\
)\
)\
)

ifdef $(3)_OVERRIDE_SRCDIR
$(2)_OVERRIDE_SRCDIR ?= $$($(3)_OVERRIDE_SRCDIR)
Expand Down Expand Up @@ -1138,7 +1142,7 @@ ifneq ($$(call qstrip,$$($(2)_SOURCE)),)
ifeq ($$(call qstrip,$$($(2)_LICENSE_FILES)),)
$(Q)$$(call legal-warning-pkg,$$($(2)_BASENAME_RAW),cannot save license ($(2)_LICENSE_FILES not defined))
else
$(Q)$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$(call UPPERCASE,$(4)),$$($(2)_RAWNAME),$$($(2)_BASENAME_RAW),$$(F),$$($(2)_DIR)/$$(F),$$($(2)_HASH_FILE))$$(sep))
$(Q)$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$(call UPPERCASE,$(4)),$$($(2)_RAWNAME),$$($(2)_BASENAME_RAW),$$(F),$$($(2)_DIR)/$$(F),$$($(2)_HASH_FILES))$$(sep))
endif # license files

ifeq ($$($(2)_REDISTRIBUTE),YES)
Expand Down
2 changes: 1 addition & 1 deletion package/pkg-utils.mk
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,7 @@ define legal-manifest # {HOST|TARGET}, pkg, version, license, license-files, sou
echo '"$(2)","$(3)","$(4)","$(5)","$(6)","$(7)","$(8)"' >>$(LEGAL_MANIFEST_CSV_$(1))
endef

define legal-license-file # {HOST|TARGET}, pkgname, pkgname-pkgver, filename, file-fullpath, pkg-hashfile
define legal-license-file # {HOST|TARGET}, pkgname, pkgname-pkgver, filename, file-fullpath, pkg-hashfiles
mkdir -p $(LICENSE_FILES_DIR_$(1))/$(3)/$(dir $(4)) && \
{ \
support/download/check-hash $(5) $(4) $(6); \
Expand Down

0 comments on commit 5d36710

Please sign in to comment.