summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-06-09minor code cleanup in the build systemLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09re-add ability to use cbfs grub.cfg as defaultLeah Rowe
i removed this before, when making grub multi-tree, because the design i used in an earlier version of the patch actually added the grub.elf generation to grub source itself, but then i decided to hack around the grub build system from lbmk/cbmk instead re-add this functionality, so that users can easily insert their own custom grub.cfg into cbfs without needing to re-build their image. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09trees: exit with error if project undefinedLeah Rowe
i was originally looser about this, because i also wanted the trees script to generically run "make" from any directory, but this behaviour was error-prone and it is no longer used in the build system. disable it, in the interest of stability. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09build: also make a lock file during release buildLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09lib.sh: more useful lock messageLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09create a lock file during buildsLeah Rowe
prevent duplicate main instances of the build system from running the lock file is deleted when the parent process exits, alongside the tmpdir deletion the build system must only ever be run ot one instance at a time, per work directory Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09git.sh: hide e() output on for loopLeah Rowe
this for loop is a hack to make sure that all the sources get nuked (using nuke.list files). hide the messages so that they do not appear when running just any command in the trees script. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09lib.sh: fix regressionLeah Rowe
i was being a bit too clever about some optimisations revert this change. otherwise, nothing will download or build Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09git.sh: download xtree *before*, not afterLeah Rowe
downloading it after means that if an error occurs when downloading the xtree project, the main project will still be there and nothing will mandate the downloading of the xtree project. whereas, if we grab the xtree project first, then the main project won't get saved to src/ this makes the build system a bit more resilient under fault conditions, but otherwise doesn't change behaviour. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09git.sh: fix deletion path in nuke()Leah Rowe
i accidentally forgot to include src/ in the prefix Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09lib.sh: less confusing error in download()Leah Rowe
don't say "file missing", because it may be present! instead, say that the download failed. this covers both contexts: internet failed and thus no file present, or the file is present but checksum verification failed. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09lib.sh: hide stderr on download()Leah Rowe
on the initial check, the output is confusing because it will say "checksum verification failed" if the file doesn't already exist, but then goes to download. only say checksum failed if a download occured, and the check failed, otherwise report nothing except that the file already exists. this will not reduce the ability to debug issues later on, and it will reduce the amount of confusion for users. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09lib.sh: simplify download()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09lib.sh: fix redundancy in download()Leah Rowe
it was only downloading the main url, even when it should use the backup. fix it by actually using the for loop variable. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09lib.sh: simplify singletree()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09git.sh: further simplify nuke()Leah Rowe
it's a very compact nuke Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09git.sh: simplify link_crossgcc()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09git.sh: simplify nuke()Leah Rowe
do not over-engineer such a trivial thing. seriously. all we're doing is nuking some files. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09Merge pull request 'Add dependency scripts for Fedora 40 and Ubuntu 24.04' ↵Leah Rowe
(#220) from fuel-pcbox/lbmk:master into master Reviewed-on: https://codeberg.org/libreboot/lbmk/pulls/220
2024-06-09add crossgcc tarballs to config/submodules/Leah Rowe
support redundant downloads, and enable inclusion of these tarballs inside release archives, for offline builds. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-08git.sh: support downloading *files* as submodulesLeah Rowe
when we download coreboot, we currently don't have a way to download crossgcc tarballs, so we rely on coreboot to do it, which means running the coreboot build system to do it; which means we don't get them in release archives, unless we add very hacky logic (which did exist and was removed). the problem with coreboot's build system is that it does not define backup links for each given tarball, instead relying on gnu.org exclusively, which seems OK at first because the gnu.org links actually return an HTTP 302 response leading to a random mirror, HOWEVER: the gnu.org 302 redirect often fails, and the download fails, causing an error. a mitigation for this has been to patch the coreboot build system to download directly from a single mirror that is reliable (in our case mirrorservice.org). while this mitigation mostly works, it's not redundant; the kent mirror is occasionally down too, and again we still have the problem of not being able to cleanly provide crossgcc tarballs inside release archives. do it in config/submodules, like so: module.list shall say the relative path of a given file, once downloaded, relative to the given source tree. module.cfg shall be re-used, in the same way as for git submodules, but: subfile="url" subfile_bkup="backup url" do this, instead of: subrepo="url" subrepo_bkup="backup url" example entries in module.list: util/crossgcc/tarballs/binutils-2.41.tar.xz util/crossgcc/tarballs/gcc-13.2.0.tar.xz util/crossgcc/tarballs/gmp-6.3.0.tar.xz util/crossgcc/tarballs/mpc-1.3.1.tar.gz util/crossgcc/tarballs/mpfr-4.2.1.tar.xz util/crossgcc/tarballs/nasm-2.16.01.tar.bz2 util/crossgcc/tarballs/R06_28_23.tar.gz the "subrev" variable (in module.cfg) has been renamed to "subhash", so that this makes sense, and that name is common to both subfile/subrepo. the download logic from the vendor scripts has been re-used for this purpose, and it verifies files using sha512sum. therefore: when specifying subrepo(git submodule), subhash will still be a sha1 checksum, but: when specifying subfile(file, e.g. tarball), subhash will be a sha512 checksum the logic for both (subrepo and subfile) is unified, and has this rule: subrepo* and subfile* must never *both* be declared. the actual configuration of coreboot crossgcc tarballs will be done in a follow-up commit. this commit simply modifies the code to accomodate this. over time, this feature could be used for many other files within source trees, and could perhaps be expanded to allow extracting source tarballs in leiu of git repositories, but the latter is not yet required and thus not implemented. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-08git.sh: remove unnecessary line breakLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-08vendor.sh: add a return at the end of mkdirsLeah Rowe
i don't like that it's not there, because of the quirks in sh behaviour. put it there to put my mind at ease. otherwise, this doesn't change any behaviour. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-08vendor.sh: move download logic to lib.shLeah Rowe
in future revisions, i will make tarballs become subfiles, to complement submodules. e.g. crossgcc tarballs in coreboot Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-08lib.sh: shorten a string in e()Leah Rowe
line exceeds 80 characters Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07move uefiextract to elf/uefitool/Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07Add dependency scripts for Fedora 40 and Ubuntu 24.04fuel-pcbox
2024-06-07git.sh: fix submodule pathLeah Rowe
i accidentally cloned to tmpdir rather than tmpgit oops! Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07git.sh: simplify prep_submodules()Leah Rowe
copying the module list into tmpdir/ no longer makes sense, because it was only done before when we supported either running the list from "git submodule update", or module.list. since we only support handling of module.list, we can greatly simplify this function. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07git.sh: unified handling of git clone/reset/amLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: simplified copy_elf() handlingLeah Rowe
don't create elfdir, create dest_dir, which is elfdir plus the location within it only create dest_dir within copy_elf, which is only called if actually compiling the code this avoids creating empty elf directories, and it generally cleans up all handling, unifying the handling of directories into a single function, namely copy_elf() which already exists Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07git.sh: simplify submodule handlingLeah Rowe
there were stragglers remaining, from when we used to actually run "git submodule update", but this was removed. clean up the submodule functions and merge them together. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07git.sh: provide feedback for repository downloadsLeah Rowe
otherwise, it's not clear to the operator what's happening i'm normally against such verbose feedback, because it's bloat, but this minimal amount of feedback will make the build system more pleasant to use, especially during testing. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07git.sh: download "depend" projects *before*Leah Rowe
don't do it after, because that means the main project is saved under src/ before we know whether the subrepo was downloaded. the "depend" variable (in config/git/) is no longer used for projects that go in subdirectories of a parent; now, we use config/submodules/ for this type of dependency. download the "depend" projects (as per config/git/) first. this way, if they fail, the main one will fail, but if they succeed and main fails, you can just run the main download again and it won't fail. this fixes a bug where, depending on how you download a set of projects and depending on the order which you do so, a given project can become un-downloadable on current design, because git will complain that a directory already exists. this fix is done not only in code (by this commit), but by prior configuration changes. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07git.sh: reduced indentation in fetch_submoduleLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07git.sh: reduced indentation in prep_submodulesLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07git.sh: *never* run git submodule updateLeah Rowe
only use config/submodules/ which the build system then uses to run git clones manually, replicating the submodules feature. we must never use a project's own gitmodules feature, because we can't easily control it. better to let it break first, and then figure out what modules to add manually, so that we have only what we need for each project. it's done this way, because git's own submodules feature doesn't have very good error checking in general, nor does it have good redundancy. with the current design, we can declare backup repositories for each submodule. we replicate it precisely. for example: 3rdparty/vboot this is a coreboot submodule, and we handle that in the coreboot trees. however, our current design also allows you to do this even if the upstream repository does not contain a .gitmodules file Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07lib.sh: rename variable for clarityLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: don't check empty path in copy_elf()Leah Rowe
it could lead to some whackiness later on Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: fix build issue caused by bad elf checkLeah Rowe
we're not checking for bad elfs, but the check itself was bad due to a quirk in how sh works. really, really obscure bug. fixed now! if the given directory didn't actually exist, or there were no files in it, it'd be searching for the file named "*" which is obviously wrong Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: fix listfile check in copy_elf()Leah Rowe
don't check that the variable is empty check that the file itself exists or not this should fix the recent build issues Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: don't say check elf/ if build.list missingLeah Rowe
(on single tree projects. this complements the last patch) Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: don't do elfcheck if build.list missingLeah Rowe
otherwise, some checks are done too soon, and nothing gets built. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07define mdfiles/images in config/submodules/docs/Leah Rowe
again: the "depend" variable must never be used for subprojects that point to a subdirectory of the main project, because there's no clean way of handling this in case of error conditions. make it a submodule under config/submodules/. this is for the documentation, including static site generator documentation, and image files (photos). as of this revision, there are now only those "depend" projects defined in config/git/, where the destination directory of the subject is not a subdirectory of the main project, so: in a subsequest revision, i will mitigate an existing bug whereby failure of the dependency project leaves the main one still intact, breaking builds; this revision enables that to be done. from now on, subproject-to-subdirectory-of-main-project will be avoided in config/git/; config/submodules/ will be used. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07libopencm3 to config/submodules/ on stm32-vserprogLeah Rowe
same as the previous patch, we must no longer use "define" variables in config/git/ when the path is a subdirectory of a given project, because it means that the download can only happen after the main one, and currently if that fails, the download of the main repo would remain intact, breaking future builds in ways that we can't control - to be clear, it could be controlled, but with added code complexity in the build system, so: put it in config/submodules/ Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07add tinyusb to config/submodule/ for pico-sdkLeah Rowe
don't define it as a "depend" variable in config/git/, because it means putting the files in a subdirectory of an existing project was was already then downloaded, and that means it can't be downloaded first; if the download of it fails, the old download is left intact. this bug isn't currently fixed in the build system, at all, so this and other patches are being made to mitigate it. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07config/git: use "depend" for serprog dependenciesLeah Rowe
this brings the handling of serprog projects in sync with canoeboot, which relies on the "depend" variable to get the needed submodules, because cbmk does not download submodules for these projects lbmk does download submodules. i want it in sync with cbmk for this, to make merging easier between both projects, because i'm going to make a change on both projects, whereby config/submodules/ is used exclusively Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06trees: unified coreboot makeargsLeah Rowe
in particular, the coreboot build system may auto-download submodules when building cbfstool; vboot for instance. we do not want such unpredictable behaviour, so now we use UPDATED_SUBMODULES=1 when building coreboot utilities. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06trees: use multiple threads to build cbutilsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06move handle_coreboot_utils to script/treesLeah Rowe
it doesn't really make sense placed in lib.sh, because it's only called from script/trees Signed-off-by: Leah Rowe <leah@libreboot.org>