summaryrefslogtreecommitdiff
path: root/script/trees
AgeCommit message (Collapse)Author
2024-06-25lib.sh: simplify use of environment variablesLeah Rowe
don't have a separate variable for them. just export them directly and use them directly. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24trees: try xgcc build twice if first attempt failsLeah Rowe
sometimes buildgcc just fails for like no reason. we had this the other day and another fix was made to the trees script, to mitigate; the user ran it again and buildgcc worked just fine. run it twice, and then call err only if the second one fails. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-23trees: don't check if xgcc is already builtLeah Rowe
just run the make crossgcc command anyway. coreboot's own build system checks itself, and much more reliably, but the check is more thorough and a bit slower. in rare cases, lbmk may come into build issues with xgcc, and if you run the build again, it will always fail every time because the checks is based on whether the xgcc directory exists, rather than checking each individual crossgcc binary. checking every binary is also possible, but as i said, the coreboot build system already does that, so let's defer to coreboot's own handling of it. remove the directory check. this will slow down the build process a little bit, but should improve reliability under fault conditions. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22use backticks on eval commands, not subshellsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22lib.sh: remove badcmd()Leah Rowe
it's bloat. telling the user to rtfm is something that we already do on irc; they will still ask how to do everything, and ignore the message from badcmd(), or they will automatically know to rtfm. i'm on a massive purge, removing bloat from lbmk as part of Libreboot Build System Audit 6. all bloat must go. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22lib.sh: more unified config handlingLeah Rowe
replace it with logic that simply uses "." to load files directly. for this, "vcfg" is added as a variable in coreboot target.cfg files, referring to a directory in config/vendor/ containing a file named pkg.cfg, and this file then contains the same variables as the erstwhile config/vendor/sources config/git files are now directories, also containing pkg.cfg files each with the same variables as before, such as repository link and commit hash this change results in a noticeable reduction in code complexity within the build system. unified reading of config files: new function setcfg() added to lib.sh setcfg checks if a config exists. if a 2nd argument is passed, it is used as a return value for eval, otherwise a string calling err is passed. setcfg output is passed through eval, to set strings based on config; eval must be used, so that the variables are set within the same scope, otherwise they'd be set within setcfg which could lead to some whacky results. there's still a bit more more to do, but this single change results in a substantial reduction in code complexity. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-21trees: more robust check to avoid "make fetch"Leah Rowe
do not use shorthand here. the test was failing to produce the desired result under some circumstances, for example when i did "./update release" i got this: make: Entering directory '/home/lbdev/lbmk/release/20240612-62-ga6b1a6bd/libreboot-20240612-62-ga6b1a6bd_src/src/stm32-vserprog' make: *** No rule to make target 'fetch'. Stop. make: Leaving directory '/home/lbdev/lbmk/release/20240612-62-ga6b1a6bd/libreboot-20240612-62-ga6b1a6bd_src/src/stm32-vserprog' ERROR script/trees: !mk src/stm32-vserprog fetch ERROR ./update: excmd: script/trees -f ERROR script/roms: Unhandled non-zero exit: ./update ERROR ./build: excmd: script/roms serprog ERROR ./update: build_release release/20240612-62-ga6b1a6bd: stm32 ERROR ./update: can't build rom images in the above circumstance, run_make_command was executed, which is not the desired behaviour; rather, fetch_project_trees or fetch_project_repo should be called, and then the script should immediately exit. it should also exit, without downloading anything, if a changelog file exists as in release archives. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-20minor code cleanup in the build systemLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-16unify coreboot elfdir (DO_NOT_FLASH)Leah Rowe
use a common string when setting this path Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15trees: simplify copy_elf()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15trees: remove unnecessary check in copy_elfLeah Rowe
we don't need to check whether this variable is set, because checking an empty path will also cause the same return in the next line. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15grub: insert background in memdisk insteadLeah Rowe
the background is only a few kb. the whole rationale before was to limit the space used in memdisk, but this decision was made when the background was much bigger; it has since been optimised greatly, and the grub modules were heavily reduce, so it should be safe. grub's memdisk breaks when you add too much data to it. as part of simplifying the rest of lbmk, this change removes some more bloat from the rest of lbmk. handling this in the memdisk is much simpler than handling it with cbfstool. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-14unified checks for variable initialisationLeah Rowe
new function chkvars() does the job Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-14lib.sh: remove the items() functionLeah Rowe
it's pretty much just doing the same thing as ls -1 remove it! Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09re-add ability to use cbfs grub.cfg as defaultLeah Rowe
i removed this before, when making grub multi-tree, because the design i used in an earlier version of the patch actually added the grub.elf generation to grub source itself, but then i decided to hack around the grub build system from lbmk/cbmk instead re-add this functionality, so that users can easily insert their own custom grub.cfg into cbfs without needing to re-build their image. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09trees: exit with error if project undefinedLeah Rowe
i was originally looser about this, because i also wanted the trees script to generically run "make" from any directory, but this behaviour was error-prone and it is no longer used in the build system. disable it, in the interest of stability. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09add crossgcc tarballs to config/submodules/Leah Rowe
support redundant downloads, and enable inclusion of these tarballs inside release archives, for offline builds. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: simplified copy_elf() handlingLeah Rowe
don't create elfdir, create dest_dir, which is elfdir plus the location within it only create dest_dir within copy_elf, which is only called if actually compiling the code this avoids creating empty elf directories, and it generally cleans up all handling, unifying the handling of directories into a single function, namely copy_elf() which already exists Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07git.sh: download "depend" projects *before*Leah Rowe
don't do it after, because that means the main project is saved under src/ before we know whether the subrepo was downloaded. the "depend" variable (in config/git/) is no longer used for projects that go in subdirectories of a parent; now, we use config/submodules/ for this type of dependency. download the "depend" projects (as per config/git/) first. this way, if they fail, the main one will fail, but if they succeed and main fails, you can just run the main download again and it won't fail. this fixes a bug where, depending on how you download a set of projects and depending on the order which you do so, a given project can become un-downloadable on current design, because git will complain that a directory already exists. this fix is done not only in code (by this commit), but by prior configuration changes. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: don't check empty path in copy_elf()Leah Rowe
it could lead to some whackiness later on Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: fix build issue caused by bad elf checkLeah Rowe
we're not checking for bad elfs, but the check itself was bad due to a quirk in how sh works. really, really obscure bug. fixed now! if the given directory didn't actually exist, or there were no files in it, it'd be searching for the file named "*" which is obviously wrong Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: fix listfile check in copy_elf()Leah Rowe
don't check that the variable is empty check that the file itself exists or not this should fix the recent build issues Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: don't say check elf/ if build.list missingLeah Rowe
(on single tree projects. this complements the last patch) Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: don't do elfcheck if build.list missingLeah Rowe
otherwise, some checks are done too soon, and nothing gets built. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06trees: unified coreboot makeargsLeah Rowe
in particular, the coreboot build system may auto-download submodules when building cbfstool; vboot for instance. we do not want such unpredictable behaviour, so now we use UPDATED_SUBMODULES=1 when building coreboot utilities. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06trees: use multiple threads to build cbutilsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06move handle_coreboot_utils to script/treesLeah Rowe
it doesn't really make sense placed in lib.sh, because it's only called from script/trees Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06trees: skip single-tree build if a build existsLeah Rowe
this replicates the same behaviour as multi-tree builds, checking for files inside the relevant elf/ directory Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06trees: also print "DONE! check elf/dir" on singleLeah Rowe
e.g. ./update trees -b flashprog tell the user to check elf/flashprog Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06trees: handle build-test on multi-tree projectsLeah Rowe
for example, now flashprog binaries could be placed in the elf/ directory, under elf/flashprog/ Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06move cfgsdir/datadir variables to lib.shLeah Rowe
it's also used from script/roms, in addition to trees move these variables to a common file used everywhere Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06handle build.list from config/data/, not config/Leah Rowe
certain code checks for build.list, to skip it, for example in items() we already use config/data/grub to store grub config data that applied to all trees create these directories too: config/data/coreboot config/data/u-boot config/data/seabios move the respective build.list files in here, and also to config/data/grub now multi-tree projects contain, per directory, just the target.cfg file and the patches directory. this is much cleaner, because some of the logic can be simplified more Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06don't use build.list to detect multi-tree projectsLeah Rowe
instead, check for the presence of target.cfg files not in config/project/ but config/project/tree/ the way this check is done, it merely returns 1 if config/project/*/target.cfg is detected, and returns 0 in all other cases, even if config/project/target.cfg exists that way, if the maintainer accidentally adds a target.cfg in the main directory, the given multi-tree project will not break Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-03better help text on invalid commandsLeah Rowe
adding help again is a bad idea. code should never document itself; that's what documentation is for. so, make the code do a better job telling the user where to find documentation. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-02make GRUB multi-tree and re-add xhci patchesLeah Rowe
Re-add xHCI only on haswell and broadwell machines, where they are needed. Otherwise, keep the same GRUB code. The xHCI patches were removed because they caused issues on Sandybridge-based Dell Latitude laptops. See: https://codeberg.org/libreboot/lbmk/issues/216 The issue was not reported elsewhere, including on the Haswell/Broadwell hardware where they are needed, but the build system could only build one version of GRUB. The older machines do not need xHCI patches, because they either do not have xHCI patches, or work (in GRUB) because they're in EHCI mode when running the payload. So, the problem is that we need the xHCI patches for GRUB on Haswell/Broadwell hardware, but the patches break Sandybridge hardware, and we only had the one build of GRUB. To mitigate this problem, the build system now supports building multiple revisions of GRUB, with different patches, and each given coreboot target can say which GRUB tree to use by setting this in target.cfg: grubtree="xhci" In the above example, the "xhci" tree would be used. Some generic GRUB config has been moved to config/data/grub/ and config/grub/ now looks like config/coreboot/ - also, the grub.cfg file (named "payload" in each tree) is copied to the GRUB source tree as ".config", then added to GRUB's memdisk in the same way, as grub.cfg. Several other design changes had to be made because of this: * grub.cfg in memdisk no longer automatically jumps to one in CBFS, but now shows a menuentry for it if available * Certain commands in script/trees are disabled for GRUB, such as *config make commands. * gnulib is now defined in config/submodule/grub/, instead of config/git/grub - and this mitigates an existing bug where downloading gnulib first would make grub no longer possible to download in lbmk. The coreboot option CONFIG_FINALIZE_USB_ROUTE_XHCI has been re-enabled on: Dell OptiPlex 9020 MT, Dell OptiPlex 9020 SFF, Lenovo ThinkPad T440p and Lenovo ThinkPad W541 - now USB should work again in GRUB. The GRUB payload has been re-enabled on HP EliteBook 820 G2. This change will enable per-board GRUB optimisation in the future. For example, we hardcode what partitions and LVMs GRUB scans because * is slow on ICH7-based machines, due to GRUB's design. On other machines, * is reasonably fast, for automatically enumerating the list of devices for boot. Use of * (and other wildcards) could enable our GRUB payload to automatically boot more distros, with minimal fuss. This can be done at a later date, in subsequent revisions. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-28trees: use CPUS=x on regular coreboot makeLeah Rowe
i already do this on crossgcc, but overlooked it on regular builds where i just use -j, but coreboot's build system makes use of the CPUS= option in make use XBMK_THREADS for this Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-26rename include/option.sh to include/lib.shLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-26general code cleanup in the build systemLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25trees: reset makeargs per target/projectLeah Rowe
it wasn't being reset before. when coreboot is being built, i add to makeargs every time. if multiple targets are being built, the make command would end up looking something like: make -C src/coreboot/default UPDATED_SUBMODULES=1 \ UPDATED_SUBMODULES=1 (the parameter would be printed twice) of course, this doesn't check whether that parameter is added already in target.cfg for a given target, but that's ok because i won't add that one in target.cfg i baked it into the code, only when handling coreboot, because that was easier than either putting it in makeargs for every coreboot target.cfg, or again modifying the code to handle that; the current solution is the cleanest. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25trees: also use UPDATED_SUBMODULES=1 on crossgccLeah Rowe
i overlooked this in the previous revision Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25trees: add UPDATED_SUBMODULES to coreboot makeLeah Rowe
we do not want submodules to be downloaded after the fact. we only handle this on ./update trees -f coreboot Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-25trees: write -C on the make command first not lastLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-23update/trees: remove unused variableLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-19minor code cleanup in the build systemLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15remove check_project() (always set variables)Leah Rowe
in lbmk, we call check_project() to set variables such as projectname, version, version date this is unnecessary, because all main scripts use this functionality anyway do it by default Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-11move script/*/* to script/Leah Rowe
there are only two scripts under script/ now, and there probably won't be many more. lbmk's design has simplified to such a degree that the two-level directory structure is no longer necessary. the existing command structure has not changed. for example: ./build roms list ./update trees -f coreboot default these will still work, but the symlinks to "build" are now strictly for backwards compatibility; they may be removed at a later date, but i'll keep the current design for now. this also leads to a quirk, for example: ./build roms all ./update roms all these now do the exact same thing, whereas "./update roms all" would have previously been an invalid command. Signed-off-by: Leah Rowe <leah@libreboot.org>