summaryrefslogtreecommitdiff
path: root/script
AgeCommit message (Collapse)Author
2024-06-30trees, multi: download bare project *before* treesLeah Rowe
when downloading multi-tree projects, the rev can be reset to HEAD instead of the actual rev for a given target. this occurs when the bare repo (e.g. src/coreboot/coreboot) does not exist and has to be downloaded first. bare repository downloading does not rely on target.cfg, in this context, only pkg.cfg, but it uses the same variable names (e.g. "rev"). instead of using a separate variable name, thus increasing code complexity (which is the exact opposite of what i want to do), do the bare repository download first. this means that the git.sh script is much cleaner now, for multi-tree projects, in that it *only* copies the bare repo then runs git_prep; in that context, the bare repo is cloned directly by calling the relevant function from script/trees, which is the same behaviour as when cloning single-tree project sources. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: unified multi-tree configuration handlingLeah Rowe
the same function that loads configurations for single-tree projects has been merged with the function for multi-tree configs in git.sh, and that functionality has been removed from git.sh; now it is all unified in the trees script. as the saying goes: write one program to do one thing well. the purpose of git.sh is to download source code, but not to handle configuration files; the latter is meant to be handled by the trees script, which then calls into git.sh before running the build logic for that given project. additionally: the "seen" files are no longer handled, at all. the logic there was added ages ago, because at the time, i was considering whether to separate configuration into a new repository, so that users could more easily make their own configuration, so it was a guard against misconfiguration. however, that decision was canceled and we're always very careful not to introduce a loop; if a loop does occur, the worst that can possibly happen is you waste some CPU cycles. Instead, print (on standard output) what config file is being used, so the operator can see when an infinite loop occurs. ALSO: remove _setcfgarg in load_project_config() it was used to skip when a target.cfg file didn't exist, specifically on single-tree projects, but this is now handled using -f instead, on the while loop inside that function, so _setcfgarg is now a redundant variable. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees: unified handling of source downloadsLeah Rowe
run it from load_project_config() Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29git.sh: rename Fetch_project_trees fetch_targetsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29git.sh: rename fetch_project_repo to fetch_projectLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees: better skip-clean string in handle_src_treeLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees: add return to check_coreboot_utils()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees: simplify "utilmode" variable initialisationLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees mkpayload_grub: remove unnecessary checkLeah Rowe
these variables are initialised empty, then populated by reading a configuration file. it may be that in some cases, we want these variables to be empty. besides that, the "setcfg" command before it will throw an error if the module file is missing, and it is assumed that the variables would be set there. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28tidy up some setvars listsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28roms: explicitly shift by 1 on the "all" targetLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28roms: add return value for the list commandLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28roms: build u-boot *before* checking ubootelfLeah Rowe
the current check is flawed, because if u-boot doesn't exist, but a given build would be the file verified by the first check, the check would still fail even after then building u-boot. building it first will make this check pass, under such a condition. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28remove more unnecessary checks on trees commandsLeah Rowe
the trees script already performs these checks Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: don't hardcode use of mkpayload_grubLeah Rowe
instead, make it a helper function, defined in target.cfg this means that we can also do the same with other projects in the future, and it is expected that we will have to. these helper functions are used in cases where we want additional actions to be performed. actually, the helper could be anything. for example, you could write: mkhelper="./build foo bar" and it would do that (at the point of execution, PWD is the root directory of the build system) Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: simplify single/multi handling in main()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: rename build_projects to build_projectLeah Rowe
the script used to support building multiple single-tree projects, but this behaviour was buggy and unused, so it was removed. rename the build_projects variable accordingly. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: err if target.cfg not given if multi-treeLeah Rowe
it was always by design that an error should occur, if a target.cfg file does not exist on multi-tree projects, but we previously did not support target.cfg files on single-tree projects. single-tree target.cfg support was later added, and it was done by making target.cfg optional there, but i accidentally made it optional on multi-tree projects. in practise, all multi-tree projects included target.cfg, but this was not being enforced in code. this patch should fix the issue. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: set btype if target.cfg doesn't existLeah Rowe
set it to "auto", because otherwise it'll be unset, which means that kconfig type is assumed. the build system is designed in such a way that multi-tree is assumed, if the target build system uses kconfig files. target.cfg is optional on single-tree but not multi-tree, so it's ok to set something here. basically, kconfig-type projects will never be single-tree. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27roms: remove unnecessary assignment to cbdirLeah Rowe
cbdir isn't used in this script Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: avoid kconfig make commands genericallyLeah Rowe
don't hardcode the check based on whether the current project is grub. instead, define "btype" in target.cfg if unset, we assume kconfig and permit kconfig commands e.g. make menuconfig, make silentoldconfig, etc this is to avoid the deadliest of sins: project-specific hacks Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: condense elfcheck() a bitLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: shorten the final confirmation messageLeah Rowe
must stay below 80 characters per line Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27lib.sh: simplified TMPDIR handlingLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: remove redundant space in printfLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: explicitly err if OPTARG is not setLeah Rowe
we currently rely on -e to make this happen Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: only permit one single-tree projectLeah Rowe
the current logic for handling multiple single-tree projects is quite error-prone, and uses recursion. since we don't actually use it this way, remove that feature. the most correct way to do it is with a for loop. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: call err if multiple flags are providedLeah Rowe
this script is designed to only run a single flag. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: explicitly set mode for -bLeah Rowe
doing nothing means that if a flag is passed, and then another flag overriding it, the resulting action will not be correct; only one flag should be provided anyway, but some users may feel a bit more adventurous. mitigate it. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27roms: re-add compression of coreboot imagesLeah Rowe
i accidentally removed this feature, during prior cleanup. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-25roms: build coreboot *after* checking variablesLeah Rowe
otherwise, release=n is ignored and an image is built in the elf/ directory, even if it's still skipped for bin/ avoid doing unnecessary work per-release by checking the variables before building coreboot via script/trees Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-25lib.sh: simplify use of environment variablesLeah Rowe
don't have a separate variable for them. just export them directly and use them directly. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24roms main(): confirm what serprog images are builtLeah Rowe
just like we do for coreboot images, do it for serprog. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24roms: remove unused variable nameLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24roms: remove redundant printfLeah Rowe
we don't need this message here, because the final confirmation at the end of main() says which targets were built. saying what individual rom images were built is just needless bloat, especially with the new simplified lbmk design; we no longer provide lots of rom images with different keymaps, because we now expect the user to insert a gkb file themselves with cbfstool. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24roms: optimise u-boot elf checkLeah Rowe
because we use crossgcc here, blindly running trees -f means needlessly re-running buildgcc, which then checks for gcc binaries, even though we already know that the u-boot binary exists. skip this check if u-boot exists. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24roms: simplify build_roms()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24roms: make the bin/elf message more politeLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24roms: re-add final confirmation of targetsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24roms: rename functions for extra clarityLeah Rowe
the names of these functions do not match their true intent. fix that by renaming them appropriately. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24roms: build coreboot early to avoid duplicate workLeah Rowe
we're building it per coreboot configuration file, rather than per-target; the latter is more appropriate, and saves on compilation time. do it per-target.cfg, not per coreboot configuration. this works because the trees script compiles all images per target, for each given coreboot configuration within that target, e.g. libgfxinit _corebootfb and _txtmode. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24trees: try xgcc build twice if first attempt failsLeah Rowe
sometimes buildgcc just fails for like no reason. we had this the other day and another fix was made to the trees script, to mitigate; the user ran it again and buildgcc worked just fine. run it twice, and then call err only if the second one fails. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-23trees: don't check if xgcc is already builtLeah Rowe
just run the make crossgcc command anyway. coreboot's own build system checks itself, and much more reliably, but the check is more thorough and a bit slower. in rare cases, lbmk may come into build issues with xgcc, and if you run the build again, it will always fail every time because the checks is based on whether the xgcc directory exists, rather than checking each individual crossgcc binary. checking every binary is also possible, but as i said, the coreboot build system already does that, so let's defer to coreboot's own handling of it. remove the directory check. this will slow down the build process a little bit, but should improve reliability under fault conditions. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-23roms: general code cleanupLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22roms: only support SeaBIOS/SeaGRUB on x86Leah Rowe
Never, ever build images where GRUB is the primary payload. These options have been removed from target.cfg handling: * seabios_withgrub * grub_withseabios The "payload_grub" variable now does the same thing as the old "seabios_withgrub" variable, if set. The "grubonly" configuration is retained, and enabled by default when SeaGRUB is enabled (non-grubonly also available). Due to lbmk issue #216, it is no longer Libreboot policy to make GRUB the primary payload on any board. GRUB's sheer size and complexity, plus the large number of memory corruption issues similar to it that *have* been fixed over the years, tells me that GRUB is a liability when it is the primary payload. SeaBIOS is a much safer payload to run as primary, on x86, due to its smaller size and much more conservative development; it is simply far less likely to break. If GRUB breaks in the future, the user's machine is not bricked. This is because SeaBIOS is the default payload. Since I no longer wish to ever provide GRUB as a primary payload, supporting it in lbmk adds needless bloat that will later probably break anyway due to lack of testing, so let's just assume SeaGRUB in all cases where the user wants to use a GRUB payload. You can mitigate potential security issues with SeaBIOS by disabling option ROM execution, which can be done at runtime by inserting integers into CBFS. The SeaBIOS documentation says how to do this. Libreboot's GRUB hardening guide still says how to add a bootorder file in CBFS, making SeaBIOS only load GRUB from CBFS, and nothing else. This, combined with the disablement of option ROM execution (if using Intel graphics), pretty much provides the same security benefits as GRUB-as-primary, for example when setting a GRUB password and GPG checks, with encrypted /boot as in the hardening guide. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22roms: remove support for "grubonly" seabiosLeah Rowe
due to lbmk issue #216, it is now unwise to use grub as the primary payload on any machine; the sheer complexity of grub and the number of memory corruption bugs that have been fixed due to auditing over the years, means more such bugs exist. we now provide seabios as the primary payload on all x86 ports, but provide a "grubfirst" configuration where a bootorder file in seabios can be added via cbfs, which tells seabios to load grub from cbfs first, while still allowing use of the boot select menu by pressing esc in seabios. well, the "grubonly" option also disables the seabios esc menu, so that *only* grub runs. there is no point in using this unless you want to harden your setup, for example if you want to set up encrypted /boot and boot that from grub, and have a grub password disallowing unauthorised bootup of your machine. see grub hardening guide; https://libreboot.org/docs/linux/grub_hardening.html at least as of today, 22 June 2024, that page already says how to manually disable the seabios menu in the same way, if that is the setup you want. alternatively, a user may be wily enough to edit target.cfg for their board and compile a rom that only has the grub payload in it, if that is what the user wishes to do. regardless, the default configurations provided by lbmk must never be unsafe, norc should the build system support such unsafe settings; yes, grub as primary payload is technically still supported in lbmk. actually, at the time of this revision, i have half a mind to remove that functionality altogether, so that only seabios is allowed as primary payload, when compiling a rom image that also has grub, chainloading grub from the seabios menu instead. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22use backticks on eval commands, not subshellsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22lib.sh: remove badcmd()Leah Rowe
it's bloat. telling the user to rtfm is something that we already do on irc; they will still ask how to do everything, and ignore the message from badcmd(), or they will automatically know to rtfm. i'm on a massive purge, removing bloat from lbmk as part of Libreboot Build System Audit 6. all bloat must go. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22lib.sh: more unified config handlingLeah Rowe
replace it with logic that simply uses "." to load files directly. for this, "vcfg" is added as a variable in coreboot target.cfg files, referring to a directory in config/vendor/ containing a file named pkg.cfg, and this file then contains the same variables as the erstwhile config/vendor/sources config/git files are now directories, also containing pkg.cfg files each with the same variables as before, such as repository link and commit hash this change results in a noticeable reduction in code complexity within the build system. unified reading of config files: new function setcfg() added to lib.sh setcfg checks if a config exists. if a 2nd argument is passed, it is used as a return value for eval, otherwise a string calling err is passed. setcfg output is passed through eval, to set strings based on config; eval must be used, so that the variables are set within the same scope, otherwise they'd be set within setcfg which could lead to some whacky results. there's still a bit more more to do, but this single change results in a substantial reduction in code complexity. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-21trees: more robust check to avoid "make fetch"Leah Rowe
do not use shorthand here. the test was failing to produce the desired result under some circumstances, for example when i did "./update release" i got this: make: Entering directory '/home/lbdev/lbmk/release/20240612-62-ga6b1a6bd/libreboot-20240612-62-ga6b1a6bd_src/src/stm32-vserprog' make: *** No rule to make target 'fetch'. Stop. make: Leaving directory '/home/lbdev/lbmk/release/20240612-62-ga6b1a6bd/libreboot-20240612-62-ga6b1a6bd_src/src/stm32-vserprog' ERROR script/trees: !mk src/stm32-vserprog fetch ERROR ./update: excmd: script/trees -f ERROR script/roms: Unhandled non-zero exit: ./update ERROR ./build: excmd: script/roms serprog ERROR ./update: build_release release/20240612-62-ga6b1a6bd: stm32 ERROR ./update: can't build rom images in the above circumstance, run_make_command was executed, which is not the desired behaviour; rather, fetch_project_trees or fetch_project_repo should be called, and then the script should immediately exit. it should also exit, without downloading anything, if a changelog file exists as in release archives. Signed-off-by: Leah Rowe <leah@libreboot.org>