summaryrefslogtreecommitdiff
path: root/script/trees
AgeCommit message (Collapse)Author
2024-06-30trees, multi: download bare project *before* treesLeah Rowe
when downloading multi-tree projects, the rev can be reset to HEAD instead of the actual rev for a given target. this occurs when the bare repo (e.g. src/coreboot/coreboot) does not exist and has to be downloaded first. bare repository downloading does not rely on target.cfg, in this context, only pkg.cfg, but it uses the same variable names (e.g. "rev"). instead of using a separate variable name, thus increasing code complexity (which is the exact opposite of what i want to do), do the bare repository download first. this means that the git.sh script is much cleaner now, for multi-tree projects, in that it *only* copies the bare repo then runs git_prep; in that context, the bare repo is cloned directly by calling the relevant function from script/trees, which is the same behaviour as when cloning single-tree project sources. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: unified multi-tree configuration handlingLeah Rowe
the same function that loads configurations for single-tree projects has been merged with the function for multi-tree configs in git.sh, and that functionality has been removed from git.sh; now it is all unified in the trees script. as the saying goes: write one program to do one thing well. the purpose of git.sh is to download source code, but not to handle configuration files; the latter is meant to be handled by the trees script, which then calls into git.sh before running the build logic for that given project. additionally: the "seen" files are no longer handled, at all. the logic there was added ages ago, because at the time, i was considering whether to separate configuration into a new repository, so that users could more easily make their own configuration, so it was a guard against misconfiguration. however, that decision was canceled and we're always very careful not to introduce a loop; if a loop does occur, the worst that can possibly happen is you waste some CPU cycles. Instead, print (on standard output) what config file is being used, so the operator can see when an infinite loop occurs. ALSO: remove _setcfgarg in load_project_config() it was used to skip when a target.cfg file didn't exist, specifically on single-tree projects, but this is now handled using -f instead, on the while loop inside that function, so _setcfgarg is now a redundant variable. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees: unified handling of source downloadsLeah Rowe
run it from load_project_config() Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29git.sh: rename Fetch_project_trees fetch_targetsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29git.sh: rename fetch_project_repo to fetch_projectLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees: better skip-clean string in handle_src_treeLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees: add return to check_coreboot_utils()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees: simplify "utilmode" variable initialisationLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees mkpayload_grub: remove unnecessary checkLeah Rowe
these variables are initialised empty, then populated by reading a configuration file. it may be that in some cases, we want these variables to be empty. besides that, the "setcfg" command before it will throw an error if the module file is missing, and it is assumed that the variables would be set there. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28tidy up some setvars listsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28remove more unnecessary checks on trees commandsLeah Rowe
the trees script already performs these checks Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: don't hardcode use of mkpayload_grubLeah Rowe
instead, make it a helper function, defined in target.cfg this means that we can also do the same with other projects in the future, and it is expected that we will have to. these helper functions are used in cases where we want additional actions to be performed. actually, the helper could be anything. for example, you could write: mkhelper="./build foo bar" and it would do that (at the point of execution, PWD is the root directory of the build system) Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: simplify single/multi handling in main()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: rename build_projects to build_projectLeah Rowe
the script used to support building multiple single-tree projects, but this behaviour was buggy and unused, so it was removed. rename the build_projects variable accordingly. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: err if target.cfg not given if multi-treeLeah Rowe
it was always by design that an error should occur, if a target.cfg file does not exist on multi-tree projects, but we previously did not support target.cfg files on single-tree projects. single-tree target.cfg support was later added, and it was done by making target.cfg optional there, but i accidentally made it optional on multi-tree projects. in practise, all multi-tree projects included target.cfg, but this was not being enforced in code. this patch should fix the issue. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: set btype if target.cfg doesn't existLeah Rowe
set it to "auto", because otherwise it'll be unset, which means that kconfig type is assumed. the build system is designed in such a way that multi-tree is assumed, if the target build system uses kconfig files. target.cfg is optional on single-tree but not multi-tree, so it's ok to set something here. basically, kconfig-type projects will never be single-tree. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: avoid kconfig make commands genericallyLeah Rowe
don't hardcode the check based on whether the current project is grub. instead, define "btype" in target.cfg if unset, we assume kconfig and permit kconfig commands e.g. make menuconfig, make silentoldconfig, etc this is to avoid the deadliest of sins: project-specific hacks Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: condense elfcheck() a bitLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: shorten the final confirmation messageLeah Rowe
must stay below 80 characters per line Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: remove redundant space in printfLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: explicitly err if OPTARG is not setLeah Rowe
we currently rely on -e to make this happen Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: only permit one single-tree projectLeah Rowe
the current logic for handling multiple single-tree projects is quite error-prone, and uses recursion. since we don't actually use it this way, remove that feature. the most correct way to do it is with a for loop. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: call err if multiple flags are providedLeah Rowe
this script is designed to only run a single flag. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: explicitly set mode for -bLeah Rowe
doing nothing means that if a flag is passed, and then another flag overriding it, the resulting action will not be correct; only one flag should be provided anyway, but some users may feel a bit more adventurous. mitigate it. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-25lib.sh: simplify use of environment variablesLeah Rowe
don't have a separate variable for them. just export them directly and use them directly. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-24trees: try xgcc build twice if first attempt failsLeah Rowe
sometimes buildgcc just fails for like no reason. we had this the other day and another fix was made to the trees script, to mitigate; the user ran it again and buildgcc worked just fine. run it twice, and then call err only if the second one fails. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-23trees: don't check if xgcc is already builtLeah Rowe
just run the make crossgcc command anyway. coreboot's own build system checks itself, and much more reliably, but the check is more thorough and a bit slower. in rare cases, lbmk may come into build issues with xgcc, and if you run the build again, it will always fail every time because the checks is based on whether the xgcc directory exists, rather than checking each individual crossgcc binary. checking every binary is also possible, but as i said, the coreboot build system already does that, so let's defer to coreboot's own handling of it. remove the directory check. this will slow down the build process a little bit, but should improve reliability under fault conditions. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22use backticks on eval commands, not subshellsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22lib.sh: remove badcmd()Leah Rowe
it's bloat. telling the user to rtfm is something that we already do on irc; they will still ask how to do everything, and ignore the message from badcmd(), or they will automatically know to rtfm. i'm on a massive purge, removing bloat from lbmk as part of Libreboot Build System Audit 6. all bloat must go. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-22lib.sh: more unified config handlingLeah Rowe
replace it with logic that simply uses "." to load files directly. for this, "vcfg" is added as a variable in coreboot target.cfg files, referring to a directory in config/vendor/ containing a file named pkg.cfg, and this file then contains the same variables as the erstwhile config/vendor/sources config/git files are now directories, also containing pkg.cfg files each with the same variables as before, such as repository link and commit hash this change results in a noticeable reduction in code complexity within the build system. unified reading of config files: new function setcfg() added to lib.sh setcfg checks if a config exists. if a 2nd argument is passed, it is used as a return value for eval, otherwise a string calling err is passed. setcfg output is passed through eval, to set strings based on config; eval must be used, so that the variables are set within the same scope, otherwise they'd be set within setcfg which could lead to some whacky results. there's still a bit more more to do, but this single change results in a substantial reduction in code complexity. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-21trees: more robust check to avoid "make fetch"Leah Rowe
do not use shorthand here. the test was failing to produce the desired result under some circumstances, for example when i did "./update release" i got this: make: Entering directory '/home/lbdev/lbmk/release/20240612-62-ga6b1a6bd/libreboot-20240612-62-ga6b1a6bd_src/src/stm32-vserprog' make: *** No rule to make target 'fetch'. Stop. make: Leaving directory '/home/lbdev/lbmk/release/20240612-62-ga6b1a6bd/libreboot-20240612-62-ga6b1a6bd_src/src/stm32-vserprog' ERROR script/trees: !mk src/stm32-vserprog fetch ERROR ./update: excmd: script/trees -f ERROR script/roms: Unhandled non-zero exit: ./update ERROR ./build: excmd: script/roms serprog ERROR ./update: build_release release/20240612-62-ga6b1a6bd: stm32 ERROR ./update: can't build rom images in the above circumstance, run_make_command was executed, which is not the desired behaviour; rather, fetch_project_trees or fetch_project_repo should be called, and then the script should immediately exit. it should also exit, without downloading anything, if a changelog file exists as in release archives. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-20minor code cleanup in the build systemLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-16unify coreboot elfdir (DO_NOT_FLASH)Leah Rowe
use a common string when setting this path Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15trees: simplify copy_elf()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15trees: remove unnecessary check in copy_elfLeah Rowe
we don't need to check whether this variable is set, because checking an empty path will also cause the same return in the next line. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-15grub: insert background in memdisk insteadLeah Rowe
the background is only a few kb. the whole rationale before was to limit the space used in memdisk, but this decision was made when the background was much bigger; it has since been optimised greatly, and the grub modules were heavily reduce, so it should be safe. grub's memdisk breaks when you add too much data to it. as part of simplifying the rest of lbmk, this change removes some more bloat from the rest of lbmk. handling this in the memdisk is much simpler than handling it with cbfstool. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-14unified checks for variable initialisationLeah Rowe
new function chkvars() does the job Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-14lib.sh: remove the items() functionLeah Rowe
it's pretty much just doing the same thing as ls -1 remove it! Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09re-add ability to use cbfs grub.cfg as defaultLeah Rowe
i removed this before, when making grub multi-tree, because the design i used in an earlier version of the patch actually added the grub.elf generation to grub source itself, but then i decided to hack around the grub build system from lbmk/cbmk instead re-add this functionality, so that users can easily insert their own custom grub.cfg into cbfs without needing to re-build their image. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09trees: exit with error if project undefinedLeah Rowe
i was originally looser about this, because i also wanted the trees script to generically run "make" from any directory, but this behaviour was error-prone and it is no longer used in the build system. disable it, in the interest of stability. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-09add crossgcc tarballs to config/submodules/Leah Rowe
support redundant downloads, and enable inclusion of these tarballs inside release archives, for offline builds. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: simplified copy_elf() handlingLeah Rowe
don't create elfdir, create dest_dir, which is elfdir plus the location within it only create dest_dir within copy_elf, which is only called if actually compiling the code this avoids creating empty elf directories, and it generally cleans up all handling, unifying the handling of directories into a single function, namely copy_elf() which already exists Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07git.sh: download "depend" projects *before*Leah Rowe
don't do it after, because that means the main project is saved under src/ before we know whether the subrepo was downloaded. the "depend" variable (in config/git/) is no longer used for projects that go in subdirectories of a parent; now, we use config/submodules/ for this type of dependency. download the "depend" projects (as per config/git/) first. this way, if they fail, the main one will fail, but if they succeed and main fails, you can just run the main download again and it won't fail. this fixes a bug where, depending on how you download a set of projects and depending on the order which you do so, a given project can become un-downloadable on current design, because git will complain that a directory already exists. this fix is done not only in code (by this commit), but by prior configuration changes. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: don't check empty path in copy_elf()Leah Rowe
it could lead to some whackiness later on Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: fix build issue caused by bad elf checkLeah Rowe
we're not checking for bad elfs, but the check itself was bad due to a quirk in how sh works. really, really obscure bug. fixed now! if the given directory didn't actually exist, or there were no files in it, it'd be searching for the file named "*" which is obviously wrong Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: fix listfile check in copy_elf()Leah Rowe
don't check that the variable is empty check that the file itself exists or not this should fix the recent build issues Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: don't say check elf/ if build.list missingLeah Rowe
(on single tree projects. this complements the last patch) Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-07trees: don't do elfcheck if build.list missingLeah Rowe
otherwise, some checks are done too soon, and nothing gets built. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06trees: unified coreboot makeargsLeah Rowe
in particular, the coreboot build system may auto-download submodules when building cbfstool; vboot for instance. we do not want such unpredictable behaviour, so now we use UPDATED_SUBMODULES=1 when building coreboot utilities. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-06trees: use multiple threads to build cbutilsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>