summaryrefslogtreecommitdiff
path: root/script/trees
AgeCommit message (Collapse)Author
2024-07-19put cachedir in environmental variableLeah Rowe
XBMK_CACHE is now used, instead of hardcoding cache/ this is exported initialised to cache/, if unset. this means you can set your own directory, and it means ./update release will use the same directory. this means bandwidth wastage is further avoided. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17trees: hardcode makeargs for xgccLeah Rowe
if coreboot itself is being handled, crossgcc has the correct makeargs, setting the number of build threads. however, other projects can specify "xtree" pointing to a given coreboot tree, and build crossgcc for it. one workaround may be to use trees -d coreboot TREE, but then extra code would have to be written to make it avoid other things like building cbfstool, which is not required for just building crossgcc. the cleanest way to do it is to simply hardcode it. the value is set exactly the same as regular coreboot makeargs. this fixes a bug, where some builds of crossgcc are made on a single thread, rather than using XBMK_THREADS. this patch forces it to always use CPUS=$XBMK_THREADS Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17trees: remove unnecessary commandLeah Rowe
we no longer need to remove cache/ per project, because it's removed in bulk at the end, in the main build script, when generating release archives. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17unify caching of files into cache/Leah Rowe
hash/ becomes cache/hash/ repo/ becomes cache/repo/ Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17trees: auto-delete+auto-rebuild if project changesLeah Rowe
actual source code is not scanned, but config directories are scanned. simply get the checksum of each file under config/ pertaining to a given project/tree, and also for the given target. coreboot utilities are also handled. if it changes, in any way, delete and re-build automatically. such deletions should probably still be done manually, as part of understanding the build system, but this change should make the build system much easier to use during development. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17trees: also remove single-tree repo/p/Leah Rowe
single-tree repos were not previously cached, but now they are and they have to be handled. this, as also alluded to in the previous commit, is done when preparing release archives (XBMK_RELEASE=y) Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17trees: remove repo/p, not src/p/pLeah Rowe
repo/p/ does not have its revision reset, so it changes unpredictably, and it's not used in builds. this used to be src/p/p/ - the context here is multi-tree projects, in source archives. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-16remove util/autoportLeah Rowe
upstream has merged all of the changes that it contained, so we don't need this anymore. we'll have the newer upstream changes on the next general revision updates for coreboot, within config/coreboot/ Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-13trees: move dependency building to new functionLeah Rowe
configure_project is a bit big. move the dependencies build logic to a new function. it may be desirable in future to make the way that function works the way all build commands are done. for example: ./update trees -b coreboot x230_12mb would become: ./update trees -b coreboot/x230_12mb this would enable to mix and match multi/single tree projects. for now, leave things as they are. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-11trees: fix bad comparison in configure_project()Leah Rowe
we used to set cmd only to these values: build_project build_targets however, now we set them to: build_project build_targets $@ the latter cannot be measured reliably, but we were checking whether cmd equalled: build_targets now we instead check that it does not equal: build_project Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-11trees: don't continue if no argument givenLeah Rowe
main() used to be the only function executed from outside of main(), in this script, but now we source a config file and then run the build afterward. when a flag is provided without OPTARG, this means that we are continuing such action erroneously. to mitigate this, return 1 in that instance, and handle it in the line that calls main(), making it exit with zero status (success). Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10trees: general code cleanupLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10trees: merge build_targets() with handle_targets()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10trees: use wrapper for dry-running make commandsLeah Rowe
instead of using lots of if/else conditions, do that once and set a variable, dry, to : if not doing a dry run, the variable is empty. prefix this variable in places where you don't want a certain action to be performed, on dry runs. more specifically, : does *nothing* and always returns with zero status (success). this results in cleaner code, and a small sloccount reduction. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10trees: remove project-specific hacksLeah Rowe
move the coreboot-specific includes into mkhelper.cfg for that project. on some projects, we need variables from mkhelper.cfg to be global, so I was including serprog and coreboot mkhelper.cfg files in this script. instead, set a new variable "mkhelpercfg" pointing to the config file. if it doesn't exist, create and then point to a temporary (empty) mkhelper.cfg file. the rom.sh include has been moved to coreboot mkhelper.cfg The only remaining project-specific logic, in this trees script, is now the coreboot crossgcc handling, but this needs to be there as it's also used to build U-Boot. The way this now works, certain includes are done twice. For example, include/rom.sh will be included once globally, outside of main(), and then again in configure_project(). This means that certain functions will be defined twice. I'm uncertain if shell has anything equivalent to an ifdef guard as in C, but we actually want this here anyway, and it shouldn't cause any problems. It's a bit of a hack, but otherwise results in much cleaner code. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10trees: fix bad comparisonLeah Rowe
pro-tip: don't do this at 3AM do massive changes like this, no later than 1AM. the intent anyway is for -d to cause no build dependencies to be handled, but the current logic says to only handle them if -d is set! fix it by removing the ! part Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-09trees: support -d (dry run) for custom build logicLeah Rowe
-d does the same as -b, except for actually building anything! in effect, it does the same as -f (fetch) except that the resulting variable assignments will not be recursive (as with -f). if -d is passed, configuration is still loaded, defconfig files are still cycled through, and more importantly: helper functions are still processed. the grub, serprog and coreboot helper functions have been modified to return early (zero status) if -d is passed. this behaviour will be used to integrate vendor.sh logic in with the trees script, for cases where the user wants to only handle vendor files. e.g.: ./update trees -b coreboot x230_12mb this would download the files as usual, build coreboot, with those files, and then build the payloads. but: ./update trees -d coreboot x230_12mb this would download the files, NOT build coreboot, and NOT build the payloads. this change increases the sloccount a bit, but i'm relying on the fact that the vendor.sh script already re-implements config handling wastefully; the plan is to only use trees. for now, simply stub the same ./vendor download command. there is one additional benefit to doing it this way: this method is *per-kconfig* rather than per-target. this way, one kconfig might specify a given vendor file that is not specified in the other. although the stub still simply handles this per target, it's done in premake, which means that the given .config file has been copied. this means that when i properly re-integrate the logic into script/trees, i'll be able to go for it per-kconfig. the utils command has been removed, e.g. ./update trees -b coreboot utils default the equivalent is now: ./update trees -d coreboot default this would technically download vendor files, but here we are specifying a target for which no kconfigs exist; a check is also in place, to avoid running the vendor file download logic if tree==target the overall effect of this change is that the trees script no longer contains any project-specific logic, except for the crossgcc build logic. it does include some config/data mkhelper files at the top, for serprog and coreboot, so that those variables defined in those files can be global, but another solution to mitigate that will also be implemented in a future commit. the purpose of this and other revisions (in the final push to complete lbmk audit 6 / cbmk audit 2) is to generalise as much logic as possible, removing various ugly hacks. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-07rom.sh: new file, to replace script/romsLeah Rowe
stub it from the trees script. the way it works now, there is less code in the build system. ./build roms this is no longer a thing ./build roms serprog this is also no longer a thing. instead, do: ./update trees -b coreboot targetnamehere ./update trees -b pico-serprog ./update trees -b stm32-vserprog the old commands still works, which causes the new commands to run coreboot roms now appear in elf/, not bin/, as before, but those images now contain payloads. NOTE: to contradict the above: ./build roms is no longer a thing, in that it's now deprecated, but backward compatibility is present for now. it will be removed in a future release. ./build roms list also still works! it will do: ./update trees -b coreboot list also: ./update trees -b grub list this is now possible too if a target "list" is provided, for multi-tree sources, the targets are shown. there is another difference: seagrub roms are now seagrub_, instead of seabios_withgrub. seabios-only roms are no longer provided, where grub is also enabled; only seagrub is used. the user can easily remove the bootorder file, if they want seabios to not try grub first. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-06trees: rename load_target_config()Leah Rowe
rename it to configure_project, because the function now also handles building (a little bit), not just mere loading of configuration files. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-06trees: support build dependencies, not just fetchLeah Rowe
We already have this to an extent, e.g. the xtree variable. The xtree variable could probably be removed, in favour of this, and used for the same purpose. It works like this, for example: build_depend="coreboot/default grub/xhci seabios u-boot/gru_bob flashprog" the "/" denotes a tree, if it's a multi-tree project. However, specifying the entire multi-tree project without slash is possible, for example: build_depend="coreboot" this would specify that all coreboot trees must be built. This functionality will be used in follow-up commits, centralising script/trees into mk on the main directory, repacing "build". Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-05trees: just do makeargs on coreboot, not cbmakeargLeah Rowe
stick the makeargs in mkhelper i previously did cbmakeargs because the old revisions had to define makeargs per-target otherwise. mkhelper was done specifically to solve that problem. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-01trees: fix bad rm -Rf command (-Rf, not Rf)Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: allow global config on multi-tree projectsLeah Rowe
config/data/PROJECT/mkhelper.cfg can be provided, for configuration, and it is loaded *before* target.cfg there are certain instances where we repeat a lot of config per tree, in multi-tree projects. for example, we have the exact same config per grub tree, besides tree name and revision number, for things like autoconf arguments. this last problem will be addressed, in a follow-up patch, and then expanded upon for other projects. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: handle all projects if project undefinedLeah Rowe
e.g. ./update trees -f if passed, this command would download every tree similarly, the -c option can be used in this way. this solves a longstanding issue: on the current, much more efficient design, it was not possible to systematically clean every project. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees and git.sh: tidy up global variablesLeah Rowe
some of the variables only initialised in git.sh are also used in the trees script, which is technically ok because git.sh is included from the trees script, but it makes more sense to declare them in the latter. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: simplified distclean directory checkLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: only do bare multi-tree clone on gitLeah Rowe
i forgot to add the check for CHANGELOG Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: simplified multi-tree bare repo cloneLeah Rowe
the git_prep function already creates the given directory where source code goes, so we don't need to do it from the trees script. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees run_make_command: simpler distclean handlingLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: condense copy_elf() a bitLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: add return to handle_defconfig()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: rename check_config to check_defconfigLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: remove variable "config_name"Leah Rowe
it's only ever used once, so just use the value that it's set to, without declaring the variable itself Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: rename variable "config" to defconfigLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: remove unnecessary commontsLeah Rowe
behaviour described there is obvious just from analysing the while loop condition Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: condense run_make_command() a bitLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: condense handle_makefile() a bitLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: mv load_project_config load_target_configLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees, multi: download bare project *before* treesLeah Rowe
when downloading multi-tree projects, the rev can be reset to HEAD instead of the actual rev for a given target. this occurs when the bare repo (e.g. src/coreboot/coreboot) does not exist and has to be downloaded first. bare repository downloading does not rely on target.cfg, in this context, only pkg.cfg, but it uses the same variable names (e.g. "rev"). instead of using a separate variable name, thus increasing code complexity (which is the exact opposite of what i want to do), do the bare repository download first. this means that the git.sh script is much cleaner now, for multi-tree projects, in that it *only* copies the bare repo then runs git_prep; in that context, the bare repo is cloned directly by calling the relevant function from script/trees, which is the same behaviour as when cloning single-tree project sources. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-30trees: unified multi-tree configuration handlingLeah Rowe
the same function that loads configurations for single-tree projects has been merged with the function for multi-tree configs in git.sh, and that functionality has been removed from git.sh; now it is all unified in the trees script. as the saying goes: write one program to do one thing well. the purpose of git.sh is to download source code, but not to handle configuration files; the latter is meant to be handled by the trees script, which then calls into git.sh before running the build logic for that given project. additionally: the "seen" files are no longer handled, at all. the logic there was added ages ago, because at the time, i was considering whether to separate configuration into a new repository, so that users could more easily make their own configuration, so it was a guard against misconfiguration. however, that decision was canceled and we're always very careful not to introduce a loop; if a loop does occur, the worst that can possibly happen is you waste some CPU cycles. Instead, print (on standard output) what config file is being used, so the operator can see when an infinite loop occurs. ALSO: remove _setcfgarg in load_project_config() it was used to skip when a target.cfg file didn't exist, specifically on single-tree projects, but this is now handled using -f instead, on the while loop inside that function, so _setcfgarg is now a redundant variable. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees: unified handling of source downloadsLeah Rowe
run it from load_project_config() Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29git.sh: rename Fetch_project_trees fetch_targetsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29git.sh: rename fetch_project_repo to fetch_projectLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees: better skip-clean string in handle_src_treeLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees: add return to check_coreboot_utils()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees: simplify "utilmode" variable initialisationLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-29trees mkpayload_grub: remove unnecessary checkLeah Rowe
these variables are initialised empty, then populated by reading a configuration file. it may be that in some cases, we want these variables to be empty. besides that, the "setcfg" command before it will throw an error if the module file is missing, and it is assumed that the variables would be set there. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28tidy up some setvars listsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-28remove more unnecessary checks on trees commandsLeah Rowe
the trees script already performs these checks Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-06-27trees: don't hardcode use of mkpayload_grubLeah Rowe
instead, make it a helper function, defined in target.cfg this means that we can also do the same with other projects in the future, and it is expected that we will have to. these helper functions are used in cases where we want additional actions to be performed. actually, the helper could be anything. for example, you could write: mkhelper="./build foo bar" and it would do that (at the point of execution, PWD is the root directory of the build system) Signed-off-by: Leah Rowe <leah@libreboot.org>