summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-07-26general code cleanup in the build systemLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22rom.sh: don't dry-run mkcoreboottarLeah Rowe
same as the last change. make the main function a wrapper that dry-runs the real function. if the "dry" variable is blank, it executes. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22rom.sh: don't run mkcorebootbin on trees -dLeah Rowe
don't let it execute during dry builds Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22support ./mk in place of ./update treesLeah Rowe
it's a shortcut command. a new symlink. ./update trees -b flashprog can instead do: ./mk -b flashprog Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22trees: initialise badhash no n, not empty stringLeah Rowe
this is another alternative to the previous fix. this one is therefore now a pre-emptive fix, in case other code is written in the future that makes use of badhash. the badhash variable in a y/n variable, so initialise to n. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22rom.sh: fix buggy deletion of cbutilsLeah Rowe
when badhash=y, the utils should be deleted, but the check is deleting if badhash isn't n. if the hash check isn't being performed, then this will always be the case and the utils are always deleted. make it positively delete the file only if badhash=y, not when it isn't n. while this may not sound very different, it will prevent the utils being deleted and re-build endlessly in other cases, like when building release archives and running the inject --nuke mode on every image that gets built. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22rom.sh: also add grub to seabios imagesLeah Rowe
we want multiple seagrub images made, with different keymaps, but we only want one non-seagrub image. however, we also want grub in the non-seagrub image. it just means that seabios is primarily what the user wants, and they might occasionally use grub, whereas the seagrub images are for people who primarily want grub but may occasionally access the seabios menu. right now, the seabios images really only contain seabios, but there's no harm in adding grub to them. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22rom.sh: copy tmprom to TMPDIR for modificationLeah Rowe
don't rely on build/coreboot.rom staying in place, because sometimes it can get purged under certain conditions, due to idiosyncrasies in the coreboot build system, even when we don't explicitly clean it Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22rom.sh: re-add seabios-only roms and grub keymapsLeah Rowe
this time, only handle multiple keymaps on seagrub images. for images where seabios is first but does not immediately load grub, whether grub is still available in flash, just do one image (US Qwerty) this still results in fewer images per target than Libreboot 20240612, but should prevent most users from being annoyed. i got a few people asking repeatedly, and i hadn't documented yet how to add keymap.gkb or how to remove bootorder, to get a different keymap or disable seagrub respectively. i anticipate that i'll get such questions a lot, even if i do document it, so i'm reversing that decision. it doesn't result in much extra code. the new design in lbmk makes this sort of thing much simpler. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-22emphasis on readme that lbmk isn't a coreboot forkLeah Rowe
i actually only made this change so that the revision changes, so that the release directory changes when doing: ./update release this is to test whether such location change affects the build time when using ccache. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-21include/rom.sh: use ccache when building corebootLeah Rowe
ccache now required, in build dependencies Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-19vendor.sh: don't use XBMK_CACHE for appdiraudit6Leah Rowe
the me_extract function prefixes it with PWD in some cases, but we can't predict where appdir will point to. the "app" directory is not intended to be a cache anyway, so it doesn't make sense to put it in the cache directory. it's essentially scratch memory. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-19put cachedir in environmental variableLeah Rowe
XBMK_CACHE is now used, instead of hardcoding cache/ this is exported initialised to cache/, if unset. this means you can set your own directory, and it means ./update release will use the same directory. this means bandwidth wastage is further avoided. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-18git.sh: warn when a cached clone failsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-18git.sh: fix typo in git commandLeah Rowe
the || : condition should be used, whereas i just wrote : by mistake. this was done in a previous change. fix it now. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-18git.sh: fix lack of error exits on faultLeah Rowe
a previous change made it more redundant, falling back on old behaviour (direct downloading, not cached), but the way it's done means that the function never returns an error condition in practise. this patch fixes it. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-18build: remove tmp/ on release archivesLeah Rowe
i overlooked this before. remove it. the directory happened to be empty when i tested archives, but it's still not a good thing that we have it. remove it! Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17trees: hardcode makeargs for xgccLeah Rowe
if coreboot itself is being handled, crossgcc has the correct makeargs, setting the number of build threads. however, other projects can specify "xtree" pointing to a given coreboot tree, and build crossgcc for it. one workaround may be to use trees -d coreboot TREE, but then extra code would have to be written to make it avoid other things like building cbfstool, which is not required for just building crossgcc. the cleanest way to do it is to simply hardcode it. the value is set exactly the same as regular coreboot makeargs. this fixes a bug, where some builds of crossgcc are made on a single thread, rather than using XBMK_THREADS. this patch forces it to always use CPUS=$XBMK_THREADS Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17allow using coreboot's build system to add payloadLeah Rowe
lbmk must still define payloads, but specific configs may use coreboot's build system instead. you might use this to add your own config with, say, tianocore payload, using coreboot.git to build it, rather than using lbmk's choice of payloads. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17trees: remove unnecessary commandLeah Rowe
we no longer need to remove cache/ per project, because it's removed in bulk at the end, in the main build script, when generating release archives. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17build: remove cache/ on making releasesLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17unify caching of files into cache/Leah Rowe
hash/ becomes cache/hash/ repo/ becomes cache/repo/ Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17unified cache file handling for vendorfile/subfileLeah Rowe
lib.sh download() is used by subfile handling in git.sh, e.g. crossgcc tarballs, and also the vendor scripts. vendor files are cached, but not subfiles for repos. cache both, under cache/file/, saved with the name equal to the checksum, so: cache/file/CHECKSUM also move vendorfiles/app/ to cache/app/ in this change. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17git.sh: remove previous tmprepo if args>5Leah Rowe
if doing a retry, the directory may still exist, which would make git clone yield an error response; the existing directory will have been the one that failed to reset, so let's delete it. the one deleted is not the cache (repo/PROJECT/), thus otherwise maintaining current behaviour. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17git.sh: try direct clone if cached git failsLeah Rowe
normally, a project is cached at repo/PROJECT/, and cloned from there to the final destination. errors lead to a calling of $err, but this will result in a return if done from inside a subshell, of non-zero value, so use this to re-try with a 6th argument when calling tmpclone(). in most cases, this fallback will never kick in, but it will kick in resetting or patching the cached clone fails; specifically, we are interested in the reset part. a given project name may change repositories in lbmk at a given time. if this happens, and the old one is cached, the overall result of this patch is that lbmk will fall back to the old behaviour, where git urls are tried directly, without caching. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17git.sh: re-try git pull three timesLeah Rowe
mitigate jittery internet connections Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17trees: auto-delete+auto-rebuild if project changesLeah Rowe
actual source code is not scanned, but config directories are scanned. simply get the checksum of each file under config/ pertaining to a given project/tree, and also for the given target. coreboot utilities are also handled. if it changes, in any way, delete and re-build automatically. such deletions should probably still be done manually, as part of understanding the build system, but this change should make the build system much easier to use during development. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17trees: also remove single-tree repo/p/Leah Rowe
single-tree repos were not previously cached, but now they are and they have to be handled. this, as also alluded to in the previous commit, is done when preparing release archives (XBMK_RELEASE=y) Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17trees: remove repo/p, not src/p/pLeah Rowe
repo/p/ does not have its revision reset, so it changes unpredictably, and it's not used in builds. this used to be src/p/p/ - the context here is multi-tree projects, in source archives. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17git.sh: don't download to src/project/project/Leah Rowe
re-use repo/project/ this means that single- and multi-tree projects now have a unified cached git repo location, as per the new rules, thus saving on disk space usage. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-17git.sh: cache git downloads to repo/Leah Rowe
do it based on the URL, e.g. https://review.coreboot.org/coreboot becomes repo/coreboot the downside is if you have two projects with repo urls specifying the same string at the end, but this isn't the case at the moment and likely won't be the case, but it's a theoretical issue. this saves on bandwidth when downloading identical submodule repos between multiple trees within the same multi-tree project for example, coreboot 3rdparty/vboot is no longer downloaded more than once, instead cloned locally on subsequent downloads. if repo/DIR exists, git-pull is attempted, but errors do not result in a non-zero exit, by design. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-16remove util/autoportLeah Rowe
upstream has merged all of the changes that it contained, so we don't need this anymore. we'll have the newer upstream changes on the next general revision updates for coreboot, within config/coreboot/ Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-13trees: move dependency building to new functionLeah Rowe
configure_project is a bit big. move the dependencies build logic to a new function. it may be desirable in future to make the way that function works the way all build commands are done. for example: ./update trees -b coreboot x230_12mb would become: ./update trees -b coreboot/x230_12mb this would enable to mix and match multi/single tree projects. for now, leave things as they are. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-11Merge pull request 'Update U-Boot to v2024.07' (#225) from ↵Leah Rowe
alpernebbi/lbmk:uboot-v2024.07 into master Reviewed-on: https://codeberg.org/libreboot/lbmk/pulls/225
2024-07-11u-boot: Fix display initialization on gru boardsAlper Nebi Yasak
U-Boot has migrated to using upstream device-tree files for gru boards, but the clock driver doesn't yet support setting rates for a certain clock that upstream uses for the eDP display. It happens to work without it, so for now remove the clock setting until the driver is fixed. Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
2024-07-11u-boot: Avoid breaking build for U-Boot-only binman imagesAlper Nebi Yasak
Commit 46e01c0e1dad ("u-boot: Avoid building U-Boot-only binman images") added a patch that prevents an error while building U-Boot, due to some U-Boot images needing a copy of BL31 that we are not passing in. Removing build instructions for these images isn't really necessary, when we can instead tell the build tool that it shouldn't exit with an error. It checks a BINMAN_ALLOW_MISSING environment variable for this, but just unconditionally replace the check with the argument. Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
2024-07-11u-boot: Update to v2024.07Alper Nebi Yasak
Set default U-Boot revision to v2024.07 and rebase patches on top of that. One patch that fixes drawing box characters (UTF-8 to CP437) had an alternative merged, another hack we have to fix regulator issues is no longer neccessary as the issue is fixed, and my QEMU patches were merged upstream, so drop these patches. One patch we have to disable binman images can be replaced by a simpler alternative so drop it too. Upstream kconfig status is still unstable, so updating configs with `make oldconfig` would miss important upstream changes, since they rely on carrying defaults via upstream defconfigs. Update the configs as such, like before: - Turn old configs into defconfigs (./update trees -s u-boot) - Save the diff from old upstream defconfig (diffconfig $theirs $ours) - Update U-Boot revision, rebase patches, and clean old trees - Prepare new U-Boot tree (./update trees -f u-boot) - Review the diffconfigs to see if any options were renamed upstream - Copy over the new upstream defconfigs and apply earlier diff - Turn new defconfigs into configs (./update trees -l u-boot) Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
2024-07-11trees: fix bad comparison in configure_project()Leah Rowe
we used to set cmd only to these values: build_project build_targets however, now we set them to: build_project build_targets $@ the latter cannot be measured reliably, but we were checking whether cmd equalled: build_targets now we instead check that it does not equal: build_project Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-11trees: don't continue if no argument givenLeah Rowe
main() used to be the only function executed from outside of main(), in this script, but now we source a config file and then run the build afterward. when a flag is provided without OPTARG, this means that we are continuing such action erroneously. to mitigate this, return 1 in that instance, and handle it in the line that calls main(), making it exit with zero status (success). Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10trees: general code cleanupLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10trees: merge build_targets() with handle_targets()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10trees: use wrapper for dry-running make commandsLeah Rowe
instead of using lots of if/else conditions, do that once and set a variable, dry, to : if not doing a dry run, the variable is empty. prefix this variable in places where you don't want a certain action to be performed, on dry runs. more specifically, : does *nothing* and always returns with zero status (success). this results in cleaner code, and a small sloccount reduction. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10trees: remove project-specific hacksLeah Rowe
move the coreboot-specific includes into mkhelper.cfg for that project. on some projects, we need variables from mkhelper.cfg to be global, so I was including serprog and coreboot mkhelper.cfg files in this script. instead, set a new variable "mkhelpercfg" pointing to the config file. if it doesn't exist, create and then point to a temporary (empty) mkhelper.cfg file. the rom.sh include has been moved to coreboot mkhelper.cfg The only remaining project-specific logic, in this trees script, is now the coreboot crossgcc handling, but this needs to be there as it's also used to build U-Boot. The way this now works, certain includes are done twice. For example, include/rom.sh will be included once globally, outside of main(), and then again in configure_project(). This means that certain functions will be defined twice. I'm uncertain if shell has anything equivalent to an ifdef guard as in C, but we actually want this here anyway, and it shouldn't cause any problems. It's a bit of a hack, but otherwise results in much cleaner code. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10remove executable permission on include/Leah Rowe
files under include/ should never be executed directly Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10rom.sh: actually say if a cbutil existsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10rom.sh: avoid re-building cbutils if builtLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10rom.sh: only before cbutils before corebootLeah Rowe
otherwise, due to the idiosyncratic nature of the coreboot build system, the coreboot.rom gets wiped out. cbutils is still handled by premake. ensure that payloads are only inserted just after running the coreboot make command. fixes a build issues introduced on 9020sff, previously unhandled. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-10trees: fix bad comparisonLeah Rowe
pro-tip: don't do this at 3AM do massive changes like this, no later than 1AM. the intent anyway is for -d to cause no build dependencies to be handled, but the current logic says to only handle them if -d is set! fix it by removing the ! part Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-09minor cleanupLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-07-09trees: support -d (dry run) for custom build logicLeah Rowe
-d does the same as -b, except for actually building anything! in effect, it does the same as -f (fetch) except that the resulting variable assignments will not be recursive (as with -f). if -d is passed, configuration is still loaded, defconfig files are still cycled through, and more importantly: helper functions are still processed. the grub, serprog and coreboot helper functions have been modified to return early (zero status) if -d is passed. this behaviour will be used to integrate vendor.sh logic in with the trees script, for cases where the user wants to only handle vendor files. e.g.: ./update trees -b coreboot x230_12mb this would download the files as usual, build coreboot, with those files, and then build the payloads. but: ./update trees -d coreboot x230_12mb this would download the files, NOT build coreboot, and NOT build the payloads. this change increases the sloccount a bit, but i'm relying on the fact that the vendor.sh script already re-implements config handling wastefully; the plan is to only use trees. for now, simply stub the same ./vendor download command. there is one additional benefit to doing it this way: this method is *per-kconfig* rather than per-target. this way, one kconfig might specify a given vendor file that is not specified in the other. although the stub still simply handles this per target, it's done in premake, which means that the given .config file has been copied. this means that when i properly re-integrate the logic into script/trees, i'll be able to go for it per-kconfig. the utils command has been removed, e.g. ./update trees -b coreboot utils default the equivalent is now: ./update trees -d coreboot default this would technically download vendor files, but here we are specifying a target for which no kconfigs exist; a check is also in place, to avoid running the vendor file download logic if tree==target the overall effect of this change is that the trees script no longer contains any project-specific logic, except for the crossgcc build logic. it does include some config/data mkhelper files at the top, for serprog and coreboot, so that those variables defined in those files can be global, but another solution to mitigate that will also be implemented in a future commit. the purpose of this and other revisions (in the final push to complete lbmk audit 6 / cbmk audit 2) is to generalise as much logic as possible, removing various ugly hacks. Signed-off-by: Leah Rowe <leah@libreboot.org>