summaryrefslogtreecommitdiff
path: root/include
AgeCommit message (Collapse)Author
2024-05-22git.sh: move link_crossgcc to end of fileLeah Rowe
link_crossgcc is called after git_am_patches, so put it after ward Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-22git.sh: move xgcc linking to a new functionLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-22git.sh: skip submodules if .gitmodules missingLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-22git.sh: merge patch_submodules in prep_submodulesLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-22git.sh: split submodule handling to new functionLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-22git.sh: remove errant line breakLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-22git.sh: remove another meaningless checkLeah Rowe
again, the directory in question is simply used in a for loop using asterisk (git_am_patches) and the for loop simply won't iterate if either the directory doesn't exist or it contains no items. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-22git.sh: shorter variable namesLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-22git.sh: remove meaningless checkLeah Rowe
in the function that immediately follows, it starts two for loops that check every item in that directory, using the asterisk wildcard. if the directory does not exist, then the for loop will simply break on first pass. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-22git.sh: remove variable not meaningfully usedLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-21git.sh: fix invalid command in git_prep()Leah Rowe
"./update project trees" is a leftover from the old build system design, prior to audits. this particular call is for when xtree is defined, which means that a given tree must rely on the given coreboot tree defined by xtree. the "xtree" tree is downloaded, so that its crossgcc builds can be re-used to save time when building targets across many trees. this is because trees often use identical crossgcc builds. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-20git.sh: allow patching submodulesLeah Rowe
for single-tree project (e.g. flashprog): config/submodule/PROJECT/MODNAME/patches for multi-tree project (e.g. coreboot): config/submodule/PROJECT/TREE/MODNAME/patches MODNAME is e.g.: 3rdparty/vboot directory in coreboot: would become vboot (the submodule codepath is filtered to up to the final slash) another example: submodire src dir 3rdparty/foo/bar MODNAME would be "bar" Add whatever patches you like to a given submodule. An example patch is included in this commit. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-19git.sh: don't delete .git if src/project/projectLeah Rowe
otherwise, "./update release" will epicly fail Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-19git.sh: remove fetch_from_upstream()Leah Rowe
the function is very small and only called once, from fetch_project_trees() merge it into fetch_project_trees() Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-19option.sh: don't return 1 in mkrom_tarballLeah Rowe
one of the calling functions relies on the return value to be always 0, so these error conditions in mkrom_tarball have been altered to cause an *exit* (non-zero) instead. in practise, the commands in question were printf commands run after tho directory they output to had been created, so write access would probably not be an issue. nonetheless, technically correct is the best kind of correct. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-19option.sh: mktar_release to mkrom_tarballLeah Rowe
that's all it's used for, to compress the rom images Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-18vendor.sh: further simplify config handlingLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-18vendor.sh: greatly simplified config handlingLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-18vendor.sh: move config checks to detect_firmwareLeah Rowe
they will later be removed, as part of unification against other files that perform the same tasks Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-18vendor.sh: print an error upon ill-defined targetLeah Rowe
previously, it was attempting to load the configs and silently failing. we must provide feedback to the user. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-18vendor.sh: remove redundant checkLeah Rowe
we literally do this check twice, in immediate succession. avoid wasted i/o Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-18vendor.sh: simplify defconfig checkLeah Rowe
re-use check_defconfig from option.sh Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-18git.sh: Remove .git if XBMK_RELEASE=yLeah Rowe
The build system already deletes .git in all source directories for each given release, but does so at the very end; it still does, but now it is deleted one by one per project, to save space during very large builds (release sizes vary wildly, depending on how many trees exist for coreboot basically). If you're building entirely in tmpfs (as I do), this could be a problem if you have lots of .git/ directories. This change reduces disk usage, or in the above example, memory usage when running the build system from tmpfs. This complements another recent change, where ROM images are compressed per target during release builds, rather than all at the very end of the process. It is part of a series of optimisations, to reduce the memory and disk usage of the build system, and to reduce I/O wastage in general. This change will not be the last of such changes! Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-16lbmk: allow easier sync with cbmkLeah Rowe
an equivalent change has been made in cbmk. certain lbmk-specific variable names have been made generic, with certain functions and other variables moved around. i maintain sync between libreboot and canoeboot, where both projects can have the same behaviours, and most of the merge conflicts have to do with variable names containing "LBMK", "lbmk", "cbmk" or "CBMK", or indeed "canoeboot" and "libreboot" LBMK/lbmk/CBMK/cbmk variables between canoeboot and libreboot now contain the string XBMK/xbmk it should now be *much* easier to merge build system changes between lbmk and cbmk. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15option.sh: delete check_git()Leah Rowe
it's only used from main() in the main build script, and it's very small, as is main() therefore, move the logic into main() Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15remove check_project() (always set variables)Leah Rowe
in lbmk, we call check_project() to set variables such as projectname, version, version date this is unnecessary, because all main scripts use this functionality anyway do it by default Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15build/roms: create full release tarball nameLeah Rowe
set relname from option.sh under check_project() now the release logic simply has to move a directory Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15option.sh: don't bother checking for GNU tarLeah Rowe
releases aren't reproducible anyway. we were using options available in gnu tar for this. it will be revisited at a later date. however, the next time this is done, we will use another method because there are in fact portable ways to create tarballs reproducibly, documented on reproducible-builds.org to be revisited, at a later date. for now, remove bloat. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15option.sh: remove insert_version_files()Leah Rowe
it's only called once, from this file, within a small function, and the function itself is very small. remove, and put the contents of the function in the calling function. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-15unified sha512sum creation for tarballsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-14move rom tarball creation to script/romsLeah Rowe
export LBMK_RELEASE="y" if this is done, the tarball is created instead of a directory, and the rom images are nuked using ./vendor inject with the nuke option, inserting the correct version files; the rom directory is deleted now the release script logic simple renames existing tarballs. the benefit of this change is fewer lines of code, and now lbmk doesn't use an insane amount of disk space when building a *lot* of release images (the uncompressed directories are deleted after each build) Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-11mrc.sh: remove redundant function extract_ref()Leah Rowe
extract_ref() is no longer used Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-11remove haswell mrc blob (libre raminit stable now)Leah Rowe
broadwell mrc is retained, because it's needed on 820 g2 it's no longer needed on haswell, because nri is stable. nri is short for "native ram initialisation", and libreboot provides this for: thinkpad t440p, thinkpad w541, dell optiplex 9020 mt, and dell optiplex 9020 sff remove, in line with libreboot's binary blob reduction policy previous revisions, prior to the recent release, stated that it would be retained for compatibility, but it's really not right to retain it, because doing so violates libreboot's policy the recent release excluded mrc-based rom images for haswell machines, providing only those rom images that use the libre raminit, while retaining support for mrc in the build system, so that users could still run the lbmk inject script on older release roms that use mrc again: libreboot's binary blob reduction policy is very clear: https://libreboot.org/news/policy.html it is a policy that can be summarised, thus: if a blob can be avoided, it must be avoided. therefore, we will avoid the Haswell MRC raminit blob Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-11remove all status checks. only handle release.Leah Rowe
the release variable is all we need, turning a target on or off for a given release. the status checks were prone to bugs, and unnecessary; it also broke certain benchmark scripts. it's better to keep the lbmk logic simpler. board status will be moved to the documentation instead. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-11git.sh: remove errant commentLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-11merge script/vendor/* into include/vendor.shLeah Rowe
stub it from the main build script the commands remain identical: ./vendor download arguments_here ./vendor inject arguments_here Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-06merge include/err.sh with include/option.shLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-06err.sh: correct copyright infoLeah Rowe
i replaced 2022, 2023 with 2022, 2024 when updating the years, as per modifications, but the 2023 copyright doesn't become invalidated change it to 2022-2024 instead, which is correct Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-05-03err.sh: update copyright infoLeah Rowe
i added a few changes during this year so far, 2024 update the copyright years Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-04-28build/roms: simplified status handlingLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-04-28option.sh: don't use nproc (not portable)Leah Rowe
export LBMK_THREADS=x where x is an integer. this is already supported for setting the number of build threads, but if not set it uses nproc. openbsd doesn't have nproc. default to 1 thread. now you MUST set threads. e.g. in linux do: export LBMK_THREADS=$(nproc) preliminary work is being done to make lbmk run on openbsd! Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-04-27allow disabling status checks during buildsLeah Rowe
export LBMK_STATUS=n if not set, the status checks and confirmation dialogs persist. if set to y they persist. if you set it to n, all checks are disabled, so e.g.: ./build roms all this would once again build all targets, regardless of status. this is if you want the old behaviour. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-04-26build/roms: report status when building imagesLeah Rowe
export LBMK_VERSION_TYPE=x x can be: stable, unstable in target.cfg files, specify: status=x x can be: stable, unstable, broken, untested if unset, lbmk defaults to "unknown" if LBMK_VERSION_TYPE is set, no confirmation is asked if the given target matches what's set (but what's set in that environmental variable can only be stable or unstable) if LBMK_RELEASE="y", no confirmation is asked, unless the target is something other than stable/unstable "unstable" means it works, but has a few non-breaking bugs, e.g. broken s3 on dell e6400 whereas, if raminit regularly fails or it is so absolutely unreliable as to be unusable, then the board should be declared "broken" untested means: it has not been tested With this change, it should now be easier to track whether a given board is tested, in preparation for releases. When working on trees/boards, status can be set for targets. Also: in the board directory, you can add a "warn.txt" file which will display a message. For example, if a board has a particular quirk to watch out for, write that there. The message will be printed during the build process, to stdout. If status is anything *other* than stable, or it is unstable but LBMK_VERSION_TYPE is not set to "unstable", and not building a release, a confirmation is passed. If the board is not specified as stable or unstable, during a release build, the build is skipped and the ROM is not provided in that release; this is in *addition* to release="n" or release="y" that can be set in target.cfg, which will skip the release build for that target if "n" Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-04-26exports variables from err.sh, not buildLeah Rowe
LC_COLLATE and LBMK_RELEASE are important variables. we want to make sure that these are seen by everything. since err.sh is included from all scripts, doing it there will accomplish just that. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-04-25lbmk: export TMPDIR from err.sh, not buildLeah Rowe
lbmk sets TMPDIR to /tmp, and then creates a tmpdir, then exports *that* as the value of TMPDIR. this unified TMPDIR location then contains all subsequent files and directories, when any script or program makes use of /tmp, via mktemp. at least, that's the theory! in practise, because it was only being properly exported from the main build scripts, subscripts that are then called were not exporting it, at least that is my assumption because in some cases, i found that the coreboot build system was leaving errant files behind outside of our own TMPDIR, and that build system did not seem to be setting TMPDIR itself; more debugging is needed. anyway: use the exact same logic, but do it from err.sh. since err.sh is included from every lbmk script, that means it will always be exported when running every single part of lbmk. this should reduce the chance that mktemp creates files and directories outside of our custom TMPDIR location. this is because in lbmk, we mitigate unhandled tmpdirs/files by unifying it in the manner described, then deleting the entire TMPDIR on exit from the main lbmk parent process (the main script that the user called from, which is always the "build" file). in lbmk, effort is made to clean up temporary files properly, without relying on this catch-all, but we can't rely on that. the catch-all should also be as robust as possible. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-04-24haswell: only provide NRI-based ROMs in releasesLeah Rowe
release="n" is set in target.cfg on haswell build targets that use mrc.bin script/update/release exports LBMK_RELEASE="y" script/build/roms skips building a given target if release="n" in target.cfg *and* LBMK_RELEASE="y" you could also do the export yourself before running ./build roms, for example: export LBMK_RELEASE="y" ./build roms all This would skip these ROM images. The native haswell raminit is now stable enough in my testing, that I wish to delete the MRC-based targets. This is in line with Libreboot's Binary Blob Reduction Policy, which states: if a blob can be avoided, it should be avoided. The problem is that users often run the inject script in *lbmk* from Git, instead of from the src release archive. I forsee some users running this on modern lbmk with older release images. If the mrc-based target isn't there, the user may use an NRI-based target name, and think it works; they will insert without MRC. I foresaw this ages ago, which is why Caleb and I ensured that the script checks hashes, and hashes are included in releases. Therefore: for the time being, keep the MRC-based configs in lbmk but do not include images for them in releases. This can be done indefinitely, but I'll probably remove those configs entirely at some point. On the following boards, Libreboot now will *only* provide NRI-based ROM images for the following machines: * Dell OptiPlex 9020 SFF * Dell OptiPlex 9020 MT * Lenovo ThinkPad T440p * Lenovo ThinkPad W541/W540 I now recommend exclusive use of NRI-based images, on Haswell hardware. It's stable enough in my testing, and now supports S3. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-03-27allow users to specify number of build threadsLeah Rowe
lbmk otherwise uses nproc to set the number of build threads, in these places: * generic make commands in script/update/trees * crossgcc make command in script/update/trees the -T0 option is also used in script/update/release, when running tar. with this change, you can do: export LBMK_THREADS=x where x is the number of threads. when you then run lbmk, your chosen number of threads will override the default. this may be useful on a host that does not have a lot of memory. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-03-27safer, simpler error handling in lbmkLeah Rowe
in shell scripts, a function named the same as a program included in the $PATH will override that program. for example, you could make a function called ls() and this would override the standand "ls". in lbmk, a part of it was first trying to run the "fail" command, deferring to "err", because some scripts call fail() which does some minor cleanup before calling err. in most cases, fail() is not defined, and it's possible that the user could have a program called "fail" in their $PATH, the behaviour of which we could not determine, and it could have disastrous effects. lbmk error handling has been re-engineered in such a way that the err function is defined in a variable, which defaults to err_ which calls err_, so defined under include/err.sh. in functions that require cleanup prior to error handling, a fail() function is still defined, and err is overridden, thus: err="fail" this change has made xx_() obsolete, so now only x_ is used. the x_ function is a wrapper that can be used to run a command and exit with non-zero status (from lbmk) if the command fails. the xx_ command did the same thing, but called fail() which would have called err(); now everything is $err example: rm -f "$filename" || err "could not delete file" this would now be: rm -f "$filename" || $err "could not delete file" overriding of err= must be done *after* including err.sh. for example: err="fail" . "include/err.sh" ^ this is wrong. instead, one must do: . "include/err.sh" err="fail" this is because err is set as a global variable under err.sh the new error handling is much cleaner, and safer. it also reduces the chance of mistakes such as: calling err when you meant to call fail. this is because the standard way is now to call $err, so you set err="fail" at the top of the script and all is well. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-01-26git.sh: also reset xtree/tree_depend hereLeah Rowe
it's still necessary here, to prevent the same bug identified in the previous patch. Signed-off-by: Leah Rowe <leah@libreboot.org>
2024-01-21git.sh: fix bad call to ./updateLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>