summaryrefslogtreecommitdiff
path: root/resources/scripts/build/boot/roms_helper
AgeCommit message (Collapse)Author
2023-08-27move resources/scripts/ to script/Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-27handle/config/file: rename to handle/make/configLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-27build/src/for: rename to handle/make/fileLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-27scripts: put quotes around file/directory namesLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-26much, much stricter, more verbose error handlingLeah Rowe
lbmk is much more likely to crash now, in error conditions, which is a boon for further auditing. also: in "fetch", remove the downloaded program if fail() was called. this would also be done for gnulib, when downloading grub, but done in such a way that gnulib goes first. where calls to err write "ERROR" in the string, they no longer say "ERROR" because the "err" function itself now does that automatically. also: listmodes/listoptions (in "lbmk") now reports an error if no scripts and/or directories are found. also: where a warning is given, but not an error, i've gone through in some places and redirected the output to stderr, not stdout as part of error checks: running anything as root, except for the "./build dependencies *" commands, is no longer permitted and lbmk will throw an error mrc downloads: debugfs output no longer redirected to /dev/null, and stderr no longer redirected to stdout. everything is verbose. certain non-error states are also more verbose. for example, patch_rom in blobs/inject will now state when injection succeeds certain actual errors(bugs) were fixed: for example, build/release/roms now correctly prepares the blobs hash files for a given target, containing only the files and checksums in the list. Previously, a printf message was included. Now, with this new code: blobutil/inject rightly verifies hashes. doing all of this in one giant patch is cleaner than 100 patches changing each file. even this is yet part of a much larger audit going on in the Libreboot project. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-23scripts: never exit 1, always call err insteadLeah Rowe
this same change has been applied, selectively, to certain return statements. the general rule is this: the return statement should only be used to direct logic within a script, where certain non-errors states are used to skip certain actions; the exit command should *never* be used to return non-zero, except by err(). in so doing, we ensure easier debugging of the build system also: strip_rom_image in build/release/roms was running "continue" when a rom file didn't exist, despite not being a while/for loop. i make it return (non-error condition) instead it's ok for a script to exit 0, where appropriate, but perhaps a function could also be written for it Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-23unify err functions across scriptsLeah Rowe
include/err.sh this new handling also does mundane things, such as tell you what script b0rked Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-22memtest86+ v6.20Riku Viitanen
uses 32-bit variant for x86_32 arch. 64-bit for x86_64. resources/scripts/build/src/for: modified it a bit. when building e.g. "memtest86plus/build32" it correctly fetches "memtest86plus" instead. but builds memtest86plus/build32, which is inside that git repo Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-08-21general code cleanup on lbmk shell scriptsLeah Rowe
in update/blobs/download, i saw instances where appdir was being deleted with rm -r, but the more appropriate command would rm -Rf. this is now fixed. other than that, i've mostly just simplified a bunch of if statements and consolidated some duplicated logic (e.g. if/else block for dependencies in build_dependencies() of update/blobs/download one or two functions and/or variables have been renamed, for greater clarity in the code, also removed a few messages that were redundant used printf instead of echo, in a few places, also fixed up the indentation in a few places Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-21unify build/defconfig and modify/defconfigLeah Rowe
they fundamentally perform the same action: copy the .config file and run make, but build runs make-all, while modify runs make-oldconfig or make-menuconfig merge this functionality together also: ./handle config file ^ this is the new syntax, not: ./build defconfig for for example: ./handle config file -b coreboot x200_8mb <-- build x200 rom ./handle config file -m coreboot x200_8mb <-- modify configs ./handle config file -u coreboot x200_8mb <-- make-oldconfig ./handle config file -u seabios ./handle config file -b u-boot yes, 1 script and a sloccount reduction of 52. and the audit? it continues. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-20unify build/clean into ./build release srcLeah Rowe
handle it all in the 1 script quite a few clean scripts are still present, so resources/scripts/build/clean/ still exists. 23 sloc reduction. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-20unify most module build scriptsLeah Rowe
Some of them weren't even used at all, such as the flashrom build script. the bios_extract build script existed but was never used, because we only called (from blobutil) a python script from in there, without actually compiling anything! resources/script/build/src/for Usage, e.g.: ./build src for memtest86plus It also handles fetch. This script is intended largely for those codebases that are quite simple, requiring trivial or no intervention besides running "make". 37 sloc reduction. Not a lot, but the audit continues! These optimisations add up. I started at 3300 sloc in resources/scripts and me target is 2k (2000) sloc. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-20unify grub scripts under one directoryLeah Rowe
new commands are thus, build grub payloads: ./build grub payload (formerly ./build payload grub) build grub utils: ./build grub utils (formerly ./build module grub) The scripts is build/module/ will mostly be deleted. I say mostly, because some of them are being moved instead. The deleted ones will be ones that basically just run "make" in the target directory. They will be unified, in a follow-up patch. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-19consolidate u-boot/seabios/coreboot build scriptsLeah Rowe
See file: resources/scripts/build/defconfig/for It is based on: resources/scripts/build/payload/u-boot The u-boot payload script has been deleted, as has the seabios payload script; the build/boot/roms logic has been heavily simplified too, by removing the logic for building of elf files based on defconfig. SeaBIOS, U-Boot and coreboot all use defconfig-type infrastructure for their build systems, and they are fundamentally the *same* in how to compile each codebase, at least in an lbmk context, regardless of actual (and very huge) differences in these codebases. Several hundred sources-lines of code have been eliminated by this change, drastically simplifying everything; U-Boot payload compiling also now errors out when a single build fails, instead of continuing. Also: build/boot/roms no longer re-compiles a coreboot target that was already compiled, which is the same behaviour observed for payloads. (this means you must now manually delete a target, when you wish to re-build it; the build/boot/roms logic now more or less just runs cbfstool; blobutil is handled from build/defconfig/for) ALSO: Since crossgcc is now handled by build/defconfig/for, not build/boot/roms, standalone compiling of u-boot is now possible. This has been tested. You compile it like so: ./build defconfig for u-boot or specific trees, e.g. ./build defconfig for u-boot default One other consequence of this patch is that re-building the same ROM image is now much faster, because the same builds are re-used unless deleted. This could be useful when testing grub.cfg changes, for example, if that's all you change. With things like ccache used (not yet used robustly in lbmk), this could speed things up more, depending on the codebase. This patch demonstrates the raw power of lbmk; it is a very simple and highly efficient build system, and now much more so! Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-16merge coreboot/u-boot download logic to one scriptLeah Rowe
they are fundamentally the same, in an lbmk context. they are downloaded in the same way, and compiled in the same way! (Kconfig infrastructure, board-specific code, the way submodules are used in git, etc) ~200 sloc reduction in resources/scripts the audit begins Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-07-29scripts: fix indentation in switch/case blocksLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-07-17Revert "build/boot/roms mkUBootRoms: initialise variables"Leah Rowe
This reverts commit 7c90a4077fa1038394e3d470268ea6e509c2d734. causes another build bug. i'm helping someone with the bug now, i think the workaround for now would be to just use bash, on this script. until i can figure something better out.
2023-07-17build/boot/roms mkUBootRoms: initialise variablesLeah Rowe
they were outside the scope, outside of the if statements. in some shells, this is ok. we use "sh" so the user could have any shell. be a bit nicer to the more asininely technically correct sh implementations out there Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-07-16coreboot: re-add asus kgpe-d16/kcma-d8/kfsn4-dreLeah Rowe
Libreboot 20220710 was the last release to support these boards. I plan to eventually port code differences between D8/D16 to Dasharo, for KCMA-D8 support in Dasharo, to then use in Libreboot for both KCMA-D8 and KGPE-D16, but I have no plans to update the KFSN4-DRE code, at least for now. Libreboot 20220710 used coreboot 4.11, whereas this patch makes use of coreboot 4.11_branch; the crossgcc toolchains no longer compile on modern distros, so I spent time patching those (tested in Debian Sid, will also work on Arch Linux and so on). The acpica downloads now fail, in 4.11_branch, because Intel made some changes upstream for these tarball downloads. Newer coreboot works around this by grabbing tarballs from github, itself a non-ideal solution, but I digress; this patch changes coreboot crossgcc (in 4.11_branch) to download the acpica tarball from libreboot rsync, where I've added it. This patch also re-introduces the PIKE2008 fix, where empty option ROMs for these are inserted into CBFS. This prevents SeaBIOS from loading the real option ROMs, which would cause SeaBIOS to hang. This means that SAS drives are not supported in SeaBIOS, for these boards in Libreboot. I previously said, in the Censored Libreboot c20230710 announcement, that I would *only* merge D8/D16 when I've added Dasharo support to Libreboot, and use that, but the work to make coreboot 4.11_branch compile is something I'm quite proud of and I see no reason to exclude from lbmk master branch. Honestly, there's not much different than 4.11, code-wise. I *probably* won't use 4.11_branch for the next Libreboot release, on D8/D16. By then, I might have Dasharo integrated in lbmk instead. We shall see. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-07-08build/boot/roms: fix coreboot-version in releasesLeah Rowe
This error was observed, in the coreboot build system: In file included from src/lib/version.c:4: build/build.h:10:32: error: 'libreboot' undeclared here (not in a function) 10 | #define COREBOOT_MAJOR_VERSION libreboot-20230625 | ^~~~~~~~~ src/lib/version.c:35:46: note: in expansion of macro 'COREBOOT_MAJOR_VERSION' 35 | const unsigned int coreboot_major_revision = COREBOOT_MAJOR_VERSION; | ^~~~~~~~~~~~~~~~~~~~~~ This happened on the 20230625 *release archive*, when a user tried to build for W541 MRC on an Arch Linux container. This change fixes the error. I never got the error on my end when build testing the release archives, but this will prevent the error. Fix it by only inserting libreboot version string YYYYMMDD representing the Libreboot version. (libreboot uses ISO dates as version numbers) Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-06-25build/roms_helper: reset d521fca7, backport fixesLeah Rowe
I keep getting random linker issues when running: ./build boot roms all I think the issue lies somewhere in here, from when I did that massive audit. So I'm undoing the audit which mostly re-factored the code style here. These changes are being backported: f338697b build/boot/roms: Support removing microcode 941fbcb run coreboot utils from own directory f256ce98 build/boot/roms: say board name on stderr I removed this change: 6d6bd5ee (the script now uses dedicated utils directory) additionally: cbutils is built much earlier on in the script, first thing after initialising variables the other changes not backported are all code style changes, and I believe these are responsible. if no other fixes occur to this fire before the next libreboot release, then my hunch was right. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-06-25build/boot/roms: say board name on stderrLeah Rowe
That way, I can more easily debug build issues with specific boards, e.g. ./build boot roms all 2>lbmk.err.log Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-06-24build/roms: distclean coreboot before each buildLeah Rowe
don't clean it, distclean it Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-06-24run coreboot utils from own directoryLeah Rowe
this means coreboot can now be distcleaned safely, before and after each build of a rom image Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-06-20build/boot/roms_helper nicer indent on switch loopLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-06-19build/boot/roms: Support removing microcodeLeah Rowe
From now on, the following rules are available for all mainboards, in resources/coreboot/boardname/board.cfg: * blobs_required="n" or "y" * microcode_required="n" or "y" The blobs setting, if set to "n", simply renames filename.rom to filename_noblobs.rom. The microcode setting, if set to "n", copies the ROM (with or without _noblobs) to filename_nomicrocode.rom (if blobs="n", it would be filename_noblobs_nomicrocode.rom). Where "nomicrocode" is set, ROMs with microcode will still be provided by lbmk and in relesase, but ROMs will also be provided alongside it that lacks any microcode updates. If the *original* ROM already lacks microcode updates, then the original ROM will be *renamed* to include "nomicrocode" in the name. This is done on images for ARM platforms, for instance, where microcode is never used whatsoever. Example filenames now generated: seabios_e6400_4mb_libgfxinit_corebootfb_noblobs_nomicrocode.rom seabios_e6400_4mb_libgfxinit_corebootfb_noblobs.rom seabios_withgrub_hp8300usdt_16mb_libgfxinit_corebootfb_colemak_nomicrocode.rom seabios_withgrub_hp8300usdt_16mb_libgfxinit_corebootfb_colemak.rom uboot_payload_gru_kevin_libgfxinit_corebootfb_noblobs_nomicrocode.rom A vocal minority of people were not happy with some of the changes made in Libreboot last year, including on existing supported hardware from before those changes were made. I did this before the last release, out of respect: https://libreboot.org/news/gm45microcode.html (re-add mitigations for no-microcode setup on GM45) This new change is done as an further, extended courtesy. Tested and works fine. (testing using cbfstool-print) Actual Libreboot policy about binary blobs is nuanced. See: https://libreboot.org/news/policy.html (reduction policy) and: https://libreboot.org/freedom-status.html (implementation) Well, the status page talks about descriptor vs non-descriptor on Intel platforms, and where me_cleaner is used (on platforms that need Intel ME firmware), it regards the descriptored setups to be blob-free if coreboot does not require binary blobs. In this paradigm, microcode updates are not considered to be binary blobs, because they aren't technically software, they're more like config files that just turn certain features on or off within the CPU. However, for lbmk purposes, "noblobs" means that, after the ROM is fully ready to flash on the chip, there will be no blobs in it (except microcode). So for example, an X200 that does not require ME firmware is considered blob-free under this paradigm, even though Libreboot policy regards X230 as equally libre when me_cleaner is used; in this setup, ROMs will not contain "blobfree" in the filename, for X230 (as one example). Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-06-13Revert "Remove most of Ferass's lbmk contributions"Leah Rowe
This reverts commit a4ea2867319471d9fe7d4ee540881e0286b4d3cf. The licensing audit has been abandoned. I will not be re-licensing in bulk to MIT. I can still use MIT license on new works, e.g. utilities, but there's really no pressing need to re-license lbmk. It's just shell scripts, and most of what it interacts with (coreboot, grub, seabios) is GPL anyway. So who cares? Ferass's patch was removed due to refusal to re-license, but the decision to re-license has been canceled. I'm now aiming for a quick stable release.
2023-05-27Remove most of Ferass's lbmk contributionsLeah Rowe
The primary purpose of my intense auditing has been to improve lbmk's coding style and fix bugs but there is a secondary purpose: know precisely who owns what, because I want to re-license as much as possible of lbmk under *MIT*, instead of the current GNU licensing. MIT is vastly superior, because it grants *actual* freedom to the user, permits *sublicensing* and it is vastly more compatible with other GPL combinations; for example, MIT license is compatible with GPL2-only whereas lbmk's current mix of GPLv3-or-later and GPLv3-only is legally incompatible with GPLv2-only. Re-licensing under MIT will most likely result in more contributions to Libreboot's build system in the future, especially as it will attract a lot more commercial interest. Contrary to the popular arguments, copyleft is a liability to the free software movement and results in less code being written; in practise, permissively licensed code gets more public contributions, including from commercial entities, even if companies can theoretically make something proprietary out of it (in practise, anyone inclined can just use the upstream and proprietary forks almost always die). Copyleft propaganda is fundamentally flawed. See: <https://unixsheikh.com/articles/the-problems-with-the-gpl.html> Anyway, I've been doing a combination of: * Seeking permission from other copyright holders, for re-licensing * Deleting, or moving, other contributions; for example, splitting certain contributions into separate files so that originally modified files become unencumbered. This latter solution is a result of *code cleanup* arising from the audit. For Ferass's contributions, I opted to seek *permission*, and permission was denied. In full compliance with this legal imperative, I'm acting accordingly; this commit removes all of Ferass's changes that converted lbmk to posix shell scripts, thus removing his copyright on the affected files, bypassing his authority entirely. Therefore, lbmk is largely now bash-dependent. In practise, nobody is going to use anything other than a GNU system to build Libreboot, because many projects that Libreboot makes use of rely heavily on GNU; for example, coreboot's build system makes heavy use of GNU-specific extensions in *GNU Make*, and likely contains many bashisms. Of course, Libreboot also compiles GNU GRUB. I would much rather have MIT-licensed Bash scripts than GPL-licensed posix SCL scripts. This reverts the changes from Ferass El Hafidi, for the following commits, with some exceptions: * 7f5dfebf7d37c56d9c7993aaa17c59070cb5aec9 * f787044642236917c9c4dbcaa48a6b0648097db0 Exception: download/mrc not reverted, because that was already a fork of an existing script under coreboot's build system, and their script was GPLv2. i cannot/will not re-license this file (ergo, 7f5dfebf7d37c56d9c7993aaa17c59070cb5aec9 change remains intact, on this file) resources/scripts/build/boot/roms_helper, these changes have been kept: * 7e6691e9 - Add ARMv7 and AArch64 support * dec2d720 - add myself in the build/roms_helper script (added 2021 copyright for the change below) * b7405656 - Workaround for grub's slow boot ^ these changes will be re-factored, splitting them out of the file into a new file. This will be done in a future lbmk revision. (in some cases, it makes sense to keep a change but split it, allowing the main file to be re-licensed without the change in it) This is part of a much larger series of licensing audits. It's likely that lbmk will be posix-compliant (in its shell scripts) again some day, because I'm planning to rewrite most of these scripts (the ones modified in this patch), and many of them (e.g. individual download scripts) are subject to future deletion in a planned overhaul of the download logic for third party projects. In addition: these changes are being kept (no attempt to re-license them will be made): * cff081c6 - Fix grub's slow boot (1 year, 5 months ago) <Vitali64> * 4c851889 - Add macbook*1 16mb configs (1 year, 6 months ago) <Vitali64> Ferass's work that remains will be split into dedicated files containing them, where feasible. In the case of grub.cfg (for GNU GRUB), I don't care because it's a script for an engine (GRUB shell) that's under GPL anyway, so who really cares about MIT license. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-05-13build/build/roms: simplify mkCoreboot() argumentsLeah Rowe
2023-05-13build/boot/roms: don't use subshells frivilouslyLeah Rowe
use make -BC instead of cd
2023-05-13build/boot/roms: remove errant debug lineLeah Rowe
i added this in the last revision it was put there to debug something that i fixed before pushing
2023-05-13build/boot/roms: simplify build_rom_images()Leah Rowe
2023-05-13build/boot/roms: use fast dd command for ich9m ifdLeah Rowe
bs 12k and count 1, rather than bs 1 and count 12k
2023-05-13build/boot/roms: don't run ich9gen twiceLeah Rowe
2023-05-13build/boot/roms: simplify moverom()Leah Rowe
2023-05-13build/boot/roms: remove unused legacy codeLeah Rowe
this cuttype is no longer used lbmk creates truncated me setups now, on ifd platforms
2023-05-13build/boot/roms: reduced code indentationLeah Rowe
2023-05-12build/boot/roms: split main() to topdown functionsLeah Rowe
the logic can now more or less be read chronologically
2023-05-12build/roms_helper: move logic into main()Leah Rowe
logic will be split from main into smaller functions, in follow-up commits
2023-05-10build/boot/roms_helper: further cleanupLeah Rowe
consolidated some for loops removed errant code
2023-05-10build/roms: general code style cleanupLeah Rowe
2023-05-10build/roms: fix faulty keymap list expansionLeah Rowe
2023-05-10build/boot/roms*: RFC 2646 complianceLeah Rowe
No more than 80 characters per line.
2023-05-09seabios: do normal config, disable oprom in vgaromLeah Rowe
previously, "normal" initmode relied on the vgarom-based seabios config, which enables option roms, but then lbmk would insert pci-optionrom-exec 0 for vgarom, and 2 for normal in libreboot, coreboot roms with "vgarom" in the filename do pci option rom execution from coreboot, and "normal" roms do execution from seabios(where seabios is the only payload provided on normal setups) this is because payloads like grub can also be used, on vgarom setups, where coreboot must handle oprom execution
2023-02-19build/boot/roms: fail when build cbutils failsLeah Rowe
2022-12-27Do not rely on bashisms and behaviour undefined by the POSIX specification.Ferass 'Vitali64' EL HAFIDI
By making lbmk fully POSIX-compliant, it will be easier to port lbmk to other systems implementing POSIX such as Alpine Linux and FreeBSD. Signed-off-by: Ferass 'Vitali64' EL HAFIDI <vitali64pmemail@protonmail.com>
2022-12-11build/boot roms: add exits for failing commandsLeah Rowe
2022-12-10build/roms: Support using "u-boot" ELF file as U-Boot payloadAlper Nebi Yasak
U-Boot runtime configuration is done with a device-tree file, which is built alongside the executable in the upstream build system, and must be available to U-Boot at runtime. This device-tree is normally not linked into the default "u-boot" ELF file. So far we have been handling it by re-creating a "u-boot.elf" from the raw binary parts by setting REMAKE_ELF, and using that as the coreboot payload. Unfortunately, that fails to build for x86 boards, more specificly the "coreboot" boards upstream. It's also possible (but discouraged) to set OF_EMBED to embed the device-tree file into the U-Boot itself, in which case we could use the "u-boot" file as the payload on the "coreboot" boards. Add support for using the "u-boot" file as the payload if "u-boot.elf" doesn't exist. Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
2022-12-10build/roms: Don't rebuild crossgcc if it was already builtAlper Nebi Yasak
The roms_helper script skips building crossgcc-i386 if its target directory exists. Skip it for other architectures as well. Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
2022-12-10build/roms: Make coreboot crossgcc usable for payloads and modulesAlper Nebi Yasak
Add the coreboot-built cross-architecture toolchains to the PATH so that modules and payloads can use them. When building for a foreign-arch board, also export CROSS_COMPILE pointing to the appropriate prefix. Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>