summaryrefslogtreecommitdiff
path: root/resources/scripts/update/blobs
AgeCommit message (Collapse)Author
2023-08-27scripts: put quotes around file/directory namesLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-26much, much stricter, more verbose error handlingLeah Rowe
lbmk is much more likely to crash now, in error conditions, which is a boon for further auditing. also: in "fetch", remove the downloaded program if fail() was called. this would also be done for gnulib, when downloading grub, but done in such a way that gnulib goes first. where calls to err write "ERROR" in the string, they no longer say "ERROR" because the "err" function itself now does that automatically. also: listmodes/listoptions (in "lbmk") now reports an error if no scripts and/or directories are found. also: where a warning is given, but not an error, i've gone through in some places and redirected the output to stderr, not stdout as part of error checks: running anything as root, except for the "./build dependencies *" commands, is no longer permitted and lbmk will throw an error mrc downloads: debugfs output no longer redirected to /dev/null, and stderr no longer redirected to stdout. everything is verbose. certain non-error states are also more verbose. for example, patch_rom in blobs/inject will now state when injection succeeds certain actual errors(bugs) were fixed: for example, build/release/roms now correctly prepares the blobs hash files for a given target, containing only the files and checksums in the list. Previously, a printf message was included. Now, with this new code: blobutil/inject rightly verifies hashes. doing all of this in one giant patch is cleaner than 100 patches changing each file. even this is yet part of a much larger audit going on in the Libreboot project. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-23scripts: never exit 1, always call err insteadLeah Rowe
this same change has been applied, selectively, to certain return statements. the general rule is this: the return statement should only be used to direct logic within a script, where certain non-errors states are used to skip certain actions; the exit command should *never* be used to return non-zero, except by err(). in so doing, we ensure easier debugging of the build system also: strip_rom_image in build/release/roms was running "continue" when a rom file didn't exist, despite not being a while/for loop. i make it return (non-error condition) instead it's ok for a script to exit 0, where appropriate, but perhaps a function could also be written for it Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-23blobs/download: copy dl_path, don't move itLeah Rowe
moving it defeats the purpose of the caching mechanism that's in place. this should avoid unnecessary downloads Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-23unify err functions across scriptsLeah Rowe
include/err.sh this new handling also does mundane things, such as tell you what script b0rked Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-21general code cleanup on lbmk shell scriptsLeah Rowe
in update/blobs/download, i saw instances where appdir was being deleted with rm -r, but the more appropriate command would rm -Rf. this is now fixed. other than that, i've mostly just simplified a bunch of if statements and consolidated some duplicated logic (e.g. if/else block for dependencies in build_dependencies() of update/blobs/download one or two functions and/or variables have been renamed, for greater clarity in the code, also removed a few messages that were redundant used printf instead of echo, in a few places, also fixed up the indentation in a few places Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-20unify build/clean into ./build release srcLeah Rowe
handle it all in the 1 script quite a few clean scripts are still present, so resources/scripts/build/clean/ still exists. 23 sloc reduction. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-20unify most module build scriptsLeah Rowe
Some of them weren't even used at all, such as the flashrom build script. the bios_extract build script existed but was never used, because we only called (from blobutil) a python script from in there, without actually compiling anything! resources/script/build/src/for Usage, e.g.: ./build src for memtest86plus It also handles fetch. This script is intended largely for those codebases that are quite simple, requiring trivial or no intervention besides running "make". 37 sloc reduction. Not a lot, but the audit continues! These optimisations add up. I started at 3300 sloc in resources/scripts and me target is 2k (2000) sloc. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-17don't call blobutil directly from lbmkLeah Rowe
it's bloat, and was only there for backwards compatibility with the old commands, but the new commands are e.g. ./update blobs inject instead of: ./blobutil inject this results in a slight code size reduction in lbmk Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-17remove download scripts, consolidate into scriptLeah Rowe
most of them were just calling the gitclone script, so remove them. the grub script was treating gnulib as a dependency. i've now added the ability to grab 1 dependency, in the gitclone script (it should be expanded later to support multiple dependencies) the gitclone script has been renamed to "fetch". the "fetch_trees" script does more or less the same thing, but calls "fetch" and handles multiple revisions if a project needs that this is more efficient, and slightly reduces the code size of lbmk! Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-17improve user feedback in blobutilLeah Rowe
make it output messages that tell the user important information. it's only subtle but it makes a difference to some people, who need confirmation. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-16merge coreboot/u-boot download logic to one scriptLeah Rowe
they are fundamentally the same, in an lbmk context. they are downloaded in the same way, and compiled in the same way! (Kconfig infrastructure, board-specific code, the way submodules are used in git, etc) ~200 sloc reduction in resources/scripts the audit begins Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-14NEW BOARD: HP EliteBook 8470p (Intel GPU)Leah Rowe
Intel GPU! The AMD ones will be tested, but assume Intel-only for now. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-14NEW BOARD: Dell Precision T1650Leah Rowe
Very nice ivybridge board that supports ECC RAM. NOTE: I couldn't get onboard graphics working yet, but this was confirmed working with a graphics card (in my case nvidia quadra k420) booted in text mode on the SeaBIOS payload. The GRUB payload also works, when loaded from SeaBIOS. Therefore, this is a SeaBIOS-only board (as far as first payload is concerned), but you can pick GRUB from the menu. You could make it "GRUB-only" in practise by setting SeaBIOS boot order to only load GRUB, and disable the SeaBIOS menu. We refer to this as "SeaGRUB". I've made lbmk use biosutilities and uefiextract, to get at the SMSC SCH5545 Environmental Control (EC) firmware. This firmware is needed for fan control. This is automatically downloaded and extracted, from Dell UEFI firmware updates. As with other blobs such as Intel ME, this firmware is then scrubbed by the release build scripts. The blobutil "inject" script can be used to re-insert it. Of note: there is no fixed offset, but no other blobs to be inserted in CBFS either, so the offset when re-inserting on release ROMs should still be the same, and thus the ROM checksums should match, when running blobutil inject. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-09NEW BOARD: HP EliteBook 2170pLeah Rowe
Another ivybridge platform, added in coreboot recently. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-05blobs/download: save ME file to correct locationLeah Rowe
blobdir is incorrect, and it means that the directory will appear under blobs/, in this case. this was an oversight on my part. this behaviour did not break anything in practise, but this patch makes the behaviour more consistent with rules. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-05blobs/download: try backup if bad hash on mainLeah Rowe
At present, the logic only tries backup URLs when an actual download fails (bad internet connection or the server is down). If the main download succeeds, but it has a bad checksum, the backup download is not attempted. Since wrongly hashed files are to be assumed useless, we may aswell delete and try the next file. This will guard against the possibility of a vendor changing their file, without changing the file name (non-versioned files, for example, may be subject to such changes). Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-08-05blobs/download: support more formats on ME extractLeah Rowe
ME extraction didn't support unar (RAR format), for regular extraction, after downloading a vendor file. For bruteforce ME extraction, after extracting a vendor archive, unar(RAR) and inno(innoextract) was not supported. This patch fixes both issues. It should be noted that as of now, the unar method has only been tested with certain HP vendor updates, and it's currently not used on any of those. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-07-29blobs/download: declare full user agentLeah Rowe
I messed up the string, when I first did this. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-07-29blobs/download: properly handle backup/main urlLeah Rowe
Immediately after the last revision, which was a hacky workaround to the problem, I realised the actual problem, and the real solution: In the switch block, check *backup* first. Then it breaks, continuing on the iteration. If it's variable for a main URL, it'll reliably go to the next check in the block, whereas if it's backup, it'll default to the first one in each case. This bug has been annoying the sh*t out of me for ages, and I've finally nailed it. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-07-29blobs/download: don't download backup on mainLeah Rowe
The script was actually downloading the backup, at all times, for each given URL. The way we handle this is quite buggy. This patch is a workaround, a dirty hack in fact, but it will do for now, because our backup URLs are always wayback links where the original URL (matching the correct main URL in the sources file) is always present, in the URL. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-07-29blobs/download: set common user agent stringLeah Rowe
Make it look like a normal web browser, downloading files. Some HTTP servers might block Wget unless this is done. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-07-29blobs/download: simplify for loopLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-07-29scripts: fix indentation in switch/case blocksLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-07-27blobutil/ec: abort if kbc1126 ec extraction failsLeah Rowe
This was an oversight on my part. Should extraction fail, we must abort. This is in preparation for addition of future mainboards, where further tweaking is required in blobutil. This error check will warn us about it. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-06-24run coreboot utils from own directoryLeah Rowe
this means coreboot can now be distcleaned safely, before and after each build of a rom image Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-06-20blobs/download: exit if no board configs foundLeah Rowe
fixes ./build boot roms all in detect_firmware(), "set" is used to get values from configs, to know if things like ME/MRC are needed on some "board" configs under resources/coreboot/, no actual coreboot configs are provided, because they are used as a reference (coreboot revision, tree name etc) for actual boards, with actual coreboot configs when attempting to build for such a board, running "set" on such non-existent files would cause a non-zero exit, when we want zero. the non-zero exit then caused the build/boot/roms command to fail, when running "all" if it found, for example, resources/coreboot/cros/ which has the above problem, in this context work around it by verifying that coreboot configs exist for the given target name, in the blobutil download script. if no such configs exist, then exit zero (success) doing so is correct, because the script is intended to do just that, erroring only if it is detected that blobs are needed for a given board, but other errors occur; if no coreboot configs exist, then no roms will be built and, therefore, no blobs are needed Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-06-17Cache downloads based on checksumRiku Viitanen
Since many boards use the same ME firmware, we could save everyone's bandwidth and time by caching the update files. Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-06-13Revert "Remove most of Ferass's lbmk contributions"Leah Rowe
This reverts commit a4ea2867319471d9fe7d4ee540881e0286b4d3cf. The licensing audit has been abandoned. I will not be re-licensing in bulk to MIT. I can still use MIT license on new works, e.g. utilities, but there's really no pressing need to re-license lbmk. It's just shell scripts, and most of what it interacts with (coreboot, grub, seabios) is GPL anyway. So who cares? Ferass's patch was removed due to refusal to re-license, but the decision to re-license has been canceled. I'm now aiming for a quick stable release.
2023-05-27blobutil: merge with main scriptLeah Rowe
make blobutil a symlink. Example of command changes: ./blobutil download x220_8mb is now: ./update blobs download x220_8mb The old command still works, for compatibility. Signed-off-by: Leah Rowe <leah@libreboot.org>