summaryrefslogtreecommitdiff
path: root/script/update
AgeCommit message (Collapse)Author
2023-10-01Greatly simplify error handling in shell scriptsLeah Rowe
Instead of having detailed error messages, run most commands through a function that calls err() under fault conditions. Where detail is still required, err() is still called manually. Where it isn't, the error message is simply whatever command was executed to cause the error. This results in a massive sloccount reduction for lbmk; specifically, 178 sloc reduction, or a 8.1% reduction. The total sloccount is now 2022, for shell scripts. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30general code formatting cleanup in shell scriptsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30simplify initialising variables in shell scriptsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-30update/*/*: unified scanning of revisions/sourcesLeah Rowe
update/blobs/download and update/project/repo both use the same logic, for setting variables with awk and a specially formatted configuration file. unify this logic under include/option.sh, and use that. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29blobs/download: move helpers to include/blobutilLeah Rowe
mkdirs() should be in include/blobutil.sh, as should extract_archive(), because that is primarily where they are used. script/update/blobs/download calls these functions aswell, but it sources include/blobutil.sh so it's OK. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29include/blobutil: try curl first, then wgetLeah Rowe
Don't use only wget. Some systems may only have curl. The user can always install wget anyway, but why not support both? I've added the right user agent string. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29include/blobutil: simplify check_defconfig()Leah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29blobs/download: do IntelME extract in one functionLeah Rowe
Just one function. Just one. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29blobs/download: do final check of _dest in fetch()Leah Rowe
This way, the file is checked regardless of what type of blob is handled, not just Intel ME. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29blobs/download: don't pass dl_path as argumentLeah Rowe
Because fetch() is only called now from blobs/download, we can reliably know what dl_path should be. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29blob scripts: unified handling of blob destinationLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29remove unused variables in blob scriptsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29merge include/fetch.sh, blobutil.sh, defconfig.shLeah Rowe
They are only ever used by script/update/blobs/*, so put them all in blobutil.sh. This cuts down on the number of scripts in lbmk. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29update/blobs/*: simplify mrc.bin handlingLeah Rowe
mrc.bin is now handled by include/mrc.sh, adapted from now-deleted script/update/blobs/mrc much of the logic has been re-written or adapted for inside script/update/blobs/download mrc links/hashes now defined in config/blobs/sources the new code is simpler (and smaller). in addition, lbmk can now easily handle mrc.bin files for other platforms such as broadwell. watch this space. the full .zip download is now cached, like with other vendor downloads. this means it won't be re-downloaded if it was already downloaded before. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-29update/blobs/*: unified download/checksum logicLeah Rowe
Use the same logic between blobs/download and blobs/mrc. The logic is taken from blobs/download. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28blobs/download: remove unnecessary linebreaksLeah Rowe
generally condense the code, but not in a way that makes the code unreadable. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28blobs/download: remove unnecessary messagesLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28blobs/download: simplify downloading of filesLeah Rowe
individual functions for downloading each archive have been removed. instead, eval is used in fetch_update(), which is now renamed to fetch(). Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28blobs/download: remove unnecessary error pipesLeah Rowe
the called functions directly call err() under fault condition, so this additional handling is redundant. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-28blobs/download: unified archive extractionLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27move build/command/options to include/option.shLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27blobs/download: unified blobdir handlingLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27blobs/download: unified archive extractionLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27blobs/download: remove errant debug lineLeah Rowe
i added that echo command when working on the function in question, but it's not needed now. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27remove script/update/blobs/extractLeah Rowe
This script is incomplete, buggy and its use is ill advised. This script can be re-added later, when more work is done. The download and/or inject script is recommended. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27blobs/download: simplify fetch_update()Leah Rowe
Do not specifically name types of firmware. Instead, pass the URLs and checksum as direct arguments. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27blobs/download: greatly simplify sources handlingLeah Rowe
remove the giant case/esac list, and set variables directly. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-27blobs/download: simplify defconfig handlingLeah Rowe
use the variable names directly, as defined in defconfig. do not hardcode the if/else chain in detect_firmware, use eval instead. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-26simplify getopts loops in shell scriptsLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25only update git submodules in project/treesLeah Rowe
do not update them in project/repos - despite what the previous commit message says, this behaviour is error prone and should be avoided. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25update/project/*: unified git reset handlingLeah Rowe
With this change, lbmk now also updates submodules on simple git clones, not just multi-tree clones. This is OK, because git does not return non-zero status when git submodule update is ran, where git submodules are not actually defined. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25update/project/*: unified patch handlingLeah Rowe
Handle patches by a function at include/git.sh Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25update/project/trees: remove extra.sh handlingLeah Rowe
This functionality has never been used, except in the erstwhile osboot project, and even then only experimentally. It was intended for use with coreboot's gerrit site, but it became Libreboot project policy that this not be relied upon, instead preferring to include patches directly within lbmk. This functionality can be re-added, if necessary. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25Update email address for Leah Rowe copyrightsLeah Rowe
also, some of them were out of date; years now updated. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25Use SPDX license headers on all scriptsLeah Rowe
This results in much cleaner copyright and license declarations. SPDX headers are legally recognised and make auditing easier. Also, remove descriptions of each script, from each script. Libreboot documentation at docs/maintain/ describes them. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-25update/repos: concatenate multiple revision filesLeah Rowe
With this change, it's still possible to have a single file at config/git/revisions, but this has been scrapped. Instead, multiple files now exist under config/git/ with the same modules declared, but the files are separated logically. List of files under config/git: * bios_extract * biosutilities * coreboot * flashrom * grub (gnulib also defined here) * me_cleaner * memtest86plus * seabios * serprog (multiple projects defined) * u-boot * uefitool The rationale behind this change is simple: in the future, we will stop relying on build systems within imported projects for the import of git submodules. Instead, we will handle them directly in lbmk. Additionally, a Linux payload is planned for Libreboot, made easier by the recent audit (script handle/make/config makes it easy to integrate Linux, and handle cross-compilers for userland utilities); a "linux" file under config/git/ could also define rules for each project besides linux, such as musl libc, busybox and other utilities. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-10build/boot/roms stragglers: properly handle errorsLeah Rowe
there were a few missing err calls i actually went through all of lbmk and found no instances where err calls were missing except in build/boot/roms_helper Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09scripts: use printf, not echo, where appropriateLeah Rowe
printf has more universal behaviour, across various implementations of sh, so it's better to use this. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09update/blobs printf statements: use double quotesLeah Rowe
single quotes are not valid Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09scripts: better handling of printf: stdout/stderrLeah Rowe
in some cases, messages that should be considered errors or warnings, were being written to the standard output, rather than written as error messages. also: one or two printf statements should specifically avoid printing errors (to any file); in these cases, stdout has been redirected to /dev/null Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09update/project/*: remove redundant checksLeah Rowe
these scripts used to be in the main directory of lbmk, and thus needed to check for root user, and also git credentials. now they are called by the main lbmk script, which also runs the same checks. avoid waste of resources by not running the same check twice. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09blobs/download: don't use the -B option in makeLeah Rowe
Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09blobs/inject: add error condition on rm commandLeah Rowe
this was overlooked, earlier on in lbmk audit 2 Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09blobs/inject: fix checksum validation if no-ucodeLeah Rowe
on e6400_4mb, the release build scripts remove nvidia's vga rom which is used on dgpu models. however, microcode is also removed in separately copied rom images the inject script was inserting vgaroms directly into these no-microcode roms, but the microcode blob is bigger than the vga rom, and cbfstool inserts into the first available free spot within cbfs, so it was inserting into the spot where cpu microcode went. this caused the rom checksum to not match what was generated during build/release/roms being executed the only real fix is to guarantee offsets within cbfs for all files, by recording what offsets were used and then calculating that during insertion so this patch is a workaround, but fixes the issue. the workaround is: don't insert blobs directly on no-microcode roms, instead insert only on microcode-based roms, then re-copy those roms and remove microcode in aptly named copies it's a bit more convoluted, but works perfectly fine. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-09use sha512sum to check downloads, not sha1sumLeah Rowe
sha-1 has known collision issues, which may not be readily exploitable yet (in our context), but we should ideally use a more secure method for checking file integrity. therefore, use sha-2 (sha512sum) for checking files. this is slower than sha-1, but checksum verification is only a minor part of what lbmk does, so the overall effect on build times is quite negligible. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-08Support multiple dependencies per projectRiku Viitanen
Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-08Cleaner parent directory creationRiku Viitanen
My previous patch b0rked memtest and others because when making sure their parent directory (the project root) exists, it would instead create the project directory (memtest86lus). The later move would then put the git repo inside that (memtest86plus/memtest86plus_123456). We just need to make sure we don't create the target directory itself. This way, there's no need to hardcode any project names. Tested by ./updating rpi-pico-serprog, memtest86plus, grub and seabios. Signed-off-by: Riku Viitanen <riku.viitanen@protonmail.com>
2023-09-05update/blobs/inject: use tmp/romdir, not TMPDIRLeah Rowe
we are copying large numbers of ROM images, and the host system may have /tmp under a tmpfs; that same host system may or may not have a lot of memory. respect the user's machine. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04curl/wget downloads: set re-try count to 3Leah Rowe
explicitly set the count to 3, so that a maximum of 3 attemps are made per download, barring fatal errors such as http 404. Signed-off-by: Leah Rowe <leah@libreboot.org>
2023-09-04don't use /tmp/ for git clones. use tmp/ instead.Leah Rowe
the /tmp/ file system may be a tmpfs, with conservative memory limits, depending on host system. it's more likely that the user will have enough disk space under tmp/ within lbmk (if they don't, they can't use lbmk anyway). that is to say: more likely that they would have the disk space, but not the memory. Signed-off-by: Leah Rowe <leah@libreboot.org>