<feed xmlns='http://www.w3.org/2005/Atom'>
<title>lbmk.git/include, branch 20240504</title>
<subtitle>libreboot build system (LibreBoot MaKe)
</subtitle>
<link rel='alternate' type='text/html' href='https://browse.libreboot.org/lbmk.git/'/>
<entry>
<title>err.sh: update copyright info</title>
<updated>2024-05-03T06:14:15+00:00</updated>
<author>
<name>Leah Rowe</name>
<email>leah@libreboot.org</email>
</author>
<published>2024-05-03T06:13:54+00:00</published>
<link rel='alternate' type='text/html' href='https://browse.libreboot.org/lbmk.git/commit/?id=9caff26332ae2c0c5c49ebb4d9946237c1419561'/>
<id>9caff26332ae2c0c5c49ebb4d9946237c1419561</id>
<content type='text'>
i added a few changes during this year so far, 2024

update the copyright years

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
i added a few changes during this year so far, 2024

update the copyright years

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>build/roms: simplified status handling</title>
<updated>2024-04-28T19:33:38+00:00</updated>
<author>
<name>Leah Rowe</name>
<email>leah@libreboot.org</email>
</author>
<published>2024-04-28T19:33:38+00:00</published>
<link rel='alternate' type='text/html' href='https://browse.libreboot.org/lbmk.git/commit/?id=91927760db296596af3946327319625c17c1afa5'/>
<id>91927760db296596af3946327319625c17c1afa5</id>
<content type='text'>
Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>option.sh: don't use nproc (not portable)</title>
<updated>2024-04-28T16:31:16+00:00</updated>
<author>
<name>Leah Rowe</name>
<email>leah@libreboot.org</email>
</author>
<published>2024-04-28T16:31:16+00:00</published>
<link rel='alternate' type='text/html' href='https://browse.libreboot.org/lbmk.git/commit/?id=55204dc4449456c6a11ed19f530b7673143450a5'/>
<id>55204dc4449456c6a11ed19f530b7673143450a5</id>
<content type='text'>
export LBMK_THREADS=x

where x is an integer. this is already supported for
setting the number of build threads, but if not set
it uses nproc.

openbsd doesn't have nproc. default to 1 thread.

now you MUST set threads. e.g. in linux do:

export LBMK_THREADS=$(nproc)

preliminary work is being done to make lbmk run
on openbsd!

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
export LBMK_THREADS=x

where x is an integer. this is already supported for
setting the number of build threads, but if not set
it uses nproc.

openbsd doesn't have nproc. default to 1 thread.

now you MUST set threads. e.g. in linux do:

export LBMK_THREADS=$(nproc)

preliminary work is being done to make lbmk run
on openbsd!

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>allow disabling status checks during builds</title>
<updated>2024-04-27T15:46:31+00:00</updated>
<author>
<name>Leah Rowe</name>
<email>leah@libreboot.org</email>
</author>
<published>2024-04-27T15:46:31+00:00</published>
<link rel='alternate' type='text/html' href='https://browse.libreboot.org/lbmk.git/commit/?id=6c4f07b35071923b4b7cf66f9064108a67d28855'/>
<id>6c4f07b35071923b4b7cf66f9064108a67d28855</id>
<content type='text'>
export LBMK_STATUS=n

if not set, the status checks and confirmation dialogs
persist. if set to y they persist.

if you set it to n, all checks are disabled, so e.g.:

./build roms all

this would once again build all targets, regardless
of status. this is if you want the old behaviour.

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
export LBMK_STATUS=n

if not set, the status checks and confirmation dialogs
persist. if set to y they persist.

if you set it to n, all checks are disabled, so e.g.:

./build roms all

this would once again build all targets, regardless
of status. this is if you want the old behaviour.

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>build/roms: report status when building images</title>
<updated>2024-04-26T19:36:42+00:00</updated>
<author>
<name>Leah Rowe</name>
<email>leah@libreboot.org</email>
</author>
<published>2024-04-26T18:41:00+00:00</published>
<link rel='alternate' type='text/html' href='https://browse.libreboot.org/lbmk.git/commit/?id=ce7fd754a3d831c563a8589f1cc35e55ce027814'/>
<id>ce7fd754a3d831c563a8589f1cc35e55ce027814</id>
<content type='text'>
export LBMK_VERSION_TYPE=x
x can be: stable, unstable

in target.cfg files, specify:
status=x
x can be: stable, unstable, broken, untested
if unset, lbmk defaults to "unknown"

if LBMK_VERSION_TYPE is set, no confirmation is asked
if the given target matches what's set (but what's set
in that environmental variable can only be stable or
unstable)

if LBMK_RELEASE="y", no confirmation is asked, unless
the target is something other than stable/unstable

"unstable" means it works, but has a few non-breaking
bugs, e.g. broken s3 on dell e6400

whereas, if raminit regularly fails or it is so absolutely
unreliable as to be unusable, then the board should be
declared "broken"

untested means: it has not been tested

With this change, it should now be easier to track whether
a given board is tested, in preparation for releases. When
working on trees/boards, status can be set for targets.

Also: in the board directory, you can add a "warn.txt" file
which will display a message. For example, if a board has a
particular quirk to watch out for, write that there. The message
will be printed during the build process, to stdout.

If status is anything *other* than stable, or it is unstable
but LBMK_VERSION_TYPE is not set to "unstable", and not building
a release, a confirmation is passed.

If the board is not specified as stable or unstable, during
a release build, the build is skipped and the ROM is not
provided in that release; this is in *addition* to
release="n" or release="y" that can be set in target.cfg,
which will skip the release build for that target if "n"

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
export LBMK_VERSION_TYPE=x
x can be: stable, unstable

in target.cfg files, specify:
status=x
x can be: stable, unstable, broken, untested
if unset, lbmk defaults to "unknown"

if LBMK_VERSION_TYPE is set, no confirmation is asked
if the given target matches what's set (but what's set
in that environmental variable can only be stable or
unstable)

if LBMK_RELEASE="y", no confirmation is asked, unless
the target is something other than stable/unstable

"unstable" means it works, but has a few non-breaking
bugs, e.g. broken s3 on dell e6400

whereas, if raminit regularly fails or it is so absolutely
unreliable as to be unusable, then the board should be
declared "broken"

untested means: it has not been tested

With this change, it should now be easier to track whether
a given board is tested, in preparation for releases. When
working on trees/boards, status can be set for targets.

Also: in the board directory, you can add a "warn.txt" file
which will display a message. For example, if a board has a
particular quirk to watch out for, write that there. The message
will be printed during the build process, to stdout.

If status is anything *other* than stable, or it is unstable
but LBMK_VERSION_TYPE is not set to "unstable", and not building
a release, a confirmation is passed.

If the board is not specified as stable or unstable, during
a release build, the build is skipped and the ROM is not
provided in that release; this is in *addition* to
release="n" or release="y" that can be set in target.cfg,
which will skip the release build for that target if "n"

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>exports variables from err.sh, not build</title>
<updated>2024-04-26T07:20:19+00:00</updated>
<author>
<name>Leah Rowe</name>
<email>leah@libreboot.org</email>
</author>
<published>2024-04-26T07:20:19+00:00</published>
<link rel='alternate' type='text/html' href='https://browse.libreboot.org/lbmk.git/commit/?id=64177dbb8e7ff9ca9c7ad847c137398f6a67cfcf'/>
<id>64177dbb8e7ff9ca9c7ad847c137398f6a67cfcf</id>
<content type='text'>
LC_COLLATE and LBMK_RELEASE are important variables. we want
to make sure that these are seen by everything.

since err.sh is included from all scripts, doing it there will
accomplish just that.

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
LC_COLLATE and LBMK_RELEASE are important variables. we want
to make sure that these are seen by everything.

since err.sh is included from all scripts, doing it there will
accomplish just that.

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>lbmk: export TMPDIR from err.sh, not build</title>
<updated>2024-04-25T18:08:53+00:00</updated>
<author>
<name>Leah Rowe</name>
<email>leah@libreboot.org</email>
</author>
<published>2024-04-25T18:08:53+00:00</published>
<link rel='alternate' type='text/html' href='https://browse.libreboot.org/lbmk.git/commit/?id=08859bb4a56b282cb566c262bd591d7271308ef3'/>
<id>08859bb4a56b282cb566c262bd591d7271308ef3</id>
<content type='text'>
lbmk sets TMPDIR to /tmp, and then creates a tmpdir, then
exports *that* as the value of TMPDIR. this unified TMPDIR
location then contains all subsequent files and directories,
when any script or program makes use of /tmp, via mktemp. at
least, that's the theory!

in practise, because it was only being properly exported from
the main build scripts, subscripts that are then called were
not exporting it, at least that is my assumption because in
some cases, i found that the coreboot build system was leaving
errant files behind outside of our own TMPDIR, and that build
system did not seem to be setting TMPDIR itself; more debugging
is needed.

anyway: use the exact same logic, but do it from err.sh. since
err.sh is included from every lbmk script, that means it will
always be exported when running every single part of lbmk. this
should reduce the chance that mktemp creates files and directories
outside of our custom TMPDIR location.

this is because in lbmk, we mitigate unhandled tmpdirs/files by
unifying it in the manner described, then deleting the entire
TMPDIR on exit from the main lbmk parent process (the main
script that the user called from, which is always the "build"
file).

in lbmk, effort is made to clean up temporary files properly,
without relying on this catch-all, but we can't rely on that.
the catch-all should also be as robust as possible.

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
lbmk sets TMPDIR to /tmp, and then creates a tmpdir, then
exports *that* as the value of TMPDIR. this unified TMPDIR
location then contains all subsequent files and directories,
when any script or program makes use of /tmp, via mktemp. at
least, that's the theory!

in practise, because it was only being properly exported from
the main build scripts, subscripts that are then called were
not exporting it, at least that is my assumption because in
some cases, i found that the coreboot build system was leaving
errant files behind outside of our own TMPDIR, and that build
system did not seem to be setting TMPDIR itself; more debugging
is needed.

anyway: use the exact same logic, but do it from err.sh. since
err.sh is included from every lbmk script, that means it will
always be exported when running every single part of lbmk. this
should reduce the chance that mktemp creates files and directories
outside of our custom TMPDIR location.

this is because in lbmk, we mitigate unhandled tmpdirs/files by
unifying it in the manner described, then deleting the entire
TMPDIR on exit from the main lbmk parent process (the main
script that the user called from, which is always the "build"
file).

in lbmk, effort is made to clean up temporary files properly,
without relying on this catch-all, but we can't rely on that.
the catch-all should also be as robust as possible.

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>haswell: only provide NRI-based ROMs in releases</title>
<updated>2024-04-24T04:27:27+00:00</updated>
<author>
<name>Leah Rowe</name>
<email>leah@libreboot.org</email>
</author>
<published>2024-04-24T04:27:27+00:00</published>
<link rel='alternate' type='text/html' href='https://browse.libreboot.org/lbmk.git/commit/?id=ed0678ae2e9b58b3eb9a7606816e21ad8798192e'/>
<id>ed0678ae2e9b58b3eb9a7606816e21ad8798192e</id>
<content type='text'>
release="n" is set in target.cfg on haswell build targets
that use mrc.bin

script/update/release exports LBMK_RELEASE="y"

script/build/roms skips building a given target if release="n"
in target.cfg *and* LBMK_RELEASE="y"

you could also do the export yourself before running ./build roms,
for example:

export LBMK_RELEASE="y"
./build roms all

This would skip these ROM images. The native haswell raminit is
now stable enough in my testing, that I wish to delete the MRC-based
targets. This is in line with Libreboot's Binary Blob Reduction Policy,
which states: if a blob can be avoided, it should be avoided.

The problem is that users often run the inject script in *lbmk* from
Git, instead of from the src release archive. I forsee some users
running this on modern lbmk with older release images. If the mrc-based
target isn't there, the user may use an NRI-based target name, and
think it works; they will insert without MRC. I foresaw this ages
ago, which is why Caleb and I ensured that the script checks hashes,
and hashes are included in releases.

Therefore: for the time being, keep the MRC-based configs in lbmk
but do not include images for them in releases. This can be done
indefinitely, but I'll probably remove those configs entirely at
some point.

On the following boards, Libreboot now will *only* provide NRI-based
ROM images for the following machines:

* Dell OptiPlex 9020 SFF
* Dell OptiPlex 9020 MT
* Lenovo ThinkPad T440p
* Lenovo ThinkPad W541/W540

I now recommend exclusive use of NRI-based images, on Haswell
hardware. It's stable enough in my testing, and now supports S3.

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
release="n" is set in target.cfg on haswell build targets
that use mrc.bin

script/update/release exports LBMK_RELEASE="y"

script/build/roms skips building a given target if release="n"
in target.cfg *and* LBMK_RELEASE="y"

you could also do the export yourself before running ./build roms,
for example:

export LBMK_RELEASE="y"
./build roms all

This would skip these ROM images. The native haswell raminit is
now stable enough in my testing, that I wish to delete the MRC-based
targets. This is in line with Libreboot's Binary Blob Reduction Policy,
which states: if a blob can be avoided, it should be avoided.

The problem is that users often run the inject script in *lbmk* from
Git, instead of from the src release archive. I forsee some users
running this on modern lbmk with older release images. If the mrc-based
target isn't there, the user may use an NRI-based target name, and
think it works; they will insert without MRC. I foresaw this ages
ago, which is why Caleb and I ensured that the script checks hashes,
and hashes are included in releases.

Therefore: for the time being, keep the MRC-based configs in lbmk
but do not include images for them in releases. This can be done
indefinitely, but I'll probably remove those configs entirely at
some point.

On the following boards, Libreboot now will *only* provide NRI-based
ROM images for the following machines:

* Dell OptiPlex 9020 SFF
* Dell OptiPlex 9020 MT
* Lenovo ThinkPad T440p
* Lenovo ThinkPad W541/W540

I now recommend exclusive use of NRI-based images, on Haswell
hardware. It's stable enough in my testing, and now supports S3.

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>allow users to specify number of build threads</title>
<updated>2024-03-27T07:38:16+00:00</updated>
<author>
<name>Leah Rowe</name>
<email>leah@libreboot.org</email>
</author>
<published>2024-03-27T07:14:47+00:00</published>
<link rel='alternate' type='text/html' href='https://browse.libreboot.org/lbmk.git/commit/?id=332778973c3bca867dd2aa608b89a017a2e44b77'/>
<id>332778973c3bca867dd2aa608b89a017a2e44b77</id>
<content type='text'>
lbmk otherwise uses nproc to set the number of build threads,
in these places:

* generic make commands in script/update/trees
* crossgcc make command in script/update/trees

the -T0 option is also used in script/update/release, when running
tar.

with this change, you can do:

export LBMK_THREADS=x

where x is the number of threads. when you then run
lbmk, your chosen number of threads will override
the default. this may be useful on a host that does
not have a lot of memory.

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
lbmk otherwise uses nproc to set the number of build threads,
in these places:

* generic make commands in script/update/trees
* crossgcc make command in script/update/trees

the -T0 option is also used in script/update/release, when running
tar.

with this change, you can do:

export LBMK_THREADS=x

where x is the number of threads. when you then run
lbmk, your chosen number of threads will override
the default. this may be useful on a host that does
not have a lot of memory.

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</pre>
</div>
</content>
</entry>
<entry>
<title>safer, simpler error handling in lbmk</title>
<updated>2024-03-27T01:50:31+00:00</updated>
<author>
<name>Leah Rowe</name>
<email>leah@libreboot.org</email>
</author>
<published>2024-03-27T01:19:39+00:00</published>
<link rel='alternate' type='text/html' href='https://browse.libreboot.org/lbmk.git/commit/?id=6ebab10caa5be6fc1cfd244e745851687d4bd70d'/>
<id>6ebab10caa5be6fc1cfd244e745851687d4bd70d</id>
<content type='text'>
in shell scripts, a function named the same as a program included in
the $PATH will override that program. for example, you could make a
function called ls() and this would override the standand "ls".

in lbmk, a part of it was first trying to run the "fail" command,
deferring to "err", because some scripts call fail() which does
some minor cleanup before calling err.

in most cases, fail() is not defined, and it's possible that the user
could have a program called "fail" in their $PATH, the behaviour of
which we could not determine, and it could have disastrous effects.

lbmk error handling has been re-engineered in such a way that the
err function is defined in a variable, which defaults to err_ which
calls err_, so defined under include/err.sh.

in functions that require cleanup prior to error handling, a fail()
function is still defined, and err is overridden, thus:

err="fail"

this change has made xx_() obsolete, so now only x_ is used. the x_
function is a wrapper that can be used to run a command and exit with
non-zero status (from lbmk) if the command fails. the xx_ command
did the same thing, but called fail() which would have called err();
now everything is $err

example:

	rm -f "$filename" || err "could not delete file"

this would now be:

	rm -f "$filename" || $err "could not delete file"

overriding of err= must be done *after* including err.sh. for
example:

err="fail"
. "include/err.sh"

^ this is wrong. instead, one must do:

. "include/err.sh"
err="fail"

this is because err is set as a global variable under err.sh

the new error handling is much cleaner, and safer. it also reduces
the chance of mistakes such as: calling err when you meant to
call fail. this is because the standard way is now to call $err,
so you set err="fail" at the top of the script and all is well.

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
in shell scripts, a function named the same as a program included in
the $PATH will override that program. for example, you could make a
function called ls() and this would override the standand "ls".

in lbmk, a part of it was first trying to run the "fail" command,
deferring to "err", because some scripts call fail() which does
some minor cleanup before calling err.

in most cases, fail() is not defined, and it's possible that the user
could have a program called "fail" in their $PATH, the behaviour of
which we could not determine, and it could have disastrous effects.

lbmk error handling has been re-engineered in such a way that the
err function is defined in a variable, which defaults to err_ which
calls err_, so defined under include/err.sh.

in functions that require cleanup prior to error handling, a fail()
function is still defined, and err is overridden, thus:

err="fail"

this change has made xx_() obsolete, so now only x_ is used. the x_
function is a wrapper that can be used to run a command and exit with
non-zero status (from lbmk) if the command fails. the xx_ command
did the same thing, but called fail() which would have called err();
now everything is $err

example:

	rm -f "$filename" || err "could not delete file"

this would now be:

	rm -f "$filename" || $err "could not delete file"

overriding of err= must be done *after* including err.sh. for
example:

err="fail"
. "include/err.sh"

^ this is wrong. instead, one must do:

. "include/err.sh"
err="fail"

this is because err is set as a global variable under err.sh

the new error handling is much cleaner, and safer. it also reduces
the chance of mistakes such as: calling err when you meant to
call fail. this is because the standard way is now to call $err,
so you set err="fail" at the top of the script and all is well.

Signed-off-by: Leah Rowe &lt;leah@libreboot.org&gt;
</pre>
</div>
</content>
</entry>
</feed>
