| From: o1bigtenor via talk <talk@gtalug.org> | On Wed, Jul 7, 2021 at 5:52 PM D. Hugh Redelmeier via talk | <talk@gtalug.org> wrote: | > | > == why you might wish to grow the ESP == | > | > In a UEFI system, booting starts with the ESP (EFI System Partition). | > | > Apparently Windows creates a 100M ESP during a fresh install. | Does this also happen if there is no Win quotient in the partition? If you are asking: how much does Windows add to the requirement for space? I don't know. Also: each manufacturer may decidedd to put more or less diagnostic software in the ESP too. If you are asking: how big an ESP would a Linux installation create? I don't know. The installers I use give a suggestion but let me override. What's a good size for an ESP? My current guess is that 200M is plenty and isn't too wasteful given how large disks are these days. On my XPS 15 notebook (the one that prompted the adventure): Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda2 200623 95057 105566 48% /boot/efi (More was used during the firmware update.) On my desktop SSD: Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb2 364544 65436 299108 18% /boot/efi (The last firmware update was in 2014.) Blue sky uses of ESP: I imagine, if you want a lean installation, you could eliminate grub and boot a linux kernel directly from UEFI. - the kernel and initramdisk files would have to live in the ESP - the kernel wuld have to be in .efi form I'd love to get rid of a whole layer of software from the booting process. What does grub2 do for us? - select what to boot (but most UEFI software lets you do that too) - place the OS on filesystems/partitions other than the ESP - fiddle about when things go wrong. Especially when that thing is grub

On Thu, Jul 08, 2021 at 03:24:29PM -0400, Nicholas Krause via talk wrote:

Have you looked at: https://wiki.archlinux.org/title/EFISTUB

Not sure of how much use it is for you. Not for me currently but may be of interest.

Using grub is way more flexible. Having only one bootable kernel is not convinient which is what you essentially have if you make a UEFI boot entry pointing at a kernel directly. Sure if your system has a UEFI shell (most desktop and laptop machines seem allergic to that concept) then you could certainly boot such a kernel by picking the one you want, although you would have to then enter all the arguments for the kernel manually which is a hassle. I can't really see an actual use case for the EFISTUB. -- Len Sorensen

On Fri, Jul 09, 2021 at 03:40:19PM -0400, D. Hugh Redelmeier via talk wrote:

I readily admit that too many UEFI implementations make it awkward to choose what to boot. But the UEFI standard seems to support multiple kernels in the same ESP. (At least some ThinkCentre firmware is bad in this repsect; XPS 15 firmware is good.)

efibootmgr seems to support setting this up (it might not be the best tool). Here's what this machine's entry for fedora looks like. It's got enough stuff that you could create a distinct entry for each kernel of interest.

Boot0000* Fedora HD(2,GPT,f66e4ede-1301-47fd-af96-7f45aee7bc28,0x40800,0xb4000)/File(\EFI\fedora\shimx64.efi)

I seem to recall reading that certain implementations of UEFI would die if you wrote to the nvram too many times. Updating boot entries in UEFI each time you upgrade the kernel might be a bad idea in that case.

Apparently Microsoft dictated that if you wish to be considered Windows-ready, you must not ship with the UEFI shell.

Yes amazingly stupid to mandate one of the most convinient repair tools isn't allowed on the system.

You can build one -- the source is available. But I haven't ever done so.

-- Len Sorensen

On Sat, Jul 10, 2021 at 10:21:07AM -0400, D. Hugh Redelmeier via talk wrote:

Really? That's terrible. I often update UEFI settings.

Do you have a source for this?

I can't find any. There was a bug that made NVRAM read only on some systems in 2017 (that was in ubuntu 17.10), but nothing else I can find. I can find some comments about trying to reduce the writes to NVRAM to avoid wear, but nothing specific I can find that actually says people have had NVRAM actually wear out. It would have been some years ago.

(Updating Windows, without making it the default boot, is really annoying.)

Hmm, I seem to recall newer windows versions have been better behaved than they used to be.

What do you find it useful for? I've not really missed it. When things get sticky, I boot Linux from a USB stick.

Well being able to just go in an launch grub or something can be handy. Also firmware update tools for some network cards can be run from UEFI which avoids needing to install special drivers or windows.

Where do you get a .efi for the UEFI shell? Some place that you trust. (Building it myself seems like too much effort.)

An unexplored niche: an enhanced UEFI shell.

I have only ever used the built in one on servers and some embedded systems.

(The only time I considered programming for the EFI environment was when I found a couple of cheap Win 10 devices with too small eMMC. There was no room for Linux without ditching Windows. They could not boot from the SD card interface. With a .efi SD driver, grub would be able to boot off the SD card.)

-- Len Sorensen

On Wed, Jul 14, 2021 at 09:49:30AM -0400, Alvin Starr via talk wrote:

Years ago I worked on an avionics project where we were storing data in an NVRAM. We had to write code to manage the wear-out in the cells and if I remember correctly the wear-out was in the hundreds of thousands to millions of writes before problems would occur.

I could not see normal usage causing wear-out unless someone were to have some kind of bug causing a high write rate.

Consumer grade flash often only has 10000 write cycles. Especially if it isn't expected to be updated very much. Different price point. -- Len Sorensen

| From: Alvin Starr via talk <talk@gtalug.org> | Years ago I worked on an avionics project where we were storing data in an | NVRAM. | We had to write code to manage the wear-out in the cells and if I remember | correctly the wear-out was in the hundreds of thousands to millions of writes | before problems would occur. | | I could not see normal usage causing wear-out unless someone were to have some | kind of bug causing a high write rate. The following is based on knowledge plus a fair bit of superstitious extrapolation and interpolation. NAND flash and NOR flash are different. You are surely talking about a NOR flash Lennart is talking about an unknown system. I suspect it used NAND flash, if the description is right. NOR flash: - allows update of each byte separately (most are 1 byte wide) - each update only wears the byte updated - are small (eg. 8M is large). Older consumer routers used NOR flash and that is why their flash sizes were so limited. - it is reasonable to run programs directly out of NOR flash (it looks like RAM) NAND flash - allows updates in pages, but only to previously erased pages - erases can only be done in large blocks. The size of these blocks is generally secret but I think that they are in the range of 128KiB to 1MiB. - are so alien to deal with that they have a complicated controller between them and your system. The controller emulates a disk drive. This does wear levelling. There is some work at moving this into the OS for performance reasons, but it is not common and only for data-centres. - density is everything in NAND flash systems. The original ones used a single bit per cell (SLC) but now 3 bits per cell is common. To store 3 bits per cell, 8 charge levels must be distinguished. This makes them much slower and less reliable (shorter-lived). - each cell can only be erased a small number of times (perhaps 1000) but the wear-levelling usually prevents this being a big problem. It is not part of published specifications. - consumer routers now use NAND flash for firmware and allow orders of magnitude larger firmware. For reasons I haven't investigated, it took OpenWRT years to support NAND flash routers. - it is unreasonable to run programs directly out of NAND flash. They can, however, be paged in, just as one would do from a HDD. This wrecks real-time performance, so routers would not do this. They copy the contents of flash to RAM as part of booting. Normal PCs: - have used a battery-backed-up CMOS RAM module for firmware settings. Everything is CMOS now, I think, but when this dedicated memory was added to the PC, CMOS was notable. Especially for its low standby power requirement. - use flash memories with sequential access for firmware (SPI). The power-on sequence fetches the firmware and dumps it into RAM so that it can be accessed randomly. Dedicated programs are needed to update the flash through the SPI interface. - uses RAM, just as we are used to - uses SSD, with its own controller to emulate HDD but more quickly.

On Thu 2021-07-15 @ 10:37:03 AM, D. Hugh Redelmeier via talk wrote:

NAND flash

- allows updates in pages, but only to previously erased pages

NAND flashes are divided into pages, pages are organized into blocks, and a bunch of block go together to form planes. Modern devices also have sub-pages. Erasing usually occurs on blocks, but reads and writes are to pages (or sub-pages) at a time.

- erases can only be done in large blocks. The size of these blocks is generally secret but I think that they are in the range of 128KiB to 1MiB.

The size of the erase block is *always* stated in each device's datasheet. Otherwise how else would we be able to configure the SoC's NAND controller? An SoC's NAND controller needs to know the size of the erase block in order to work with the NAND correctly.

- are so alien to deal with that they have a complicated controller between them and your system. The controller emulates a disk drive. This does wear levelling. There is some work at moving this into the OS for performance reasons, but it is not common and only for data-centres.

You're talking about "managed flash" devices, such as eMMC. True, these do have a built-in "invisible" controller between you and flash that does wear leveling etc. These devices generally look like hard-drives. However, many embedded systems have "raw flash" NAND devices on them. You're welcome to interact with them directly if you wish, but you risk wearing them out prematurely due to lack of wear leveling etc. In this case the smartest way to work with them is via the kernel's mtd subsystem which provides a software-based abstraction between your filesystem and the raw device and performs all the same duties as the aforementioned controller used with managed devices (and often have several additional features). If nothing else, at least with the software mtd layer you know and are in control of what's going on. Ideally you should use an mtd-aware filesystem on top of the kernel's mtd subsystem. Currently the best fileystems to use on top of mtd for use with raw NAND flash are UBI/UBIFS and F2FS. JFFS2 used to be a popular choice, but is considered legacy at this point.

- density is everything in NAND flash systems. The original ones used a single bit per cell (SLC) but now 3 bits per cell is common. To store 3 bits per cell, 8 charge levels must be distinguished. This makes them much slower and less reliable (shorter-lived).

- each cell can only be erased a small number of times (perhaps 1000) but the wear-levelling usually prevents this being a big problem. It is not part of published specifications.

Although SLC was the original (and therefore "old" technology) they are still very much readily available and very actively used. For example: https://www.digikey.ca/en/products/filter/memory/774?s=N4IgTCBcDaIM4BsDGACAd... SLC NAND flash is fast, usually supports a minimum of 100,000 erase cycles, but is much more expensive. The first page/block of SLC is usually guaranteed to (never?) wear out (it'll have an erase count that is an order or two higher than the rest of the flash). So based on your requirements you can either go expensive but fast and long-lasting; or cheap, slow, and a shorter life. All types of flash are still actively produced. The minimum number of expected erase cycles for any given device is *always* specified in the datasheet along with the expected data retention length. Note that the endurance of the device is almost always specified along with a given expected level of ECC. So the datasheet will say, for example: endurance: typical 60k cycles (with 8-bit ECC per (512+32) bytes In this case 512 is the sub-page size and 32 are the OOB bytes. Although single cell per bit designs were called SLC, two bits per cell designs were called "multi-level cell" (MLC). When three bits per cell technology came along they were called TLC (three-level cell). So the nomenclature goes: single → multi → three :-)

- it is unreasonable to run programs directly out of NAND flash. They can, however, be paged in, just as one would do from a HDD. This wrecks real-time performance, so routers would not do this. They copy the contents of flash to RAM as part of booting.

True. It's simply not possible to run code from a device that needs to be accessed a page (or sub-page) at a time. A CPU expects byte (or at least word) access to the instruction stream. Therefore if a raw NAND contains an executable it must be copied to some other byte-accessible medium (e.g. static RAM, dynamic RAM, etc) before it can be consumed by the CPU.

| From: Lennart Sorensen via talk <talk@gtalug.org> | On Sat, Jul 10, 2021 at 10:21:07AM -0400, D. Hugh Redelmeier via talk wrote: | > (Updating Windows, without making it the default boot, is really | > annoying.) | | Hmm, I seem to recall newer windows versions have been better behaved | than they used to be. What I said was ambiguous. What I meant was: Windows update can get into the weeds if the system isn't set up to boot Windows by default. A number of steps require a reboot and if Windows isn't the default, things can go wrong unless you are there to catch it. You don't want to be there because Windows Update is more boring than watching paint dry (except when it goes wrong). What I think you are referring to is that sometimes (when?) Windows fiddles with firmware settings / MBR to make itself the default boot. I've had this happen but I don't know when. This was really annoying in the MBR days because recovery was awkward but with UEFI it is usually easy to fix from firmware setup. Yes, this happens less these days, but it still happens.