OETF #16 - Open Extensible Firmware Interface

[Zen1th 14]

Open Extensible Firmware Interface v2.1

FOR LUA ARCHITECTURE, OEFI PRESENTS LITTLE TO NO ADVANTAGES!

Lua BIOSes that implements OEFI must allow using conventional booting.
Note that OEFI has been made to be more architecture-independent and will support ANY architecture able to read files and CPIO (basically any architecture with basic access to component and bitwise operations).

Concepts

• Application: Executable code booting using OEFI in the EFI1 or EFI2 format.
• Conventional Booting: OpenComputers default booting method. Often called "BIOS"
• Operating System (OS) : Complex Application with user interaction and/or providing an environment for programs. Concept only used in this document for Conventional Booting
• OC: Shortcut for "Open Computers"

Rationale

Due to the limitation of current OC booting method which basically searches for an init.lua file, the most big problem is that this only work for Lua architecture, and what should other architectures use, init.asm? init.test? init.o? What happens if we want one drive to contains multiple inits for multiple architectures? Well that's where Open Extensible Firmware Interface comes handy. Using . The limitations OEFI fix in Conventional Booting in OC are:

• In Conventional Booting, there is only one init script per filesystem, while in OEFI there can be multiple applications per filesystem.
• Conventional Booting doesn't have any native support for multiple applications. An OS that want to boot another one need to manually check for init.lua files in all filesystems, OEFI has oefi.getApplications() method for it, this is even worse for Applications booting on unmanaged drives
• Conventional Booting isn't made for any other architecture than Lua. And with it, like addressed below, it's impossible to have multiple applications for multiple architectures, while OEFI allows in theory to have one same drive to boot on all architectures it support, without tweaks needed from the user!

OEFI is basically a much more extensible version of OETF #1, « Standardized booting of nonstandard architectures » and Conventional booting.

2.1 Changelog

• Changed EFI2 format into a properties file for easier parsing
• Using CPIO instdead of URF for same reason: easier parsing
• Added OEFI Extensions, oefi.loadInternalFile(path) and oefi.setApplications(appTable)
• Fixed having put AwesomeCatgirl instdead of AdorableCatgirl as username for CAP conntribution

Application Table Format

The application table must contains tables as entries.
The tables must contains the drive address with as « drive » entry and the path (from drive) as « path » entry

Example:

{
  {
    drive = "af58dbe2-ff8c-433a-a871-c08723e01f25",
    path = ".efi/sampleos.efi"
  },
  {
    drive = "af58dbe2-ff8c-433a-a871-c08723e01f25",
    path = ".efi/osOnSameDrive.efi2"
  },
  {
    drive = "015f9d4c-cdfb-4686-91cb-bc8cf997f4ec",
    path = ".efi/another_os.efi"
  }
}

For architectures not supporting hash maps / dictionnaries, "drive" is index 0 and "path" is index 1

For architectures not supporting arrays inside arrays (2D arrays), the arrays should be appended in such a way it look like that: drive, path, drive, path, drive, path, ... . This is the default standard used by C for multi-dimensional arrays. If the language supports multi-dimensional arrays then the arrays of the language must be used.

Finding Applications

OEFI Applications can be found on any sort of drives, from disk drives to floppy, with raids, only if they contains a ".efi" directory.
The OEFI should search files in ".efi" directory:

If the file ends with .efi, then check if it contains CPIO signature, if no, boot in compatibility mode (OEFI1, which is not described anymore on this document).
However if the same file contains that signature, then boot as normal.
.efi2 will ALWAYS be in standard EFI2 format, unlike .efi which can be EFI or EFI2.
.efi can be EFI1 or EFI2 to keep compatibility with old OEFI Applications, and to allow 8.3 filesystems to still support OEFI (would be problematic otherwise)

API Methods

Methods are same as in Version 1, of course API version is now 2 (logic), and some new methods are being added, so here is the new list of methods:

• oefi.loadfile(path) - Loads file from boot drive
• oefi.loadInternalFile(path) - Loads file from current archive - new in OEFI 2.1
• oefi.getBootAddress() - replaces computer.getBootAddress()
• oefi.getApplications() - Return an application table as described above
• oefi.setApplications(appTable) - Set the application table (as described above) to appTable. - new in OEFI 2.1
• oefi.getAPIVersion() - Return API version number. Returns float, double or any decimal 2.1 for this version
• oefi.getImplementationName() - Returns implementation name
• oefi.getImplementationVersion() - Returns implementation version number
• oefi.returnToOEFI() - Recalls the OEFI program, the expected behavior is that the implementation will try to re-execute itself.
• oefi.execOEFIApp(drive, path) - Boot another OEFI app by using the implementation's routine
• oefi.getExtensions() - OEFI EXTensions is for non-standard features from implementation. This should always be used for implementation methods. The way this work is described below - new in OEFI 2.1

For architectures supporting tables (and global), API must be available in Global
Else, API must be available as a pointer/struct/class/anything like that available as argument to the code

OEFI Extensions work in the following way: If the architecture supports entries with keys, then a entry should be created containing the function (or its pointer), and the key should be equals to ImplementationName_MethodName_MethodVersion, ImplementationName should be same as in oefi.getImplementationName(), MethodVersion should be the revision number of a method, starts at 1. And MethodName is the name of the method. Example: SuperOEFI_DrawHandSpinner_2, which could be called (in Lua) via oefi.getExtensions().SuperOEFI_DrawHandSpinner_2()

If tables are not supported, then the array (returned by oefi.getExtensions()) should contain all the methods as other arrays. So oefi_ext will be a 2D array. Each array should contain the name (see above) at the first index (0 or 1, depends on architecture),  and a pointer to the function (or the function itself, if possible) at second index.

If 2D array too aren't supported, then like Application Table, the array should act like if they are appended to each other.

Please also note that if possible, computer.getBootAddress() and computer.setBootAddress(addr) should be REMOVED! This is due to that Applications supporting OEFI doesn't need any compatibility methods as thoses conventional booting methods are only kept when booting a OS that only support conventional booting.
Implementation can create their own methods, however it should ALWAYS BE INSIDE oefi.getExtensions(). (see above)

EFI2

EFI2 is a new format for Applications, it replaces EFI(1) in , it is a CPIO archive. All files in that archive must be placed at root and are listed here:

• app.cfg
• app.exe

app.exe is only the Application, it contains the code designed for target architecture, and will be launched by the OEFI .

app.cfg is a configuration file using "key=value" scheme. The file should look like this:

name=Application Name
version=1.0
oefiVersion=2.1
arch=Lua
archMinVer=5.2
archMaxVer=5.3

Space after key or before name is parsed as it and aren't ignored, be careful!

"name" is equals to the name of the Application. Will always be a text
"version" is equals to the version of the Application, if you don't want to fill it, just make it stay to 1.0. Will always be a decimal
"arch" is a string with the name of supported architecture, "archMinVer" and "archMaxVer" are self-explanatory and will always be a decimal.
Note that for archMinVer and archMaxVer, -1 can be used if a version doesn't make sense to the architecture to have a version. (example: a 6502 architecture)
"oefiVersion" is equals to the OEFI version the Application has been designed to run with, if the "oefiVersion" field have a version incompatible with the current API version or if it's higher than the current one (ex. it's equals to 3 but we're on API v2.1, or it's equals to 1 but we're on API v2). This will also always be a decimal

With all thoses fields in mind, the EFI2 was designed to be durable, with only changes being to app.lon and URF version.

Configuration Data

Everything is explained in that image:

Component Address Packing

To shrink down components to 16 bytes, since components address (and UUIDv4 in general) are just hexadecimal numbers converted to string, we can re-convert the string to a byte array and vice-versa.
To do it, the lines (-) must be ignored, and each group of 2 characters must be interpreted as a hexadecimal string, this should be easy with architecture supporting string to number with optional base (in our case base is equals to 16)
For example it turns:

68ca0f59-ac2c-49f8-ba6a-3c4c7e5f069b

into:

{0x68, 0xCA, 0x0F, 0x59, 0xAC, 0x2C, 0x49, 0xF8, 0xBA, 0x6A, 0x3C, 0x4C, 0x7E, 0x5F, 0x06, 0x9B}

Here is a code sample submitted by AdorableCatgirl to encode and decode the addresses:

function binToHex(id)
	local f, r = string.format, string.rep
	return f(f("%s-%s%s", r("%.2x", 4), r("%.2x%.2x-", 3), r("%.2x", 6)), id:byte(1, 16))
end

function hexToBin(addr)
	addr = addr:gsub("%-", "")
	local result = ""
	for i=1, #addr, 2 do
		baddr = result .. string.char(tonumber(addr:sub(i, i+1), 16))
	end
	return baddr
end
-- example: hexToBin("68ca0f59-ac2c-49f8-ba6a-3c4c7e5f069b") == {0x68, 0xCA, 0x0F, 0x59, 0xAC, 0x2C, 0x49, 0xF8, 0xBA, 0x6A, 0x3C, 0x4C, 0x7E, 0x5F, 0x06, 0x9B}
-- "68ca0f59-ac2c-49f8-ba6a-3c4c7e5f069b" == binToHex(the array below)

Back to EEPROM data, implementation name and custom configuration are very easy to understand what they're about.

[Adorable-Catgirl 11]

I was thinking about component IDs. Shouldn't they be minified? Like, "68ca0f59-ac2c-49f8-ba6a-3c4c7e5f069b" turns into a string of 16 bytes:

{0x68, 0xCA, 0x0F, 0x59, 0xAC, 0x2C, 0x49, 0xF8, 0xBA, 0x6A, 0x3C, 0x4C, 0x7E, 0x5F, 0x06, 0x9B}

Also, OEFI implementations need to have room for custom configuration, as something like Zorya needs some EEPROM space for knowing the device that the "zorya-module" and "zorya-cfg" folders are stored on. Maybe 64 bytes or so can be dedicated to custom config? 64 bytes should be plenty of space for basic configuration, yeah? 32 bytes would be too little, probably, as a component ID can only be shrunk down to 16 bytes.

As for how to convert binary component ID to text component ID and back again:

function binid_to_hexid(id)
  local f, r = string.format, string.rep
  return f(f("%s-%s%s", r("%.2x", 4), r("%.2x%.2x-", 3), r("%.2x", 6)), id:byte(1, 16))
end

function hexid_to_binid(id)
  local lasthex = 0
  local match = ""
  local bstr = ""
  for i=1, 16 do
    match, lasthex = id:match("%x%x", lasthex)
    bstr = bstr .. string.char(tostring(match, 16))
  end
  return bstr
end

Otherwise, this looks pretty good.

[Zen1th 14]

Thanks for the review. I will make a version 2 (on same post) to add the changes you suggested and some others changes. I will also soon make a more presentable form of the specification.

[Adorable-Catgirl 11]

So, some recommendations for OEFI v2:

• Storage of addresses in a binary format
• Maybe a standard set of commands for an OEFI shell?
• "oefi.loadfile(path)", which loads a file from the boot device
• Vendor prefixes on extensions to OEFI (ie "oefi.vendor.extensionmethod()")
• BIOS configuration space of 64 bytes, at the end of the EEPROM
• Maybe a network boot protocol?
• The OEFI library should be global unless booting in compatibility mode (for example, from an init.lua file)
• OEFI implementation must allow the OS to return
• Standard for passing kernel arguments (For things like Fuchas NT or Tsuki)
• Maybe a sample OEFI implementation?
• Must support compatibility mode, though maybe there can be an option to disable compatibility mode.
• Maybe a URF-based bootable package?

[Zen1th 14]

Nice proposals, i'm still making the draft for version 2

The URF-based bootable package is a very good idea, however this would cause problem since the only implementation for writing/reading URF is uncertain of actually being standard and is only available from Lua from OC.
WIth my knowledge i could also make an application to extract/package URFs, but who would trust it anyways? So the best option is: Making an dedicated C/C++/Java app/library, finding a known archive extractor that support plugins. So based on my existing work and on the specifications, gotta try to make those

[Adorable-Catgirl 11]

Fair. Also, we could use binary CPIO for bootable packages. That would probably be the best bet. It's standard and been around for a while.

 

Also, OEFI v2 applications should end in `.efi2` to differentiate them from the OEFI v1 applications. OEFI v2 applications should also probably support a few basic arguments.

[Adorable-Catgirl 11]

.efi2 could probably be a CPIO archive. Entry point can be defined by a .cfg file in the root of the archive. Or, we could also check if it's a CPIO file or a Lua file. Dunno, it's up to you.

[Zen1th 14]

I decided to instdead use URF, sure it's less supported, but it is more adapted to OpenComputers, as it is made for OpenComputers

[Zen1th 14]

I just finished OEFIv2

[Adorable-Catgirl 11]

URF IS GONE

[Zen1th 14]

i know it was for "simplicity" reasons