Abstract

This document provides guidelines for dealing with EEPROMs and locating architecture-specific boot code.

Rationale

OpenComputers supports a wide variety of architectures. Even more so than in the real world, OpenComputers architectures can differ dramatically from one another. Some architectures run programs in a particular high-level language directly, while others simulate real or fictitious low-level ISAs. Some architectures natively deal with data in 8-bit units, while others have built-in advanced string handling and vector processing capabilities.

In contrast with this variety, OpenComputers components have a standard interface. An EEPROM containing boot code can easily end up being used on the "wrong" architecture, to say nothing of boot disks.

This standard aims to solve that problem, by providing architecture-aware guidelines for dealing with EEPROMs, and procedures for locating boot code on filesystem-based and sector-based boot media. It aims to be simple to implement on the widest possible variety of architectures.

Conventions

Unless otherwise specified, all references to text imply 7-bit ASCII codes. Behavior on encountering bytes with the high-bit set is undefined.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Architecture Identifier

Each architecture must provide a unique, preferably meaningful identifier which is specific to that architecture. This is the Architecture Identifier, or AID. The AID SHOULD be the same as the Architecture.Name annotation for the architecture, which is usually the same as the name used in the tooltip of the CPU.

An AID MUST contain no bytes other than the following characters: digits, upper and lowercase letters, periods (.), dashes (-), underscores (_), slashes (/), spaces. In addition, an AID MUST NOT begin with a space, end with a space, or contain a run of two or more spaces. An AID SHOULD begin with a capital letter, if for no other reason than to make boot code easier to tell apart from other files and directories in listings.

A new AID SHOULD NOT contain spaces, unless it is required for compatibility.

An architecture with multiple variants that are mutually incompatible SHOULD use different AIDs for each variant. Fallback schemes, such as one where multiple AIDs are tried, are architecture-specific and outside the scope of this document.

CAB-aware EEPROM images ("CABE images")

This section applies to architectures, EEPROM flashing utilities, and any hardcoded boot code (analogous to machine.lua in the "standard" Lua architecture).

A CAB-aware EEPROM image (a "CABE image") MUST begin with "--[", followed by zero or more "=", followed by "[CABE:". This is the prefix string. A CAB-compliant architecture MUST be prepared to deal with any number of "=" between 0 and 7, and a CABE image SHOULD NOT use more than 7.

The prefix string is followed by a single AID. This denotes the "intended architecture" for this CAB image. This is followed by one of two things:

• A colon (:), in which case the "main body" consists of the bytes following the colon, up until a valid suffix string is encountered.
• A suffix string, in which case the "main body" consists of the entire remainder of the image, and MUST be valid Lua 5.2 AND Lua 5.3 code.

A "suffix string" consists of an ASCII "]", followed by the same number of "=" as were in the prefix string, followed by another "]". In order to be a valid suffix string, it MUST contain precisely the same number of "=" as the prefix string contains. In particular, an otherwise valid suffix string that contains more "=" than the prefix string MUST be treated exactly the same as any other non-suffix-string byte sequence.

The "main body" contains the actual EEPROM data for that architecture. The interpretation of this data is architecture-defined.

If a CABE image contains data after a valid suffix string, that data MUST be Lua code which is cross-compatible between the Lua 5.2 and 5.3 architectures provided by OpenComputers. If this is not the intended architecture of the CABE image, this code SHOULD consist solely of an informative error call.

Any deviation whatsoever from this standard results in a non-CABE image. All handling of non-CABE images is architecture-specific, but a non-CABE image SHOULD be treated however the "main body" of a valid CABE image would be treated.

The preferred file extension for CABE images is ".cabe". The preferred MIME type is "application/x-cabe-image", though "application/octet-stream" is acceptable. CABE images MUST NOT be distributed using any "text/*" MIME type, as doing so will almost certainly corrupt binary CABE images.

Here is an example CABE image targeting a fictional architecture:

--[[CABE:HyperTalk:
ask "What is your name?"
answer "Hello," && it & "!"
]]
error"HyperTalk architecture required"

And here is one targeting a built-in Lua architecture:

--[[CABE:Lua 5.2]]
for n=1,5 do
  computer.beep(2000, 0.1)
end

EEPROM-based transparent architecture switching

A future OpenComputers release may add support for transparent architecture switching, through additional NBT data for EEPROMs. It is expected that this will consist of a single AID identifying the architecture the EEPROM is designed for. This section applies in the event of such support becoming available. To provide consistency to users, architectures SHOULD NOT attempt to implement any form of automatic architecture switching themselves.

An EEPROM flashing utility SHOULD attempt to parse all images as CABE images. If successful, it SHOULD tag the EEPROM appropriately and burn only the "main body". Otherwise, it SHOULD remove any existing architecture tag from the EEPROM and burn the entire image.

An architecture booting an EEPROM with a valid architecture tag SHOULD NOT also attempt to parse it as a CABE image. An architecture booting an EEPROM with no valid architecture tag SHOULD attempt to parse it as a CABE image, and MUST NOT affix an architecture tag itself.

Boot code

This section applies to EEPROMs intended as first-stage bootloaders, as well as programs intended to be booted by such EEPROMs.

Bootloaders SHOULD, if the boot EEPROM contains a boot device UUID, attempt to boot from that device first. A boot EEPROM contains a boot device UUID if eeprom.getData() consists entirely of an ASCII UUID, or if it begins with an ASCII UUID followed by a null byte.

Managed mode (filesystems)

Bootloader behavior on managed filesystems:

• If "/<AID>" exists and is a directory, boot "/<AID>/boot".
• If "/<AID>" exists and is a file, boot "/<AID>".
• Any further cases are architecture-specific.

If one of the above conditions are met, but its booting attempt fails, the booting process MUST NOT continue automatically. For instance, if "/<AID>" is a directory but booting "/<AID>/boot" fails, the bootloader MUST either fail with an error, or prompt a user for further action.

Exactly what "booting" entails is architecture-specific. On a Lua architecture, it consists of loading a file as Lua code and then executing it. On low-level architectures, it might consist of loading a file's contents to a fixed RAM address and jumping into it. Architectures SHOULD provide a standard way for the first-stage bootloader to tell the booted code the UUID of the filesystem it was loaded from.

Example: Consider a boot on the OC-ARM architecture. The bootloader checks if "/OC-ARM" exists. It does exist, and is a directory. The bootloader then attempts to boot "/OC-ARM/boot". It fails, because "/OC-ARM/boot" is not valid. It crashes the machine with an error message explaining the problem.

Unmanaged mode (drives)

A CAB-compliant bootable disk begins with a boot sector. This boot sector MUST be the first or second sector of the drive. If both the first and second sectors contain a valid boot sector, only the first one will be used. A boot sector begins with the ASCII string "CAB", followed by zero or more text boot records. This list of text records is terminated with an exclamation mark. If this exclamation mark is followed by the particular byte sequence {0x00, 0x1A, 0xCA, 0xBD} (null byte, CP/M end-of-file marker, two-byte magic number), then it is followed by zero or more binary boot records, terminated by a null byte.

Boot records MUST NOT extend past the end of the boot sector. Architectures MAY specify that boot records for that architecture must be text or must be binary, and MAY specify that binary boot records must be a particular endianness and/or must be sector-aligned. Bootloaders for architectures that do not specify that boot records must be text or must be binary MUST support both.

A text record matches ":<AID>=<offset>+<length>". <AID> is the AID for which the code is intended. <offset> is a decimal number, giving the byte offset at which to begin reading, OR "s" followed by a decimal number, giving the sector number at which to begin reading. <length> is a decimal number giving the number of bytes to read.

A binary record is described by the following C99 structure:

struct {
  uint8_t record_length;
  uint8_t flags;
  uint16_t load_start;
  uint32_t load_length;
  char aid[];
};

record_length is the number that must be added to the offset of this record to skip it. It MUST be equal to 8 + AID length + 1.

flags is a bitfield. The following flags are defined:

• 0x40: If set, load_start is a sector number. If clear, load_start is a byte offset.
• 0x80: If set, load_start and load_length are little-endian. If clear, load_start and load_length are in network byte order (big-endian).

load_start is either a sector number or a byte offset to begin the loading process at. load_length is the number of bytes to read. aid is the AID of the intended architecture, and MUST be null-terminated.

As with managed mode, exactly what is done with the loaded data is architecture-specific. Bootloaders that only support binary records should consider a sector to be a valid boot sector if it begins with "CAB", and locate the end of the text boot records without parsing them by searching for the first "!". Bootloaders that only support text records need not consider any bytes past the first null byte. A boot sector that contains no records is valid, and MUST prevent any attempt to read possible subsequent boot sectors.

Example 1:

CAB:Lua 5.2=s3+17:Lua 5.3=s3+17:HyperTalk=384+5100!
(followed directly by binary data:)
001A CABD (valid binary records follow)
0F (the length of the first record)
C0 (little-endian, sector offset)
0900 (start at sector number 9)
0000 0100 (load 65536 bytes)
5342 3635 3032 00 (null-terminated string "SB6502")
00 (no more binary records)

This drive contains valid boot code for Lua 5.2, Lua 5.3, HyperTalk, and SB6502. Lua 5.2 and 5.3 both use the same boot code, which is 17 bytes long and starts at the beginning of sector number 3. The HyperTalk boot code is 5100 bytes long and starts 384 bytes into the disk, which, when using 256-byte sectors, is halfway through the second sector. The SB6502 boot code is 65536 bytes long and starts at the beginning of sector number 9.

Example 2:

CAB!

This is a valid boot sector, but contains no boot records.  This is the safest way to mark a drive as non-bootable.

Concepts

• Socket: A file on the system which links two programs' communications.
• OC: Abbreviation for "OpenComputers"
• OS: Abbreviation for "Operating System"
• OpenOS: The stock OS for OC computers.
• Socket: a medium for communication between two or more programs

Rationale

To standardize any possible IPC implementations

Basic Concepts

• Background programs (or daemons) can be started with a simple wrapper, such as this example code:

thread.create(function()
	shell.execute(ENV, "/bin/daemon.lua")
end)

• Programs can communicate with sockets through the signals defined in this document
• These signals can be created with "computer.pushSignal(name: string, [...])"
• They can be received with "event.pull([timeout: number], [name:string], ...)" or "event.listen(event: string, callback: function)", or on a low-level basis with "computer.pullSignal([timeout: number])"

Signals

oipc_sock_comms(socketID: string, <... (message)>) -- OIPC Socket Communications
-- message must be of type nil, boolean, number, string, table (tables and other messages MUST NOT contain threads and functions due to the limitations of computer.pushSignal())

Notes

Please note that OIPC will only work with OSes which pass signals to all subprocesses instead of simply consuming the signal on "computer.pullSignal", "event.pull", or similar functions to wait for/listen for signals.

This is a copy of: https://github.com/GlobalEmpire/OC-Programs/blob/master/Reserved OC network card ports.txt

If you would like to reserve a number, please do any of the following:

1. Post in this thread

2. Message MajGenRelativity#4971 on Discord

3. Message MajGenRelativity on irc.esper.net

Currently Reserved Ports:

Port 14: Ethernet over OC: Reserved by SolraBizna

Port 148: GUI service: Reserved by Dustpuppy

Ports 324-325: Stargate Dialing Service: Reserved by Demokela

Port 4096: Minitel: Reserved by Izaya

Ports 4378-4379: GERTi: Reserved by Global Empire

Port 4662: Short messages: Reserved by Dustpuppy

Port 9100: Network Print Service: Reserved by Dustpuppy

Port 9900: Zorya BIOS LAN boot: Reserved by awoo

Definitions:

Host - Any computer or machine with a OC-compatible modem attached to it

Server - Host sending OHTP-valid data when an user attemps to connect

Client - Host receiving OHTP-valid data from a Server

OHTP - Open Hyper Text Protocol - Protocol used to transmit OHML - Open Hypertext Markup Language between Hosts.

Line Break - Unix style line break: line feed (\n)

Specifications:

The OHTP protocol doesn't take care of what is transporting it. Either it is using a socket API, going through modem, minitel or GERT. It assumes the API or transport layer you utilize supports sockets.

Arguments in URLs:

To put arguments inside URL, the url must end like normal, but have "?" at end, followed by "property=value" entry. If wanting to add more entries, add "&" after last entry

Example:

ohtp://1001.1002:1003.1004/index.ohml?username=admin&password=admin

writeTable/readTable, Fuchas's liblon, etc.

User Agents:

User agents are standard strings defining what browser and what layout engine a user is using. It can be used for metrics and/or compatibility purposes (in case standards aren't fully respected).

An user agent is presented in the following form:

BrowserName/BrowserVersion, LayoutEngine/LayoutEngineVersion

where LayoutEngine is the layout engine and LayoutEngineVersion is layout engine's version, etc.
The format has been thinked to be simple making it easy to parse. Since it's just 2 fields separated with a "," (Browser name and version + Layout engine name and version), and in those fields, the name and version separated with a slash (/).
Parsing user agent is in this case, just some splitting of strings.

For example with Icecap browser (coming soon):

Icecap/0.1, Geeko/1.0

Peanuts:

Peanuts are data stored as text files in a key=value scheme.
They are handled on client-side and are only edited by server. He edits it by sending a response property (Set-Peanut) to client.

A Peanut Group (peanuts) is separated by ";". Making ";" the only invalid character in peanut key and value.
Server is aware of Client peanuts by the Peanut request property. Which sends peanuts as a Peanut Group. Of course this property is optional as a client doesn't always have peanuts.

Now the most important note about peanuts: When a server send a Set-Peanut property. The peanut must be set for the whole website and not only the current page!

Connection:

OHTP DEFAULT Port SHOULD be 80

Client-Side:

Once the socket is opened, the Client can ask data to Server

The Client will send first a Request Header looking like it:

OHTP/MAJOR.MINOR REQUEST PAGE

Example, if the REQUEST is GET, and the version 1.0, with a request page equals to /foo.bar (the PAGE entry CANNOT BE EMPTY, if wanting to access default file of server, use /):

OHTP/1.0 GET /foo.bar

Currently the valid REQUEST values are:

• GET (just receive the file)
• UPLOAD (part of upcoming OHTP 1.1 document, will be used to upload files if the Client has permissions)

There are also request bodies, which just include "Property: Value" entries.

Non-standard properties, must start with "X-"

Standard Properties:
Standard properties are for now:

• Peanut - Inform server of currently saved peanuts (see Peanuts)
• User-Agent - User Agent of the current browser (see User Agents)
• Referer - If redirected from a link, equals to the previous website URL.
• Host - The hostname the browser used to connect to this website. This let the website know how it is identified. Useful for 300/301 error codes.

So a request is constructed like that: Request Header( + Request Body)

The client, even after having received the Response from Server can still send new requests. All that until socket is closed.

Example Request (whole):

OHTP/1.0 GET /index.ohml
User-Agent: MineScape/1.0, Geeko/0.9
Peanut: login=true;username=admin;password=admin
X-Non-Standard-Header: non standard value

Server-Side:

Once the Server receives a Request from a client. It must replies with a Response.

A Response is contructed like this: Response Header + Response Content

The Response Header is constructed like this:

ERRORCODE
Response Body

Response Content

It's really small, making it simple to implement.

Currently there are thoses error codes:

• 200: OK
• 300: Temporaly moved + new URI after ERRORCODE. The browser will redirect to the URI but is unable to cache.
• 301: Moved permanently + new URI after ERRORCODE (ex: 301 ohtp2://thismachine). The browser will redirect to the URI and can cache it. Best used with Host property.
• 302: Switch protocol (e.g.: 301 OHTP2) + approved protocol name after ERRORCODE
• 303: Switch network protocol (e.g.: 302 minitel) + approved protocol name after ERRORCODE
• 403: Invalid request
• 404: Page not found
• 500: Server error
• 501: Internal error sending the the fetched page (e.g. file found but not accessible from server).
• 502: Server-side language is errored (Detailled version of 500, optional)
• 503: Service temporaly down

The Response Body is just a bunch of standard properties exactly like in the client-side request body. However the valid properties changes for Server.

Standard Properties:

• Set-Peanut: Sets a peanut on client.
• Content-Size: The size of the content. Useful for long downloads!
• Content-Type: The MMTI identifier for the corresponding resources. If not found, defaults to "text/txt"

Like Request properties, unstandard ones start with "X-"

The Response Content is just the content of the fetched file (if found).

THE RESPONSE CONTENT MUST BE SEPARATED FROM RESPONSE BODY WITH TWO LINE BREAK (\n\n)

Example Response (whole):

200
Set-Peanut: isadmin=true
Content-Type: text/ohml

<ohml>
  <text>Hello World!</text>
</ohml>

FOR LUA ARCHITECTURE, OEFI PRESENTS LITTLE TO NO ADVANTAGES!

Lua BIOSes that implements OEFI must allow using conventional booting.
Note that OEFI has been made to be more architecture-independent and will support ANY architecture able to read files and CPIO (basically any architecture with basic access to component and bitwise operations).

Concepts

• Application: Executable code booting using OEFI in the EFI1 or EFI2 format.
• Conventional Booting: OpenComputers default booting method. Often called "BIOS"
• Operating System (OS) : Complex Application with user interaction and/or providing an environment for programs. Concept only used in this document for Conventional Booting
• OC: Shortcut for "Open Computers"

Rationale

Due to the limitation of current OC booting method which basically searches for an init.lua file, the most big problem is that this only work for Lua architecture, and what should other architectures use, init.asm? init.test? init.o? What happens if we want one drive to contains multiple inits for multiple architectures? Well that's where Open Extensible Firmware Interface comes handy. Using . The limitations OEFI fix in Conventional Booting in OC are:

• In Conventional Booting, there is only one init script per filesystem, while in OEFI there can be multiple applications per filesystem.
• Conventional Booting doesn't have any native support for multiple applications. An OS that want to boot another one need to manually check for init.lua files in all filesystems, OEFI has oefi.getApplications() method for it, this is even worse for Applications booting on unmanaged drives
• Conventional Booting isn't made for any other architecture than Lua. And with it, like addressed below, it's impossible to have multiple applications for multiple architectures, while OEFI allows in theory to have one same drive to boot on all architectures it support, without tweaks needed from the user!

OEFI is basically a much more extensible version of OETF #1, « Standardized booting of nonstandard architectures » and Conventional booting.

2.1 Changelog

• Changed EFI2 format into a properties file for easier parsing
• Using CPIO instdead of URF for same reason: easier parsing
• Added OEFI Extensions, oefi.loadInternalFile(path) and oefi.setApplications(appTable)
• Fixed having put AwesomeCatgirl instdead of AdorableCatgirl as username for CAP conntribution

Application Table Format

The application table must contains tables as entries.
The tables must contains the drive address with as « drive » entry and the path (from drive) as « path » entry

Example:

{
  {
    drive = "af58dbe2-ff8c-433a-a871-c08723e01f25",
    path = ".efi/sampleos.efi"
  },
  {
    drive = "af58dbe2-ff8c-433a-a871-c08723e01f25",
    path = ".efi/osOnSameDrive.efi2"
  },
  {
    drive = "015f9d4c-cdfb-4686-91cb-bc8cf997f4ec",
    path = ".efi/another_os.efi"
  }
}

For architectures not supporting hash maps / dictionnaries, "drive" is index 0 and "path" is index 1

For architectures not supporting arrays inside arrays (2D arrays), the arrays should be appended in such a way it look like that: drive, path, drive, path, drive, path, ... . This is the default standard used by C for multi-dimensional arrays. If the language supports multi-dimensional arrays then the arrays of the language must be used.

Finding Applications

OEFI Applications can be found on any sort of drives, from disk drives to floppy, with raids, only if they contains a ".efi" directory.
The OEFI should search files in ".efi" directory:

If the file ends with .efi, then check if it contains CPIO signature, if no, boot in compatibility mode (OEFI1, which is not described anymore on this document).
However if the same file contains that signature, then boot as normal.
.efi2 will ALWAYS be in standard EFI2 format, unlike .efi which can be EFI or EFI2.
.efi can be EFI1 or EFI2 to keep compatibility with old OEFI Applications, and to allow 8.3 filesystems to still support OEFI (would be problematic otherwise)

API Methods

Methods are same as in Version 1, of course API version is now 2 (logic), and some new methods are being added, so here is the new list of methods:

• oefi.loadfile(path) - Loads file from boot drive
• oefi.loadInternalFile(path) - Loads file from current archive - new in OEFI 2.1
• oefi.getBootAddress() - replaces computer.getBootAddress()
• oefi.getApplications() - Return an application table as described above
• oefi.setApplications(appTable) - Set the application table (as described above) to appTable. - new in OEFI 2.1
• oefi.getAPIVersion() - Return API version number. Returns float, double or any decimal 2.1 for this version
• oefi.getImplementationName() - Returns implementation name
• oefi.getImplementationVersion() - Returns implementation version number
• oefi.returnToOEFI() - Recalls the OEFI program, the expected behavior is that the implementation will try to re-execute itself.
• oefi.execOEFIApp(drive, path) - Boot another OEFI app by using the implementation's routine
• oefi.getExtensions() - OEFI EXTensions is for non-standard features from implementation. This should always be used for implementation methods. The way this work is described below - new in OEFI 2.1

For architectures supporting tables (and global), API must be available in Global
Else, API must be available as a pointer/struct/class/anything like that available as argument to the code

OEFI Extensions work in the following way: If the architecture supports entries with keys, then a entry should be created containing the function (or its pointer), and the key should be equals to ImplementationName_MethodName_MethodVersion, ImplementationName should be same as in oefi.getImplementationName(), MethodVersion should be the revision number of a method, starts at 1. And MethodName is the name of the method. Example: SuperOEFI_DrawHandSpinner_2, which could be called (in Lua) via oefi.getExtensions().SuperOEFI_DrawHandSpinner_2()

If tables are not supported, then the array (returned by oefi.getExtensions()) should contain all the methods as other arrays. So oefi_ext will be a 2D array. Each array should contain the name (see above) at the first index (0 or 1, depends on architecture),  and a pointer to the function (or the function itself, if possible) at second index.

If 2D array too aren't supported, then like Application Table, the array should act like if they are appended to each other.

Please also note that if possible, computer.getBootAddress() and computer.setBootAddress(addr) should be REMOVED! This is due to that Applications supporting OEFI doesn't need any compatibility methods as thoses conventional booting methods are only kept when booting a OS that only support conventional booting.
Implementation can create their own methods, however it should ALWAYS BE INSIDE oefi.getExtensions(). (see above)

EFI2

EFI2 is a new format for Applications, it replaces EFI(1) in , it is a CPIO archive. All files in that archive must be placed at root and are listed here:

• app.cfg
• app.exe

app.exe is only the Application, it contains the code designed for target architecture, and will be launched by the OEFI .

app.cfg is a configuration file using "key=value" scheme. The file should look like this:

name=Application Name
version=1.0
oefiVersion=2.1
arch=Lua
archMinVer=5.2
archMaxVer=5.3

Space after key or before name is parsed as it and aren't ignored, be careful!

"name" is equals to the name of the Application. Will always be a text
"version" is equals to the version of the Application, if you don't want to fill it, just make it stay to 1.0. Will always be a decimal
"arch" is a string with the name of supported architecture, "archMinVer" and "archMaxVer" are self-explanatory and will always be a decimal.
Note that for archMinVer and archMaxVer, -1 can be used if a version doesn't make sense to the architecture to have a version. (example: a 6502 architecture)
"oefiVersion" is equals to the OEFI version the Application has been designed to run with, if the "oefiVersion" field have a version incompatible with the current API version or if it's higher than the current one (ex. it's equals to 3 but we're on API v2.1, or it's equals to 1 but we're on API v2). This will also always be a decimal

With all thoses fields in mind, the EFI2 was designed to be durable, with only changes being to app.lon and URF version.

Configuration Data

Everything is explained in that image:

Component Address Packing

To shrink down components to 16 bytes, since components address (and UUIDv4 in general) are just hexadecimal numbers converted to string, we can re-convert the string to a byte array and vice-versa.
To do it, the lines (-) must be ignored, and each group of 2 characters must be interpreted as a hexadecimal string, this should be easy with architecture supporting string to number with optional base (in our case base is equals to 16)
For example it turns:

68ca0f59-ac2c-49f8-ba6a-3c4c7e5f069b

into:

{0x68, 0xCA, 0x0F, 0x59, 0xAC, 0x2C, 0x49, 0xF8, 0xBA, 0x6A, 0x3C, 0x4C, 0x7E, 0x5F, 0x06, 0x9B}

Here is a code sample submitted by AdorableCatgirl to encode and decode the addresses:

function binToHex(id)
	local f, r = string.format, string.rep
	return f(f("%s-%s%s", r("%.2x", 4), r("%.2x%.2x-", 3), r("%.2x", 6)), id:byte(1, 16))
end

function hexToBin(addr)
	addr = addr:gsub("%-", "")
	local result = ""
	for i=1, #addr, 2 do
		baddr = result .. string.char(tonumber(addr:sub(i, i+1), 16))
	end
	return baddr
end
-- example: hexToBin("68ca0f59-ac2c-49f8-ba6a-3c4c7e5f069b") == {0x68, 0xCA, 0x0F, 0x59, 0xAC, 0x2C, 0x49, 0xF8, 0xBA, 0x6A, 0x3C, 0x4C, 0x7E, 0x5F, 0x06, 0x9B}
-- "68ca0f59-ac2c-49f8-ba6a-3c4c7e5f069b" == binToHex(the array below)

Back to EEPROM data, implementation name and custom configuration are very easy to understand what they're about.

Multipurpose Media Type Identifier, Version 1.0

Purpose

The MMTI provides a way to identify data types without an extension, and to differentiate files with the same extension. One can only make so many recognisable extensions with 4 latin characters. Some data is not to be used as a file, but still needs a way to be identified.

Format

An MMTI string takes the format of category/type. For example, a Lua script would be application/lua. A CPIO archive would be archive/cpio, etc. MMTI strings should only use US-ASCII characters.

Unregistered MMTI strings

MMTI strings that aren't registered should prefix the type with x-, such as application/x-zoryapkg.

Categories

As of Version 1.0, the following categories are available for use.

• application - Any type of format which is to be executed
• archive - Any type of format which is used to archive other files
• media - Any type of format which contains video or audio
• model - Any type of format which contains data used to construct a 3D model
• text - Any type of format which is used to transmit textual data
• message - Any type of format which is used to transmit data over a network
• other - Any type of format which is not covered in the other categories

Registed MMTI types.

OHML is a markup language based on the eXtensible Markup Language (XML), it is a simpler and OC-graphics adapted version of HTML.

Why OHML?
It all started when i noticed that most network programs/libraries were either DNS system, or Website system.
However it is very important to maintain an universal language for webpages. It is the first step of having
a stable and universal "OCranet", which will follow with server and client implementations. Protocols, etc..

Concepts

• HRef: Shortcut for Hyperlink Reference

Document Versions

OHML patch versions (1.0.1, 1.0.5, etc.) minor changes (more understandable descriptions, new optional arguments) can be asked in comments section, might get accepted, and when features are froze, set as a new minor version of OHML specifications.

OHML minor versions (1.2, 1.1, 1.42) OHML major versions (2, 4, 5) are to be proposed in comments sections and feature froze. Once released they will replace the outdated data in this post.

Tags

Table about all tags and their (XML) arguments is available in png format, attached to this topic.

h1-h5

6-2 sized OC character font title, each letter must be made from the number of character both in width and height. The font can be any font, as long as it respects the size rule. This would usually be made using a big font library.

text

This tag allows to display a text line. If putting the text in the plain document, due to the XML nature line breaks wouldn't be counted. In the text tag they aren't counted neither, but at each tag end, a line is skipped.

script

This contains any script made for any scripting language supported by the Browser. The attribute "lang" defines the scripting language, it defaults to "application/lua" for LuaWeb, however it remains an XML tag and will not work well with multi lines, the solution is CDATA, simply add a comment and add <![[CDATA, then at the end of script do another command and put ]]> , this will effectively allow the use of multi-line scripts.

link

Displays inline text, it haves, by default, a blue color. When it is clicked the page jumps to the hyperlink reference defined by "href" attribute.

image

Contains one attribute: "file". This is the path of the image treated as an Hyperlink reference.

br

Breaks a line, which mean that a text line is skipped, this act the same as <text></text> but cannot contains any text.

button

Clickable button with text being inside the tag. The text doesn't support OHML but will still be parsed as XML (a CDATA section is then recommended).
Any <script> tag can set a listener on the button.

"onclick" listener

Arguments: mouseButton
mouseButton is "left" for left button, "middle" for middle button and "right" for right button.

box

Container. It can contains any possible tags in OHML. It can be used for styling the element's bounds are fully customizable using Positioning, and while by default it is transparent, using Styling this can be used as a resizable background color.

Tags Attributes

Positioning

Introduced in OHML v1.0.2, Positioning allows to position tags in an absolute or relative way.

Tags have optional arguments "x", "y", "width", "height" and "relative"

The "relative" argument is for using relative positions, vertical at first, and horizontal at last, we can use "up" (default) or "bottom", put a ";" for splitting, and add the horizontal value that can be "left" or "right".
Relative positions works that if for example the value is "bottom;right", and if x = -5 and y = -5, the element Y will be at the most bottom point of the page (meaning that bottom for a page with elements not going after 10 for y, the bottom would be 10), added -5, which is equivalent of minus 5, and the element X will be at most-right point (generally viewport width) and will have minus 5.

Meaning that for a page sizing 80;25, it would go at 75;20

Styling

Introduced in OHML v1.0.3, this allows elements to be stylized in little ways. This should eventually be superseded by a dedicated styling language.
The styling addition introduces many attributes, these attributes are also by default transmitted to the element's childs

bgcolor

This attribute takes an hexadecimal RGB value and sets the background color of an element. The background is effective on the bounds of the element, which are its absolute x, y, width and height.

color

This attribute takes an hexadecimal RGB value and sets the foreground color of an element. Like bgcolor, the background is effective on the bounds of the element.

Hyperlink Reference
An hyperlink reference can be relative or absolute.
If a HRef starts with any supported protocol name followed by "://" (ex: ohtp://), the link is fully absolute and will replace the current address. It must be an URI.
Otherwise, if an HRef starts with "/", it is relative to the website host (e.g: ohtp://test.com), so, the full path is: {WEBSITE HOST} + {HREF}
Finally, starts with nothing above, it is relative and should be appended to the actual URL

URI are in the same format than real ones which is "protocol://host(/page)"

OHML, being a markup language, will not support any kind of dynamic coding like <if> statements, <print> statements, etc., dynamic coding is handled by <script> tags and scripts languages supported by the browser. (Currently no script language has been made, it's coming).

For now one implementation has been made and it is the reference one called Minescape.

URF v1.1

Abstract

This document describes the URF, intended to make mass file exchange easier.

Rationale

There are many competing methods of exchanging large amounts of files, and many are incomplete, such as TAR implementations, or proprietary, such as NeoPAKv1. With this in mind, a standard format will make file exchange less error prone.

Conventions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

All strings are encoded with UTF-8, prepended with an ALI corresponding to the length of the string in bytes.

Character specifications such as NULL and DC1 are part of US ASCII, unless otherwise specified.

Concepts

• Signature: Data at the beginning of the File, marking the File as a URF format archive, and specifying the version.
• File Table: Data structure containing all file information.
• Entry: Any data sub-structure in the File Table.
• Entry Specifier byte: A byte describing how to decode the data contained in an Entry.
• Object: A filesystem structure, i.e.. "file" or "directory".
• Attributes: Data describing the size, offset, parent Object, and Object ID of an Object.
• Extended Attributes: Data describing non-core attributes such as Permissions, Owner, Security, etc.
• Producer: A program or process that generates data in this format.
• Consumer: A program or process that consumes data in this format.

Arbitrary length integers

Arbitrary length integers (ALIs) MAY be over 64-bits in precision. ALIs are little endian. For each byte, add the value of the first seven bits, shifted by 7 times the number of characters currently read bits, to the the read value and repeat until the 8th bit is 0.

Signature

The Signature MUST be URF (US-ASCII) followed by a DC1 character, or an unsigned 32-bit little endian integer equal to 1431455249. The next two bytes MUST be the version number, the first being the major version, the second being the minor version. The next two bytes MUST be DC2 followed by a NULL character.

File Table

The File Table MUST start after the Signature. This MUST contain all Entries, and MUST end with an EOH Entry. See EOH Entry.

Entry

An Entry MUST start with an Entry Specifier byte. The Entry Specifier byte MUST be followed with an ALI specifying the length. Data contained SHOULD be skipped if the Entry Specifier byte allows and the Entry Type is not understood.

Entry Specifier byte

An Entry Specifier byte MUST have the 7th bit set to 1. If the Consumer comes across an entry with the 7th bit not set to 1, the Consumer MUST stop reading the file and raise a fatal error. The 6th byte specifies if the entry is critical. The entry is critical if the 6th bit is set to 0. If the entry is critical and not understood, the Consumer should raise a fatal error; if the entry is non-critical, the Consumer SHOULD skip the entry and continue reading the file. If the 8th byte is set to 1, the Entry is a non-standard extension. Vendors MAY use this range for vendor-specific data.

Filesystem Structure

Any Object MUST have a Parent ID and an Object ID. Object ID 0 is reserved for the root directory.

File naming conventions

Object names MUST NOT contain any type of slash. Object names must also be of the 8.3 format, though the full name MAY be specified with Extended Attributes. Full names in Attributes MUST NOT contain any type of slash.

File offsets

File offsets are relative to the end of the File Table.

Entry Type: File

The Entry Specifier byte MUST be F, and the data contained MUST be the name as a string, followed by the file offset and file size represented as an ALI, then followed by the Object ID, then Parent ID.

Entry type: Directory

The Entry Specifier byte MUST be D, and the data contained MUST be the name as a string, followed by the Object ID, then Parent ID.

Entry type: Extended Attributes

The Entry Specifier byte MUST be x, and the data contained MUST be the Object ID of the Entry the Entry is describing, followed by a four byte Attribute and the value.

Currently recognized attributes include, names in US-ASCII:

• NAME: The long name of the Object (String)
• PERM: The POSIX-compatible permissions of the Object (Unsigned 16-bit integer)
• W32P: Win32-compatible permissions of the Object, which override POSIX permissions (Unsigned 8-bit Integer, Read-Only [0x01], Hidden [0x02], System [0x04])
• OTIM: Creation time of the Object (Unsigned 64-bit Integer)
• MTIM: Modification time of the Object (Unsigned 64-bit Integer)
• CTIM: Metadata update time of the Object (Unsigned 64-bit Integer
• ATIM: Access time of the Object (Unsigned 64-bit Integer)
• SCOS: Source OS of the Object (String)

Entry type: EOH

The Entry Specifier byte MUST be Z, and the data contained MUST be the offset required to reach the end of the file.

Compressed URF naming convention

In an environment with long names, the file extension SHOULD be urf followed by a period (.) and the compression method (i.e. gz, lzma, xz, deflate) In an environment with 8.3 names, the file extension MUST be one of the following:

• UMA for LZMA
• UXZ for XZ
• UGZ for Gzip
• UL4 for LZ4
• UB2 for BZip2

TODO

Document is incomplete. Document should outline how to build the Filesystem structure, etc. Document should be checked for clarity and rewritten if needed.

 and even attempted to develop

An advanced networking system for OC. I failed before, but I feel I now have the experience and knowledge to truly pull it off (thanks to working in the field). So, I would like to re-introduce the Unet Project!

The Unet framework is a full stack IP routing solution, featuring 64 bit IP address and subnet masks, VLANs, MAC (modem address) aware interfaces and the ability to be filtered by routers.

The key behind Unet is its abstraction layer, aggregating all communications media (Loopback, Wired Modems, Wireless Modems,  Tunnel Cards, etc) into an interface, completely independent of the media. This vastly simplifies the transmission process. 

These interfaces are then farther hidden behind the basic send command. When presented with an IP address, will automatically resolve which media to use and how in the following process.

1. If an interface with this address exists on the local system, the message is immediately looped back on behalf of the interface.
2. If an interface resides on the same subnet as this address,  it will be used to transmit the message
3. If a route entry is found in the local route table that points to the subnet in which this interface resides, the routing information will be used in transmission
4. If a default route is found, it will be used
5. If no route is found, a route not found message will be looped back.

In addition, an advanced series of router operating systems, both for microcontrollers (using side based transmission control) and full-fledged servers (using modem-side bindings and tunnel cards) Will allow massively scaled networks.

In addition to the Unet IP addresses, the headers will have space for auxiliary transport layer info. An int will be used for Source IP addresses, a second int will be used for Destination IP Addresses, and a third int will contain an 8-bit hop counter, and a 16-bit protocol flag showing what the packet is carrying. At this time there is no other data, and comments are more than welcome regarding what to do with the extra space

The project is still very early stage, as I work on it in my spare time at home, but should there be enough interest in this (Admittedly competing) system, I have however committed the code to Github and loaded the WiKi with information about the current designs. Anyone is welcome to discuss this with me! https://github.com/LetDevDev/unet

This topic contains links to all known Minitel protocols.

Core protocols (OSI layers 3 through 5):

• protocol-3.md - describes the format of Minitel packets and basic behavior of each node.
• protocol-4.md - describes how ordered sequences of packets should be handled.
• protocol-5.md - describes how Minitel's bidirectional streams work

Application layer protocols:

• FRequest - a simple file retrieval protocol
• MMail - simple mail delivery protocol

If you wish to have a protocol added to this list, post it here.

I may change the wording or fix bugs, but the actual specification will not / should not change.

1. Purpose

ON2 - OCNet L2 protocol - is a very simple standard is aimed at advanced network stacks. It provides protocol tagging and optional vlan separation for devices that support it.

 2. Protocol

ON2 utilizes the fact that OpenComputers network message can have multiple parameters. When a message(data frame) is sent using ON2 layer, the implementation sets first data parameter to protocol number and passes the data via second data argument. Port number(if supported by device) can be used as VLAN tag allowing network separation. Default port number(vlan) is 1

Here is an example modem call for sending data frame with protocol 0x46 (IP frame):

component.invoke(modem, "send", dest, vlan, 0x46, ip_data)

3. Implementation recommendations

Each network stack implementation should default to listening and sending on port 1. If packet of with unknown protocol number is received is should be silently dropped. Implementations should pass local/remote address and interface information(modem ID, vlan or interface object) to higher layers.

4. Protocol numbers:

Protocols are identified by numbers assigned in table below.

|-------------------------------------------------------------|
| PROTOCOL NUMBER | PROTOCOL NAME      | DEFINING STANDARD    |
|-----------------X--------------------X----------------------|
| 0x0C            | OC HOST DISCOVERY  | OETF #9              |
|-----------------X--------------------X----------------------|
| 0x46            | IPv4 or IPv6 Frame | RFC791 and RFC2460   |
|-----------------X--------------------X----------------------|
| 0xCF            | OHCP CONFIG PROTO  | OETF #10             |
|-----------------X--------------------X----------------------|
| 0x1000          | MultICE            | ???                  |
|-----------------X--------------------X----------------------|
| Above 0xFFFF    | User defined       | N/A                  |
|-------------------------------------------------------------|

You can apply for assigning protocol number by a post in this thread. 

1. Protocol

This is specification for a binary data frame(a Lua string), sent over OETF #7 protocol with protocol number 0xCF.

BYTE 0              | BYTE 1                   | DATA                       | BROADCAST
--------------------|--------------------------|----------------------------|-----------
0x00 - ADDR REQUEST | NOT SET                  | NOT SET                    | ALLOWED
0x01 - CLIENT RESP  | 0x00 IPv4 address assign | IPv4+subnet byte (4b + 1b) | NO
0x01 - CLIENT RESP  | 0x02 IPv4 gateway        | Gateway IPv4 (4b)          | NO

2. Protocol Flow

When a new client is starting up, it SHOULD broadcast `ADDR REQUEST` once. In case of no response it may retry after at least 15 seconds. If server has free addresses in address pools it should respond with `CLIENT RESP` packets, setting the client up.

OETF #9 - OC Host Discovery Protocol is a host discovery protocol for local OC networks specified by OETF #7.

2.Protocol

This is specification for a binary data frame(a Lua string), sent over OETF #7 protocol with protocol number 0x0C.

All frames are allowed to be sent as a broadcast or as a direct message.

Prefix byte | Action                              | Payload
------------|-------------------------------------|-----------
0x01        | Peer IPv4 Advertisement             | 4 byte IPv4
0x02        | IPv4 Discovery request              | Optional 4 byte IPv4
[reserved]  |                                     |
0x08        | Peer IPv6 Advertisement             | 16 byte IPv6
0x09        | IPv6 Discovery request              | Optional 16 byte IPv6

When a peer in a network receives a 'Discovery request', it MUST respond with a 'Peer Advertisement' massages with all addresses set on a given interface.

When a peer is assigned an address is SHOULD broadcast a 'Peer Advertisement' message. Some implementations may choose to broadcast those is some set periods, if they choose to do so, it's recommended to set the interval to at least 60 seconds to not stress the network.

Responses to 'Discovery request' frames SHOULD NOT be a broadcast messages.

When there is no payload in 'Discovery request' frame, all addresses set on a given interface should be advertised.

Implementations MAY choose to not implement filtering in 'Discovery request' frames, falling back to the no-payload behavior.

If you'd like to reserve an OETF document number, contact me, as I appear to have become the central authority for them. Here are the ways in which I can be contacted, in descending order of how quickly I will see your request:

• SolraBizna on IRC (irc.tejat.net, irc.esper.net)
• solra@bizna.name via email
• Private message on these forums

If you catch me at a good time, I'll have a few things to suggest on the topic of your document. If you catch me at a bad time, I'll probably just reserve you a number without question.

More information can be found at https://github.com/GlobalEmpire/GERT/wiki/GERTi

For more questions, please either post in this thread, or message MajGenRelativity#4971, or TYKUHN2#5283 on Discord. Messaging MajGenRelativity on irc.esper.net is also an option if you can't/won't use Discord.

This document provides a binary interchange format, intended primarily to support generic component IO.

Rationale

OpenComputers' component bus is designed for high-level languages. It sends and receives groups of dynamically typed values. It is intended to be user-friendly and self-discoverable, and it has largely achieved this goal. However, with low-level architectures, there is no obvious, straightforward way to represent these values. This document aims to provide a standard representation, freeing individual architects from having to devise their own representations, and minimizing unnecessary differences between architectures.

Every value that can be sent over an OpenComputers bus can be represented as described in this document, and (barring length restrictions) vice versa.

Conventions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

All signed integers are two's-complement.

Concepts

• Tag: Gives the type of a subsequent Value.
• Value: Data whose structure and meaning depend on its type.
• Tagged Value: A Tag, followed by a Value of the indicated type.
• Producer: A program or process that generates data in this format.
• Consumer: A program or process that consumes data in this format.
• Packed mode: A representation designed to occupy very little space.
• Unpacked mode: A representation designed to be easy to manipulate on 32-bit architectures.

Tags

A type tag denotes the type of a subsequent Value. In Packed mode, the tag is a 16-bit signed integer. In Unpacked mode, it is a 32-bit signed integer, aligned to a 4-byte boundary.

Values

String (UIFTAG_STRING = 0x0000–0x3FFF = 0–16383)

A UTF-8 code sequence. The tag provides the length, in bytes, of the sequence. Producers MUST NOT generate invalid code sequences, including "modified UTF-8" conventions such as non-zero NUL and UTF-8 encoded surrogate pairs. Producers MUST NOT arbitrarily prefix strings with a spurious U+FEFF BYTE ORDER MARK. Consumer handling of invalid code sequences is undefined.

If a Consumer encounters a String where a Byte Array is expected, the Consumer MAY incur a round-trip conversion to its native string type. This may mean that the bytes the Consumer actually sees differ from the original bytes where invalid code sequences occur.

Consumers MUST handle NUL bytes in a String in an appropriate manner. Consumers MUST not assume that Strings are NUL-terminated—they are not.

In Unpacked mode, additional zero bytes MUST be added to the end of the String, so that a subsequent Tag will be aligned to a 4-byte boundary.

Byte Array (UIFTAG_BYTE_ARRAY = 0x4000-0x7FFF = 16384-32767)

An arbitrary sequence of bytes. The tag, minus 16384, provides the length in bytes of the sequence.

If a Consumer encounters a Byte Array where a String is expected, the Consumer MUST interpret the Byte Array as if it were a String of the given length.

In Unpacked mode, additional zero bytes MUST be added to the end of the Byte Array, so that a subsequent Tag will be aligned to a 4-byte boundary.

End (UIFTAG_END = 0x...FFFF = -1)

A special tag signifying the end of an Array or Compound.

Null (UIFTAG_NULL = 0x...FFFE = -2)

Absence of a value. Equivalent to null and nil in various programming languages.

(Note: there is no tag -3.)

Double (UIFTAG_DOUBLE = 0x...FFFC = -4)

A 64-bit IEEE 754 floating point value. Consumers that encounter a Double where an Integer is expected MAY fail. Producers that are producing a Double which has an exact Integer representation SHOULD produce that Integer instead.

Integer (UIFTAG_INTEGER = 0x...FFFB = -5)

A 32-bit signed integer. Consumers that encounter an Integer where a Double is expected MUST convert the Integer to a Double.

Array (UIFTAG_ARRAY = 0x...FFFA = -6)

A series of Tagged Values, in a particular order, terminated by an End.

Compound (UIFTAG_COMPOUND = 0x...FFF9 = -7)

A series of pairs of Tagged Values. The order of the pairs is not significant. Each pair consists of a Key and a Value, in that order. A Key may be any type except a Byte Array, a Null, an Array, or a Compound. A Value may be any type. The list is terminated by an End. If a Consumer encounters an End as the second element of a pair, the result is undefined.

UUID (UIFTAG_UUID = 0x...FFF8 = -8)

A 128-bit RFC 4122 UUID. Regardless of endianness, the bytes are in display order. Consumers that encounter a UUID where a String is expected MUST convert the UUID to its canonical string representation, in lowercase. Producers and Consumers alike should take note that a random sequence of bytes is not necessarily a valid UUID.

True (UIFTAG_TRUE = 0x...FFF7 = -9)

A boolean true value.

False (UIFTAG_FALSE = 0x...FFF6 = -10)

A boolean false value.

TODO

This document is incomplete. Still to be written: recommendations on endianness and packing, useful common optimizations.

Rev. #2017010401 (0.1.1)

Abstract

This draft defines the NNR (Network to Network Routing) protocol as part of the OCranet family of protocols.  This document outlines how dynamic routing holds networks together in an NNR enabled OCranet network, how signalling is performed to control these networks, and the format of addressing used. This protocol is NOT the base protocol for OCranet networksand REQUIRES the support of specifications detailed in OETF #4; OCR (Ocranet Relay) protocol.

Rationale

The specifications detailed in OETF #4 only provide a means of cell forwarding for data networks and a robust topology. In the majority of cases it is not intended to be a stand-alone protocol. In a network of this type without any further support, only static routing can be achieved, requiring the manual configuration of connections along a path in a more permanent like manner.

OETF #4 specifically and generously reserves a feature allowing us to control such a network beyond the boundries of its specification and is described in detail under section "Switches", sub-section "Services" in OETF #4. The specifications detailed in this document are built upon this feature, allowing a network to dynamically configure itself given little configuration by introducing 3 concepts called SIGNALLING, DYNAMIC ROUTING, and NODE ADDRESSING.

Signalling

Signalling is the concept of communicating with a network independent of other connections to receive information from, provide information to, or configure networks, usually in a pasive manner. with NNR, signalling becomes the heart of how an OCranet operates. All signalling in an NNR enabled network occurs on VPI 0, VCI 4. All signals are represented using 8 bit identifiers in a 1 byte header immediately following the VPI and VCI fields in an OCR cell.

List of signal names and identifiers

• HELLO (0x01)
  Used by the active LOOP COORDINATOR to announce information about the network to newcomers. The information provided is as follows:
  • (8 bits) The revision of the NNR protocol supported by the network (Currently 0).
  • (128 bits) The Link Local address of the LOOP COORDINATOR (This provides the 16 bit Link Local prefix for local scope autoconfiguration)(48 bits) A 32 bit Network identifier and 16 bit Subnet identifier, if either is applicable (For global scope autoconfiguration)
• WHOIS (0x02)
  Sent by anyone to request if a particular address is in use. This can be an address of any type within the scope of the NNR addressing format [SEE BELOW]. If there is no response within 10 seconds of transmitting the WHOIS signal, then the address is considered up for grabs by the node which original sent the signal. The WHOIS signal is comprised as follows:
  • (128 bits) The address being queried
• DEIFY (0x03) (Implementation currently optional)
  When no HELLO message has been received by the LOOP COORDINATOR for a given time, It is possible that the LOOP COORDINATOR has failed or disconnected for some reason. In this event, each switch with the same Link Local prefix count the entries in their routing tables for each virtual path, including their temporary cached entries and broadcasts a DEFIY signal on the loop. Though important for reliability, due to the complexities of this operation its implementation is currently optional. The DEIFY signal is comprised as follows:
  • (128 bits) The link local address of the node they wish to become the new LOOP COORDINATOR
• INUSE (0x04)
  Broadcast message sent by a node witnessing a WHOIS broadcast with their Link Local address to inform the sender that the address is currently in use and active. The INUSE signal is comprised as follows:
  • (128 bits) The address being queried in the WHOIS signal
• DIALOUT (0x05)(Forward recursive) unicast message sent to the LOOP COORDINATOR to request to build a circuit to the specified global scope address. The DIALOUT signal is comprised as follows
  • (128 bits) The global scope address of the end node to reach
  • (16 bits) A tag number passed back the LINE signal upon successful circuit construction informing the previous-hop / initiating node which connection is ready. This     number can be anything, but should be unique within a short time frame at minimum. This number should be cached and changed at each hop as a security measure to prevent switches from becoming confused during circuit construction.
  • (30 bytes) A host-dependent (Not covered in this document) data payload containing any information the end host needs to set up a connection.
• ACK (0x06)
  Unicast sent by any node acknowledging a signal from another node. This should be sent after any unicast message to prevent nodes from repeating themselves when signal reliability is necessary. The ACK signal is comprised as follows:
  • (8 bits) The signal identifier being acknowledged.
• HUP (0x07)
  (Unidirectional/Multidirectional recursive) unicast message sent to the next-hop and/or previous-hop switch indicating to tear down / hang up the active circuit. This causes a recursive ripple effect that automatically tears down a connection. Any nodes in between that have not received the message should eventually time out and initiate an HUP signal themselves if applicable to fully dissassemble the circuit and prevent zombified links. If an HUP signal is not initiated at an endpoint node, then two HUP signals should be produced if applicable; One for the next-hop, and one for the previous hop. For security purposes, an HUP signal MUST ONLY be listened to if it arrives via the VPI & VCI pair it is associated with / forwards to. An HUP signal is comprised as follows:
  • (8 bits) The Virtual Path Identifier in the direction of the teardown
  • (16 bits) The Virtual Channel Identifier in the direction of the teardown
• LINE (0x08)
  (Reverse recursive) unicast message initiated from the end node providing all information the previous-hop needs to forward messages in a circuit to the next-hop toward the end node. The LINE signal is comprised as follows:
  • (16 bits) The tag specified during the DIALOUT signal
  • (16 bits) The VCI of the next-hop
  • (8 bits) The VPI of the next-hop

Addressing
NNR uses a 128 bit address format that contains the following information:

• (16 bits) Address type
• (16 bits) Address parameters
• (32 bits) Network identifier
• (16 bits) Subnet identifier
• (48 bits) Host identifier

A complete NNR address is represented in the following way:

TTTT:PPPP:NNNNNNNN:SSSS:HHHHHHHHHHHH

Where TTTT is the address type, PPPP is the address parameters, NNNNNNNN is the network identifier, SSSS is the subnet identifier, and HHHHHHHHHHHH is the host identifier.

Address types
Current, only two address types are supported. These are Link Local (0x0000), and Link Global (Address type 0x9001).

• Link Local addresses are unique within a loop and its next-door neighboring networks. The 16 bit parameters field in this case is called the "Local scope prefix" and provides a virtual barrier between networks on the same or adjacent loops. All nodes in the same local network share the same local scope prefix, but a switch bordering multiple networks may have more than one local scope prefix, and thus more than one link local address; One for local network group. Link local addresses are used solely for communicating within a local network group and can not be routed accross networks. A node with only a link local address has the added security of being globally invisible and unreachable without more complex routing infrastructure which is outside the scope of this document.
• Link Global addresses are unique for each network cluster (And MUST be unique within the entire OCranet scope). These addresses are used for communicating to nodes in neighboring or distant network loops.

Shorthand formatting of addresses

The subnet portion of global scope addresses is required to be present but may optionally used for routing [SEE SUBNETTING]. Likewise, the network AND subnet portions of a link local address are required but never used. For this reason, it is acceptable and RECOMMENDED to use the shorthand (double colon) operator when representing addresses for human readability. The double colon MUST only be used once in an address, and its position is dependent on the address type. A couple examples are provided below:

The Link Local address 9001:0000:37f:0000:a8f35779fe4b     can be shortened to this: 9001:0000:37f::a8f35779fe4b
The Link Global address 0000:5f:00000000:0000:a8f35779fe4b can be shortened to this: 0000:5f::a8f35779fe4b

Dynamic Routing

In order for signals to operate, processes that fit in the category of dyanmic routing are necessary. These processes are as follows:

New node joins a network loop

Host NEWHOST is attached to a network loop. The procedure is as follows:

1. NEWHOST waits for 20 seconds and listens for a HELLO signal or DEIFY signal.
       - If no signal is received, NEWHOST assigns itself a Local Scope prefix. This may be preconfigured or generated randomly. Use precaution when generating randomly to ensure           that neighboring networks do not share the same prefix. It is much easier to just assign it manually when creating a new network loop for the first time.
       - If a HELLO signal is received, NEWHOST assigns itself the Local Scope Prefix of the LOOP COORDINATOR.
       - If a DEIFY signal is recieved, NEWHOST waits another 20 seconds and resumes from step 1.
2. NEWHOST uses a UUID (It is RECOMMENDED to use the UUID of the OC network card in use if applicable) to generate a Link Local address using its Local Scope Prefix.
3. NEWHOST sends a WHOIS containing the Link Local address it generated and awaits for an INUSE signal for 10 seconds.
       - If NEWHOST receives an INUSE signal with the Link Local address it generated, a new 48 bit host identifier must be generated. After this is done, resume to step 2.
       - If NEWHOST does not receive an INUSE signal with the Link Local address it generated within 10 seconds, it assigns itself that Link Local address.
4. NEWHOST may optionally configure itself a Link Global address following steps 2 through 3 but instead using Link Global addressing.
5. In the event that NEWHOST is a switch bordering another network and an existing LOOP COORDINATOR is available, NEWHOST sends a NEIGH signal containing a route to the network it binds to the LOOP COORDINATOR.

This document may be a stub; It provides the majority of specifications for the NNR protocol but may be missing some specific features and will be updated within the near future.

Rev. #2017032401 (0.2.0)

Abstract

This draft defines the OCR (OCranet Relay) protocol as part of the OCranet family of protocols. This document outlines how OCR forms connections, how data is fowarded, and the structure of a data segment used in an OCR network.

Rationale

An agreement between computers in terms of how to communicate is important in any data network. In the past, there have been several attempts to build networks to transfer data bewteen machines, but it has been hard to come by a family of protocols that are unobtrusive, portable, passive, reliable, fast, versatile, easy to implement, and easy to maintain all at the same time.

Meanwhile, in time, curiosity has brought up questions of inter-server communication; Data transfer between OpenComputers machines accross multiple Minecraft servers. It is the research on this subject that has brought attention to a new protocol that is capable of fulfilling all of these requirements. Thus, as a pun on ARPANet, the OCranet is born. OCranet Relay protocol (OCR for short) is responsible for gluing the OCranet together in hopes of forming a large scale network for all of us.

Conventions

This document follows all guidelines proposed in RFC 2119.

Circuit switching, paths, and channels

OCR networks form a circuit switched topology. This means that routing is predetermined for any given connection; Instead of determining the path of data for every segment, the path is determined when the connection is first initiated and BEFORE communication with the destination host begins. Unless a network failure occurs, any data part of a connection should always follow the same path to get to its destination until the connection is torn down.

Virtual Paths
OCR makes use of virtual paths which represent the physical or virtual next-hop destinations a host may reach. The wire connecting each computer face to face in a network is called a LOOP. There are two standard models for constructing an OCR network: internal routing, or network interconnection only. Using an internal routing model, OCR is used to route between each host on a network, while on the other hand a network interconnection only model uses OCR only for interconnecting each network, but not the hosts inside of them. A complex network can be built by arranging multiple loops and connecting them together by using one or more OCR switch(es) [DOCUMENTED BELOW]. Using an internal routing model, each host on a loop is identified by a Virtual Path Identifier (VPI). However, using a network interconnection only model, only each network has a particular VPI. This number is 8 bits, therefore there may be a total of 256 hosts on the same loop. When using OCR with an internal routing model, VPI 0 is RESERVED for speaking directly with the host. When using OCR with a network interconnection only model, VPI 0 should be reserved for speaking to that network (ex. a switch that governs that network).

Virtual Channels
Each connection from a host is represented by a single Virtual Channel Identifier (VCI). a Virtual Channel represents a connection within a Virtual Path. The Virtual Channel Identifier is 16 bit, therefore every host can simultaneously handle a maximum of (256 * 65536) = 16,777,216 connections at any time. Bear in mind that VPI 0 is reserved, so more likely your maximum sustainable connection limit will realistically be (255 * 65536) = 16,711,680 simultaneous connections.

Switches
To separate multiple loops, a switch is deployed and is responsible for forwarding in that direction. switches are REQUIRED at minimum to be responsible for keeping track of VPI and VCI pairs as well as forwarding data along these Virtual Paths via their Virtual Channels.

In order for a switch to perform its job, additional features are RECOMMENDED. These features should control the use of Virtual Path and Virtual Channel propagation. One such example of a feature is a dynamic ROUTING PROTOCOL which might be responsible for performing an address or telephone number lookup and building a chain of Virtual Path and Virtual Channel pairs across an OCranet network. The complete concept of OCR routing protocols intended to be extensible and separate from OCR, and therefore is outside the scope of this document.

Forwarding
The process of re-transmitting a data segment to the next-hop switch or computer is called forwarding. The Virtual Path Identifier and Virtual Channel Identifier always indicates the VPI and VCI pair that matches on the next-hop switch. This means that if the computer you are speaking to is on VPI 3, VCI 7 of the next-hop switch, then your data segment MUST have a header with a Virtual Path Identifier of 3 and Virtual Channel Identifier of 7 when it is read by that switch. It is normal for the VPI and VCI fields in each data segment to be replaced at every hop along the way while propagating through a series of loops in a network.

Services
To promote extensibility, versatility, and hopefully the reliability of OCR networks VPI 0 is reserved for speaking directly to the switch. It is RECOMMENDED that each service has its own Virtual Channel Identifier at VPI 0. Furthermore, VCI 0 through VCI 4 via VPI 0 are RESERVED for future use by OCR and MUST NOT be used until an applicable standard suitably fits. It is RECCOMMENDED that features such as ROUTING PROTOCOL are provided on VPI 0, given a particular (and hopefully unchanging) VCI.

Cell format
Every segment of data transferred in an OCR network is called a 'cell'. Each cell is exactly the same size to reduce jitter effects, normalizing propagation latency. A cell is defined as a 51 octet data structure that contains a 3 byte header, providing 48 bytes of data transfer per cell. The format of the header is as follows:
 

 0              15       23 <-- Bit #
*----------------*--------*
|      VCI       |  VPI   |
*----------------*--------*

Since revision 0.1.x, the order of this structure has been reversed to increase performance of the protocol by providing proper word alignment. This makes cut through switching a little more latent, but ensures that the datatypes align properly.

Having a think about it, it would make sense if the bus ran at 1MHz. The Z80 currently runs at 2MHz, and all memory accesses on the Z80 tend to take a minimum of 3 cycles (in reality it's more like 2 cycles followed by a DRAM refresh but we'll just skip on that and assume the whole window is available for it). The MIPS3 core I'm working on also runs at 2MHz. I would like to be able to clock that at a higher speed than the main bus, but in order for that to work, the bus needs an API for it.

Easy case: Only one CPU on the bus

public int getTime(int cpuid);
public int ensureTime(int cpuid, int currentTick);
public int eatCycles(int cpuid, int startTick, int lengthInTicks);

Reason for returning an int in ensureTime is to get the correct point in time.

How to access the bus (a proposal):

int targetBusTime = this.currentTimeSliceBusTicks - (this.cycleBudget/this.clockMultiplier);
int actualBusTime = memory.ensureTime(this.cpuid, targetBusTime);

// do reads and writes here - eatCycles is called by the bus components

int endBusTime = memory.getTime(this.cpuid);
this.cycleBudget = this.currentTimeSliceCpuTicks - endBusTime;

In a system which ignores the cpuid, these can be implemented like so:

private int busTime = 0;

private int onTickStart_ResetBusTime() {
    this.busTime = 0;
}

public int getTime(int cpuid) {
    return this.busTime;
}

public int ensureTime(int cpuid, int currentTick) {
    if(this.busTime < currentTick) {
        this.busTime = currentTick;
    }
    return this.busTime;
}

public int eatCycles(int cpuid, int startTick, int lengthInTicks) {
    ensureTime(startTick);
    this.busTime += lengthInTicks;
    return this.busTime;
}

Oh yeah, the thing that makes this more fun? By "CPU", anything which does DMA could be classified as a "CPU".

If all CPUs execute in parallel, then the one-CPU approach can work just fine. Otherwise, that bus is going to suffer horribly.

Ultimately you need to track each CPU's busTime in an array (i.e. private int[] busArray;), and implement what is basically a memory allocator minus the memory.

You'll basically need to track various windows into the bus in a sorted manner. I'd say a bitfield implemented in terms of ints would be the most suitable approach.

As for ensuring that no one CPU can hog the bus the whole time, well, you could possibly inject a few windows into the bus on the bus controller. Or you could just punch anyone who decides to use a 6502 on a multiprocessor system.

One more thing, I've just noticed an issue with this "time travel" approach: There is no guarantee that I/O will be executed in the correct order or RAM updated in the correct order between CPUs.

Easy solution for hard case: Timeslicing

Each CPU gets given a reasonably short timeslice of the bus. Possibly two timeslices per tick.

Really easy solution for hard case: Don't even bother with accuracy

Just run each CPU in its own thread and completely ignore this proposal. Simple.

• Compatible with OpenOS (which would try, and fail, to run "/boot/arm.elf" as Lua code on startup)
• No conflict with OpenArms (or other ARM architectures)
• Similar to standard practice for OS kernels (/mach_kernel, /vmlinuz, /kernel...)

That's all well and good for filesystem-based booting, but what about drive-based booting? For that, I propose that the first sector of a bootable drive begin with a block in a format like the following:

BOOT:Lua 5.2,Lua 5.3:s1+1199;OC-ARM:s6+2472;MikeKaraoke:b64+102.

This idea is compatible with any partition table scheme of the "data at end of sector" type, like the MBR scheme.

(Here I have assumed that the first sector is numbered 0. If the sectors are 1-based, the spec will have to be changed to reflect that.)

I'd be happy to write all this up as a (pair of?) pseudo-RFC but I'm open to feedback first. I've heard that this topic has been discussed a lot on IRC.

D1:   2016-08-05T02:45:39+00:00

$ date -u -Iseconds

There are two address spaces for all components: the config space, and the main I/O space.

Every component has a VID (virtual ID) and that is how components are addressed. Every VID should be physically settable to any 8-bit value on each component. Any component with a VID set to 0xFF is rendered inaccessible by any methods that address by VID.

Every component also has a PID (physical ID) which indicates the physical location of this component on a given bus master.

The config space is read-only. Everything in the config space is readable by any bit width.

CONFIG SPACE

This bus is little-endian.

These fields are compulsory, and every component needs to read at least the bottom 3 address bits:

• 0x00[4]: 4-byte type ID. Forms a 4-char ASCII string.
• 0x04[1]: Bus PID.
• 0x05[1]: Bus master VID. 0xFF if using the root bus master.
• 0x06[1]: Bus PID with all bits inverted.
• 0x07[1]: Bus master VID with all bits inverted.

• 0x08[4]: 4-byte extension ID #1.
• 0x0C[4]: 4-byte extension ID #2.

Open bus is 0xFFFFFFFF, because that's what us low-level weirdos tend to expect. This applies to all busses. Because of this, if you have a type ID of 0xFFFFFFFF, that VID has no component using it.

The root bus master (it'll be on your motherboard) is of course an exception to a lot of these rules:

• VID is always 0xFF and it is completely unconfigurable.
• PID is 0xFF too. So is the bus master VID.
• If the ID is 0xFFFFFFFF, then this bus master cannot be reconfigured, but there's DEFINITELY a bus master here.
• Basically, if you have a "dumb" bus master, the config space will be open bus, which also means that a bus conflict check will fail.

Some components will have standard VID slots. Conflicts should be avoided where possible. Proposed standard slots:

• 0x00: Various BIOS EEPROMs would probably end up using this.
• 0xC0: MIPS BIOS EEPROM.
• 0xDB: Serial debug interface. ID is "Dbg0". Write to 0x00 to output byte, read from 0x00 to input byte, -1 will be returned if no byte is available, bus is actually 32 bits wide, but can be accessed by byte, reads from 0x01-0x03 are buffered.

• 0x0+++++++: User virtual memory space
• 0x800xxxxx: Cached RAM (bit mask is for 1MB)
• 0xA00xxxxx: Uncached RAM (bit mask is for 1MB)
• 0xBEvv00rr: Uncached Config space for VID=v, address=r
• 0xBFvvrrrr: Uncached Main I/O space for VID=v, address=r
• 0xC+++++++: Kernel virtual memory space

These are in MIPS assembly. There is no op reordering, but I have decided to insert NOPs in delay slots just for the sake of readability. I have also decided to use the LA ("Load Address") pseudo-op as a load-immediate so things are a bit more obvious.

To clarify: LA loads the 32-bit value you see there DIRECTLY - it does not do a memory load. (It's also reasonably likely to use 2 real ops, although I think there's really only one place here where it actually does.)

I have tried to avoid MIPSisms here.

Component presence detection: (ID in $a0, $v0 returns nonzero if there is a component)

LA      $t0,      0xBE000000
ANDI    $t1, $a0, 0x00FF
SLL     $t1, $t1, 16
ORI     $t0, $t0, $t1
LW      $t1,      0x00($t0)
NOP
ADDIU   $v0, $v0, 1 // convert 0xFFFFFFFF -> 0
JR      $ra
NOP

LA      $t0,      0xBE000000
ANDI    $t1, $a0, 0x00FF
SLL     $t1, $t1, 16
ORI     $t0, $t0, $t1
LH      $t1,      0x04($t0) // LH is a signed load, thus you get a sign extension
LH      $t2,      0x06($t0)
NOP
XOR     $v0, $t1, $t2
ADDIU   $v0, $v0, 1 // convert 0xFFFFFFFF -> 0, or 0 -> 1
JR      $ra
NOP

MOVE    $a0,      $s0
J       detect_bus_conflict
NOP
LA      $t0,      1
BEQ     $v0, $t0, _vid_does_not_exist
NOP
BNEZ    $v0,      _vid_has_bus_conflict
NOP

LA      $t0,      0xBE000000
LA      $t1,      0x00010000
LA      $t2,      255
LA      $t3,      'Dbg0'

_loop_find:

// Detect Dbg0 component
LW      $t4,      0x00($t0)
NOP
BNE     $t4, $t3, _loop_find_continue
NOP

//
// Print hello message
//
LA      $t6,      str_hello
LA      $t5,      0x01000000
LB      $t7,      0($t6) // Preload first byte
ADDU    $t5, $t5, $t0 // 0xBFxxxxxx address

// Actual print loop
_loop_hello:
SB      $t7,      0($t5)
ADDIU   $t6, $t6, 1
LB      $t7,      0($t6)
NOP
BNEZ    $t7,      _loop_hello
NOP

// Advance loop
_loop_find_continue:
ADDIU   $t2, $t2, -1
ADDU    $t0, $t0, $t1
BNEZ    $t2,      _loop_find
NOP

// Return
JR $ra
NOP

// Read-only data
str_hello: .asciiz "Hello World!\n"

I don't really have any authority on this, but thought it would make sense if everyone used the same ports for this sort of functionality.

I would suggest using, say, port 10 (doesn't really matter) for all messages containing lua to execute, and putting said lua in the last part of the packet: (modem.broadcast(10, "username", "password", "gpu.set(0, 0, \"Hello World\")"))

Why is this useful? I can now have a single drone that works with many different programs with no real configuration. I can download programs via oppm, or paste them in from the internet, and they will always use the same port to access said functionality.

• Trivially maps to every object type OpenComputers itself allows
• (implied by the above) Trivially maps to the Lua representation
• Feasible to implement in hardware
• Simple to implement in software
• Independent of architectural details
• Compact

There are two versions of this format: packed and unpacked. Packed is more compact, and is suitable for wire transmission or 8-/16-bit architectures. Unpacked is simpler to deal with on 32-bit architectures. (The packed representation, not coincidentally, takes up the same amount of space as is calculated for network packet size limiting purposes.)

Applications communicating over the network:

• SHOULD use the "pickled" representation from the Lua `serialization` API, when possible (for compatibility with Lua)
• SHOULD use the packed representation with network byte order, when the above is not possible

• MUST support the use of the "packed" representation
• MUST support native byte order
• SHOULD have optional support for network byte order, if that is different from native
• MAY have optional support for "unpacked" IO, if this makes sense on the platform

An Interchange Value is a type tag followed directly by the data. In the packed representation, tags are 16-bit. In the unpacked representation, they are 32-bit and sign-extended (such that 0xFFFE becomes 0xFFFFFFFE), but still limited to the same range).

• Tag 0x0000 - 0x3FFF: ICTAG_STRING

  A UTF-8 string, whose byte length is the low 14 bits of the tag. In the unpacked representation, the string is padded to a multiple of 4 bytes; padding is not reflected in the length. No NUL terminator is required, and NUL may freely exist within the string, but producers SHOULD avoid producing strings containing embedded NULs. Consumers MAY (but SHOULD NOT) process embedded NUL as a NUL terminator, and discard the portion of the string after it. If there are invalid UTF-8 sequences in the string, they MAY be lost during subsequent processing. Consumers that convert to another representation (such as UTF-16 for Java storage, or Unicode code sequences for display) SHOULD discard invalid UTF-8 sequences. Producers about to output an ICTAG_STRING that contain an exact, valid UUID value with lowercase digits SHOULD instead produce an ICTAG_UUID.

• Tag 0x4000 - 0x7FFF: ICTAG_BYTE_ARRAY

  An array of byte values with no particular semantics, whose length is the low 14 bits of the tag. In the unpacked representation, the string is padded to a multiple of 4 bytes; padding is not reflected in the length. Consumers that expect a string MAY treat an ICTAG_BYTE_ARRAY as an ICTAG_STRING. Consumers that expect an ICTAG_BYTE_ARRAY (e.g. disk IO) may only treat an ICTAG_STRING as an ICTAG_BYTE_ARRAY if it is still available in its original serialized form, not having made a round-trip conversion.

• Tag 0xFFF8 / -8: ICTAG_UUID

  A 128-bit UUID. Consumers that expect a string MUST convert an ICTAG_UUID into its canonical ICTAG_STRING equivalent, using lowercase digits. Note: Consumers that expect a UUID are NOT required to accept a well-formed ICTAG_STRING in its place.

• Tag 0xFFF9 / -7: ICTAG_COMPOUND

  A list of key-value pairs stored key first, value second, terminated by ICTAG_END. Any type but ICTAG_BYTE_ARRAY, ICTAG_COMPOUND, ICTAG_ARRAY, and ICTAG_NULL may appear as a key. Any type may appear as a value. If a consumer encounters an ICTAG_END where a value should go, the entire Interchange Buffer MUST be discarded as invalid.

• Tag 0xFFFA / -6: ICTAG_ARRAY

  A list of Interchange Values, terminated by ICTAG_END. Any type may appear as an element of an array.

• Tag 0xFFFB / -5: ICTAG_INT

  A signed, two's-complement 32-bit integer.

• Tag 0xFFFC / -4: ICTAG_DOUBLE

  An IEEE 754 64-bit double. Producers SHOULD convert doubles with exact signed two's-complement 32-bit integer values to ICTAG_INT. Consumers that expect an ICTAG_DOUBLE MUST be able to process an ICTAG_INT in its place, and MUST NOT treat it differently from an ICTAG_DOUBLE with the same numerical value.

• Tag 0xFFFD / -3: ICTAG_BOOLEAN

  Either true or false. In the packed representation, it is a single byte. In the unpacked representation, it is 32 bits. Zero is false, any non-zero value is true. Producers SHOULD produce all-bits-set for true. Consumers MUST treat any non-zero value as equivalent to all-bits-set.

• Tag 0xFFFE / -2: ICTAG_NULL

  A strongly-typed NULL, equivalent to Java's null and Lua's nil.

• Tag 0xFFFF / -1: ICTAG_END

  Signifies the end of an Interchange Buffer, array, or compound.

An Interchange Buffer simply consists of zero or more Interchange Values, terminated by ICTAG_END.

Issues

Should there be more integer/float types? e.g. ICTAG_SHORT, ICTAG_BYTE, ICTAG_LONG, ICTAG_FLOAT...

No. This would make writing simple processing code difficult. ICTAG_DOUBLE fully covers Lua's number range, and covers a significant portion of Java's number range as well. ICTAG_INT covers a sufficiently wide range to remove the need for floating-point math in the vast majority of IO operations.

8-/16-bit architectures cannot process 32-bit numbers easily. However, consumers on those architectures can, situationally, consume only the precision they expect, and gracefully fail when given too-large integers. (The situation would be far worse if they were expected to process an ICTAG_DOUBLE.)

Should converting integer-valued ICTAG_DOUBLE to ICTAG_INT be mandatory?

No. In situations where only integers are expected, but the original representation supports only "reals", transparent conversion is valuable. However, in situations where non-integer values are common, the conversion does not add value. It should be up to the generator to determine whether it makes sense to do the conversion.

Should converting from UUID-valued ICTAG_STRING to ICTAG_UUID be mandatory?

No. If, as is the usual case, the generator knows that subsequent use of the string as a UUID is unlikely, it can avoid the overhead.

Should accepting ICTAG_UUID in place of ICTAG_STRING be mandatory? and Should accepting ICTAG_INT in place of ICTAG_DOUBLE be mandatory?

Yes. In situations where the semantics of the values are not known, it is desirable to have "automatic simplification". "Automatic simplification" can only work if it can be expected to be reversed just as automatically when the conversion was not helpful.

Should there be ICTAG_TRUE and ICTAG_FALSE instead of a single ICTAG_BOOLEAN?

Unresolved.

How is an ICTAG_UUID structured? This is important with byte orders other than network byte order.

Unresolved. Possibilities:

• A sequence of bytes, in display order. No swapping is performed or necessary.
• (as in Java) A pair of 64-bit integers.
• (as in MFC) A 32-bit integer followed by three 16-bit integers followed by 6 loose bytes.

Java Strings are encoded as UTF-16. Interconversion between UTF-8 and UTF-16 is pretty simple, but still is overhead that isn't necessary for tasks like disk IO. In addition, there is no obvious way to preserve invalid UTF-8 sequences such that code that wishes to deal with byte arrays (like disk IO) can get the original byte sequence after a round-trip conversion trip through UTF-16. Creating such a system as part of the specification may be desired, but would then allow "invisible overhead" to creep in.

Common Optimizations

(This information is only useful to architecture implementors.)

OC-ARM uses this format for component IO. It provides two facilities that usefully reduce the overhead of this system.

• Byte array IO return: When a program expects an IO operation to return only a single byte array, it can determine the size of this byte array by reading the size needed to store the Interchange Buffer and subtracting overhead. It can then use a special method to (attempt to) read the bytes directly into memory, rather than into an Interchange Buffer.
• Truncation: In situations (such as certain types of signal processing) where only a few values are important, the program can specify that it is interested only in the first N values before storing the Interchange Buffer. It can then proceed normally; extra values are simply discarded. This is useful in extremely memory-restricted situations.