Replacing Shell automatically from lua

[Gopher 2]

In my fiddling, I found myself wanting to replace the shell with my own. This is straight-forward enough to do; you can change the environment variable "SHELL" from "/bin/sh" to point to your own program by typing "set SHELL=<pathtoyourshell>" and then "exit" at the standard OpenOS SH prompt.

I wanted to do this automatically, though, in an autorun script, and that proved a bit trickier. You can change the environment variable by calling os.setenv("SHELL","pathtoyourshell"), but even an autorun script has a shell already running behind it which will continue when the script ends.

I thought of several things that seemed like they should work, such as using computer.pushSignal to send a faked ctrl+c event before exiting back to the shell, but nothing seemed to work for one reason or another. After poking the os with a stick for a while, I eventually came up with this approach that works nicely:

local os=require("os")

if os.getenv("SHELL")=="/bin/sh" then
  print("Memory initial:",computer.freeMemory())
  local term=require("term")
  os.setenv("SHELL",require("process").running())
  local oldread=term.read
  term.read=function(...) term.read=oldread return "exit" end
  return
end

This snippet checks if the shell variable is pointing to the default shell; if it is, it replaces that shell with the path to itself. Then, it backs up and overrides term.read function and exits the program. This will return control to the shell, which will immediately call your new read function. This function immediately restores the original read function, so it will only be called once, but it returns "exit" which will cause the parent shell to exit immediately. The OS will respond to this by launching a new shell, and thanks to the new SHELL env value, this will run your program again, this time without sh running behind it, holding on to memory and resources.

I put this together to use on a custom program for robots, to let my programs access more of their limited memory, but it could also be used to good effect for any shell replacement, or contexts where you don't want users to exit your program and return to the shell.

Note, if you do this in an autorun file, your program /will/ run whenever the disk is inserted. If you wind up with a defective autorun setup, or otherwise just need access through sh to the disk's contents, you'll have to boot the computer with the disk/hd removed and disable autorun before reinserting it. You can disable autorun by running the lua program and entering this:

lua> filesystem.setAutorunEnabled(false)

then you can insert your disk and it will appear in /mnt/ without running it's autorun script.

[Molinko 43]

Nice way to drop / lock the shell, then boot your own. nice and pretty simple in the end