coderboy14

So, I wanted to develop a more unix/linux like operating system for OpenComputers, but of course Linux is a kernel, not an operating system - so I made a kernel, which I can then use for my operating system!

Of course, being designed with a unix mindset, I am trying to follow the rule that everything can be represented as a file. Currently, your devices are all represented as folders, in the /dev folder. You can actually interact with the components by opening their files. Each component has a virtual folder with the address its assigned (in a linux format, like "sda") based on the driver assigned to it. In this virtual folder, are "files" that represent every method that the driver has informed the kernel of. By opening it, and writing to the handle (always use write for this, at least with how it's currently setup), it invokes the component's methods!

Creating virtual devices, virtual files, etc. is also equally as easy. The file system controller doesn't actually care what you tell it to mount, so long as you pass it a controller that implements the right methods. Just write a driver for your virtual tree object, mount it to the file system controller, and wallah, it now may as well be a real thing. Adding a virtual component is just a tiny bit more tricky, since the '/dev' folder is a special virtual disk using custom instructions to manage the layout.

While I think I've made pretty ok progress with the project, it still has a long ways to go. Not only do I want to add some more libraries to the kernel build, but I also need to test the ones I already have, since I'm almost certain I forgot about a few bugs, or just forgot to go back and implement placeholder code. 

I am planning very soon on writing some actual documentation for everything, but right now, you're mostly on your own for figuring out how things work. The kernel has an "init.lua" script to make it bootable and run some tests, but you'd replace that with the startup code for your operating system, just making sure to invoke the kernel first.

I'll drop the GitHub link to my project. Everything related to this project is penguin themed, of course. I have only been using OCEmu so far to test my code, so I haven't the foggiest how it works on different device configurations in-game.

Here's the link to my GitHub page

TL;DR - Add configuration open to make the networks behave more realistically, allowing for network traffic to be sniffed.

There's not much more to be said here. Add the ability, in configuration at least, to make all packets sniffable. So they behave more like real packets, instead of magically reaching only their intended target and nobody else. In your code listening for network events, you'd just need to check if the target address matched your modem's address. Or maybe add this feature by default to the network library, that you can enable or disable. Preferably, this would include all ports, or at least an open to sniff all ports.

It would add some additional challenge for servers, and give the data card more of a use (encryption and decryption). It would also allow for routers to easily sniff traffic, to route it, without it necessarily needing to be the target.

So, I have been trying to find somebody who's implemented a filesystem for the unmanaged drives. I know what you must be thinking, just use the managed drive then, I'm legit doing what a managed drive does. However, for a few projects, I've wanted or needed an unmanaged filesystem. One example, is an expanded raid with minimal overhead. The simplest sort of raid, just glue all the drives together in one long linear addressable space. That parts easy, just check what section the pointer is on, and forward the operations to the proper disk. The issue is, they require that they be unmanaged, otherwise the "glue" doesn't work. 

Has anybody seen, or maybe is working on, an implementation of a filesystem for unmanaged drives. I searched here and elsewhere, but couldn't find anything. I'm also terrible at understanding filesystems myself, otherwise I would try and implement this.

Thanks for any information you can provide me.

So, for a few days now (apparently about five days according to GitHub - it seems a lot longer), I've been trying to work on a pretty simple package container solution. The idea being just like a tar container, it is just a group of files and folders, stored in a single file. Currently, I am not working on implementing any sorts of compression, that's much too difficult and I wanna make sure the core system works first. I designed a primitive specification for the package, and created a method in a library to create the package. That I was okay with. Extracting all the information from this package is where my head is starting to spin, haha. Frankly, if there's already an existing solution, I'd really prefer to use that. Programming something like this means fixing countless bugs and writing entire libraries before the product is even stable enough to be used by people in the first place! If something does exist, please do tell me.

If nothing like this exists, I would appreciate some assistance at very minimum brining version one up to at least a functional level. By help, I mean that while developing the unpacking code right now, I'm pretty much running around with my eyes closed. Anything that makes this process less horrific, is really really appreciated. This project was started, because I wanted to start a little package manager project too. I know OOPM exists, but I wanna make one, simply because I can. That's besides the point. While the package format is designed for this use, it should be usable on its own based on my format.

I know the format is ... meh ... at best. I already have a new structural system in mind for version two. However, before I go off redoing this project, I want to at least get version one working! I have pretty much very social media outlet in the App Store, even some I can't fully understand (such as QQ since I'm [procrastinating] learning Mandarin)! Discord, Skype, and all those common ones of course, so I should be easy to contact if need be. I'd really appreciate any leads for existing software you find, or any help you can provide at least getting V1 working.

-- UPDATE --

Wow. I really forgot to add the link to the GitHub repository for this project. The repository is here [https://github.com/OcilDiamond/opkgf].

Tangent about V2

While I'm talking about it, I'll mention how I plan on redoing the internal structure of the packages. Right now, to read anything you need to scan it from the start and search for flags that say that an attribute has ended, that a section has begun, etc. The main change in my new theoretical design, is that the package descriptor and metadata sections will be partly fixed-length. Meaning, in the beginning of the package will be a fixed-length table. This table will contain the pointers and sizes of various values. Meaning, regardless of the size of the title for the package, the descriptor table will always be exactly the same length. Importantly, it will contain a pointer to the body structure, where the new design is really different.

In the V2 system, there will be an actual hierarchy stored. At the lowest level, the first object a package's tree will have is (a) trunk(s). This can either be a relative trunk (meaning it allows for the package to extract to whatever directory you're in), or an absolute trunk (so it can install library files into /lib). Everything else in the package will be a child of a trunk. Along with that, every object must have a unique ID (unique, in the sense that it's not reused within the same package). To store and recall the data, instead of having every file and folder prefix with a special identifier, attributes terminating in nulls, the content, and more nulls (as V1 does) it will use a table at the beginning of the body section to find data. This table will be variable length, since it changes based on the contents, but it would be a simple linked list. The first portion of each row would be fixed-length, containing things like the file's ID, it's parent ID, it's type (file, folder, trunk), and a pointer to the next object in the list. After that comes the name of the object, but that's last since it's variable length. Meaning, if you load only the file directory table into memory, you can pick and choose which files to extract, analyse the structure, etc. Since the IDs and pointers for the files themselves are stored separately from the data, it's nice and easy to load into memory, it shouldn't be extremely large, and should be easy to work with. One change that could be made, is have two variable length attributes in each file directory row. Now each row also contains the pointers of the children of that object. That would make viewing the hierarchy itself very easy and quick to do, since jumping between nodes would be simple and easy.

I figured I would give a description of this format just in case. I may start working on this before V1 even get's finished. The reason is, this will likely be its own GitHub repository, since this data structure could be used for other things potentially too, like databases.

Again, thanks for any help you can provide, or information you can give. Sorry this message is so long, I'm not the best at writing short concise messages - as you can see, I'm more of a long, detailed, rambling on type of writer. I'll just stop writing now. Thanks again!

I was searching for a simple system to allow for transferring files between two systems without having them share components, but I couldn't find anything. Thus, I got to work writing my own. I tested this in my data centre for my development world - I like to store all my projects and downloaded programs in a huge archival storage system with many many raids. I think this counts as through enough testing.

I will warn you this is extremely beta! This is my very first working release. I'm already getting ready to work on a second revision. Most of the things wrong with it are stupid mistakes, such as forgetting to check origin ports, the very stupid fixes I used to patch the file transferring protocol (which I didn't need to fix on the server - I was having the client sent to the wrong port) and things like this. If you find any bugs, errors, or improvements, I'd appreciate the help.

The system is split into two main GitHub repositories, the repository for the server and the repository for the client

The user management system is not yet really implemented. The code by default has debug mode enabled, which creates a user named "debug" with the password of "debug". Security will later be added as a priority, but my first goal in this release, was just make it work. This took just about the entire day to program and debug, and I personally find it helpful. I hope you guys do too.

I made a video demo of the software too - again, the quality of the video is about as rough as the code. Link to the YouTube video is right here.

In OpenComputers we currently have a simple highly abstracted networking model. We do have ports, thankfully, but besides that the model is highly abstracted and thusly underpowered.

The first suggestion, is remove the abstraction on network communication in the first place. Allow this to be configured in the mod preferences file, but have it enabled by default. By default, there is no “hardware addresses”. A cable is just that, a cable. The only API by the mod is “broadcast and receive”. In the OS provided by the game by default could still have the same API as now. The different is it’d behave like broadcast, with packets being what are broadcasted. The new network card API (excluding any OS-level abstraction would look like this: 

- card:up()

- card:down()

- card:listen()

- card:transmit(raw)

Already, we now have a lot of extra things. It encourages crafting an encryption card (or whatever it’s called) because users can now snoop and preform MITM attacks on these packets, since there is no game mechanic (provides by the mod itself) preventing posing as somebody else. That also means you can simulate multiple networks cards in software while crafting only one. 

Second, something that kinda complements the idea of the first suggestion. Cable bandwidth limits. Just like normal cables, there is a limit to the amount of data we can cram into a single cable. Since the last suggestion removed the magic of hardware addresses, we actually need to listen to packets on the cable. We can only process so many packets so often. Now, us developers have an actual NEED to work on more complex softwares. We need to design things like intelligent multiplexing algorithms and routing softwares to efficiently receive, decode, process, bundle, re-encode, and transmit packets. 

There will also now be tiered network cards, that change things like the maximum number of packets a second the card can receive and transmit. It could also limit what kinds of cables the system can connect to. For example, you might need a tier four to connect to fibreglass. 

Keeping on the same note, Radio Frequencies. Just like cables, we can only send so much data through the air at once. One way we allow for more people talking wirelessly, besides just having everybody wait their turn in a long que to talk, is Frequency Multiplexing (FM). This also adds a new unique challenge, but powerful tool, to busy areas. You can allow a lot of people to talk at once using multiple frequencies, but you need to develop systems to send data to users on the right frequency to have them actually receive it.

Radio Frequencies would have different reasons to use it. Higher frequencies can transmit farther, but they use more power and they can be affected by things like weather (If I understand correctly some RF wavelengths can experience interference due to the size of the droplets). Lower frequencies would use less energy, but have lower range, but less susceptible to interference from things like weather (however, if memory serves right, lower frequencies are more noisy; translates in-game to possibly more failed packets and corruption). 

In the game, tiered wireless cards have been added. The higher tiers have the capability for wider bandwidth (meaning the range of RFs the card can use), multiple antennas (allowing use of multiple frequencies at once), and higher bandwidth (packets/second it can receive and transmit). To craft these, one good way could be like how you setup server racks. You can open a GUI where there are slots. You need better processor for higher throughput (packets/second) and higher memory for higher throughout volume capacity.

All my suggestion make the game more difficult, sure. That’s why I use it over ComputerCraft. I like having my computers be complex and expensive, they should be! They’re freaking computers.

This adds the ability to design a lot of new things, and new challenges. However, they won’t really effect new players. On your small one or Teo computer network running the default OS, everything will work perfectly fine. Only large-scale complex systems have these issues.