Jump to content
  • Sky
  • Blueberry
  • Slate
  • Blackcurrant
  • Watermelon
  • Strawberry
  • Orange
  • Banana
  • Apple
  • Emerald
  • Chocolate
  • Charcoal
  • 0
nmarshall23

Lua minifier

Question

4 answers to this question

Recommended Posts

  • 0

I've just written a lua parser in lua for that purpose.

The first part - to cut the code into 'tokens' like numbers, strings etc. - is lightweight enough to be executed from within OpenComputers.

The second part - to generate a tree representing the structure of the code - currently needs > 50 MiB during initialization.

(It should be possible to move the resource intensive parts out of OpenComputers via a file with a precomputed table.)

Obviously this is only reading the code.

But I already have ideas for the next step:

-remove comments and unnecessary whitespace (only needs the first part)

-rename variables (i.e. "thisIsMyLongVariable = thisIsMyLongFunction" -> "a = b0", can be improved by knowing variable visibility)

-replace "local a=1 local b=2" by "local a,b=1,2" (needs a more detailed analysis of the variables referenced behind the '=')

I'm going to publish the program in the forum when it's ready. (It might take some time though.)

Link to post
Share on other sites
  • 0

I'm using a LR(1) parser and I haven't separated building its parsing table and the actual parsing yet. (But that's easy since this table stays the same.)
I implemented it that way because serializing this table was less of a priority while testing. (It only takes 3 seconds to build with a lot of profiling included; fast enough for testing without the need to have a separate 'build step'.)
The parsing table itself consists of around 200 tables with 50k entries. (I'm considering adding code to make it a LALR parser which should cut down on the number of states.)
That's why. (In short: It's still work in progress and I wanted to get it running first. Now would be the time for optimization.)

EDIT: I've just implemented LALR and parsing table serialization.
The table now has 5000 entries and needs only 48 KiB disk space. (90% decrease in size)
Memory measurements using pure lua peaked at 1.5 MiB for a small file and 2.9 MiB for a 50 KiB file. So it's going in the right direction. (I still haven't done any optimization yet.)
There are several things that take their memory:
-a self made regex library to cut code into tokens
-a lot of memoization (memory -> speed tradeoff); I have to check here if I forgot to mark some tables as 'weak'.
-all the strings and tables that are the output of parsing need their space (approx. 20k strings with a few thousand tables for the 50 KiB file)

 

EDIT2: Writing a code compressor using the parsing library was quite easy. Just by removing comments and whitespace and by modifying local variables it's able to achieve a compression ration of about 2:1. (It's not even touching global variables or table indexing!)

I'm going to refine it a bit before publishing. That should be done within the next days.

Edited by mpmxyz
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use and Privacy Policy.