ME autocrafter with memory leak

[kevinkk525 8]

Hi guys,

 

I programmed an auto-crafting software that is very basic at the moment and crafts every craftable in the ME in steps of 20 until they reach e.g. 100 items. Therefore it pulls the itemlist from the ME and every x secs the craftables list from the ME.

It stores all informations in tables and prints failed crafting requests.

 

So to the problem: It works perfectly most of the time and I do not know when exactly this changes but suddenly the free RAM drops a lot and keeps getting lower until it crashes. Normally i have it at around 2.9MB free RAM.

But I can't see where in my code this happens, if I free up all my tables that are global for debugging purposes almost no RAM gets free.

 

Hopefully someone sees the error in my program, feel free to use it if you think you need it

I always run it with: while true do p.produce() component.gpu.set(150,50,tostring(computer.freeMemory())) if computer.freeMemory()<80000 then break end end

---config section
local version="0.9.9.5b"
local production_number=100
local overproduction=production_number/5 --actually steps
local cpus=25
local craft_update_time=300
local update_time=30
---

local component=require("component")
local serialization=require("serialization")
local computer=require("computer")
local p={} --functions
local me
local me_address
tmp={}
for a,b in component.list("me_controller") do tmp[#tmp+1]=a end
tmp2={}
for i=1,#tmp do tmp2[i]=component.proxy(tmp[i]) end
for i=1,#tmp2 do if #tmp2[i].getItemsInNetwork()>0 then me_address=tmp[i] end end
tmp=nil
tmp2=nil 
inProgress={}
index={} --remove local for debugging
craftables={}
items={}
local ctime=computer.uptime()
local chtime=ctime-180
local amount=1000
failed={}
local last_i=1
local blacklist={}

--add saving to file and starting from file --> not working because of task-object
--add function to add to blacklist or remove from it
--generally: return table is a security risk! (or debug possibility...)

local function getIdent(t)
    ret=""
    for a,b in pairs(t) do 
        if a~="size" then
            ret=ret..a.."="..tostring(..","
        end
    end
    ret=ret:sub(1,ret:len()-1)
    return ret
end

function p.initialize(handler)
    f=handler
    f.addTask(p.produce)
end

function p.produce()
    me=component.proxy(me_address)
    if me==nil then
        print("no me, trying again")
    else
        if check_time()==true then
            p.craftables_pull()
            if check_production()==true then
                p.items_pull()
                local it=1
                for i=last_i,#craftables do
                    if #inProgress>=cpus then
                        break
                    else
                        local name=craftables[i].getItemStack()
                        local tmp=items[items.ident[getIdent(name)]]
                        if tmp then
                            if tmp.size<production_number then
                                local tmp2
                                tmp2=getIdent(tmp)
                                if index[tmp2]==nil and blacklist[tmp.label]==nil and blacklist[tmp2]==nil then
                                    inProgress[#inProgress+1]=craftables[i].request(20)--production_number+overproduction-tmp.size)
                                    if inProgress[#inProgress].isCanceled()==true then
                                        print(tmp2)
                                        table.remove(inProgress,#inProgress)
                                        --add second chance (lower production amount)
                                    else
                                        index[#index+1]=tmp2
                                        index[tmp2]=#index
                                    end
                                end
                                tmp2=nil
                            end
                            tmp=nil name=nil
                            it=i
                        end
                    end
                end
                last_i=it it=nil
                if last_i==#craftables then
                    last_i=1
                end
            end
        end
    end
    --f.addTask(p.produce)
    if not f then
        os.sleep(1)
    end
end

function p.listTasks()
    return inProgress
end

function check_time()
    if chtime<computer.uptime()-update_time then
        chtime=computer.uptime()
        return true
    else
        return false
    end
end

function check_production()
    if #inProgress==0 then
        return true
    else
        local tmp={}
        for i=1,#inProgress do
            if inProgress[i].isDone()==true or inProgress[i].isCanceled()==true then
                if inProgress[i].isCanceled()==true then
                    print(index[i])
                else

                end
                index[index[i]]=nil
                tmp[#tmp+1]=i
            end
        end
        for i=1,#tmp do
            table.remove(inProgress,tmp[i]-i+1)
            table.remove(index,tmp[i]-i+1)
        end
        tmp=nil
        if #inProgress<cpus then
            return true
        else
            return false
        end
    end
end

function p.craftables_pull()
    local ctime2=computer.uptime()
    if ctime2-ctime>craft_update_time or ctime2-ctime<0 or #craftables==0 then
        ctime=ctime2
        craftables=nil
        craftables=me.getCraftables()
    end
    ctime2=nil
end

function p.items_pull()
    items=nil
    items=me.getItemsInNetwork()
    items.ident={}
    for i=1,#items do
        local j=items[i]
        local name=getIdent(j)
        items.ident[name]=i
        name=nil j=nil
    end
end

function p.listFailed()
    return failed
end

return p

[dgelessus 26]

Never use globals in Lua. Lua globals are evil. Lua globals want to take over the world and enslave humanity. Every time you use a global in Lua, Hannibal Lecter violently murders a kitten.
 
*ahem*
 
The problem with globals is that they stick around even after the program is done running. Locals are automatically deleted when they are no longer accessible (i. e. when the function/program ends and they aren't stored elsewhere), but any object assigned to a global takes up extra memory until the global is reassigned. This can also cause odd bugs in other programs that run afterwards, which is the other big reason why you should never use Lua globals.
 
Looking through the first few functions, there are a few things that definitely should *not* be global:

• tmp and tmp2 are set to nil after the three for loops anyway and can be safely made local.
• inProgress, index, craftables, items and failed should all be local. If you need to see them for debugging reasons, make getters for all of them, like you did with p.listTasks and p.listFailed.
• local getIdent: ret is not meant to be used elsewhere and should be local.
• p.initialize: f only seems to be checked once in an if block at the end of p.produce. Since p.produce is never used outside of p.initialize that block should never run. Just write handler.addTask(p.produce) and see if that works.
• p.produce: me should be local.
• p.produce: Near the end last_i could be left out entirely, just reuse it.
• check_time and check_production should be stored in p.

None of these look like they would cause any major memory leaks though. In any case, I'd try making all variables local to see if that helps. If you use locals for everything, you shouldn't need to worry about manually setting them to nil, especially in functions.

[kevinkk525 8]

Thanks for the effort you put into this!

All the global variables you are talking about actually were local originally when i discovered the problem, they are just global at the moment for debugging purposes but of course i could have created getters.

But you are right about the 2 functions check_time and check_production, they should be either stored in p or be local, that was quick&dirty style shouldn't happen.

p.initialize is never called, could have deleted that (was for a future extension)

 

If it's nothing in the code it may something weird with the ME... guess I have to watch more carefully or just live with it.

 

This may be too much to ask but it would be awesome if you could take a look at my GUI-API and tell me what you think about it/the code: https://github.com/kevinkk525/OC-GUI-API

http://oc.cil.li/index.php?/topic/580-gui-api-064beta/

 

Thanks for your help!