Grimrock

So I'm testing my project and there is a section where I use a "for i in allEntities(level) ..." and it's not working on my laptop where I'm playing. The thing is, it should work.
FACTS:
- if I type in the console "for i in allEntities(5) do print(i.name) end" it prints a huge list, so it's at least partly working.
- if i type in "for i in allEntities(5) do if i.name == "wall_button" then print("found one") end end" it prints "found one" four times
BUT, I carefully counted in the editor and I have EIGHT wall_button entities on level 5. So what could be breaking the search?
- if i type in "for i in allEntities(5) do if i.name == "wall_button" then print(i.x,i.y) end end" it prints four sets of coordinates. So then if I type in "for i in entitiesAt(5,27,17) do print(i.name) end" it prints "wall_button", but that was NOT one of the wall_button coordinates it found before. It found four of the eight with the allEntities, and it CAN see the others individually. So all I can think of is that something is breaking the for loop while it goes through all the entities on the level.

I can't reproduce this in the editor, and even if I do a quick export and test in game I can't reproduce it there. But I haven't been actually playing through the whole level up until that point when I do a quick export and test, so is there something I could have done while playing the level that would cause such behavior? What on earth can break the for loop?

I tried "for i in allEntities(5) do if i then if i.name == "wall_button" then print(i.x,i.y) end end end" but I only got the same four results. Are entities somehow created during gameplay that don't have a name and thus break the loop? Is there anything else I can do?

Was anything about allEntities changed in recent versions of Grimrock? I use steam on my desktop to auto-update, but for my laptop I always just copy over the newer files (like grimrock.exe, grimrock.dat, some dlls maybe)

I found that I miscounted, there are in fact 11 wall_buttons on my level 5, and the allEntities(5) search only finds 4 of them.

This code works:
for i=0,31 do for j=0,31 do for k in entitesAt(5,i,j) do if k.name=="wall_button" then print(k.x,k.y) end end end end
it prints all 11 sets of coordinates properly

Another contrast, if I type this:
local t={} for i in allEntities(5) do table.insert(t,i) end print(#t)
it prints 198. But with this code:
local t={} for i=0,31 do for j=0,31 do for k in entitiesAt(5,i,j) do table.insert(t,k) end end end print(#t)
it prints 465

I'm afraid I have no immediate insight, but for sanity checking sake you might want to consider exporting a full new copy of Grimrock from Steam and importing it on your laptop (instead of having a piece-meal upgrade of the files). You can either just take the whole LoG folder from /steamapps/ or you can actually export it into a bundle from Steam itself. It's possible (although I admit, unlikely) that there are some hidden files or perhaps a file that got missed and you have something out of date.

This seems unlikely though, it seems more like there is a problem with the allEntities(X) iterator. Just a few questions to help narrow it down...

1. Is it consistent? If you start the level again do you always get 198 (instead of the expected 465) when comparing allEntities to the for loops with entitiesAt? I wonder if something weird is going on like, after you have played the level for a bit an entity has died. That 'nil' entity might be breaking the allEntities loop but is tidied up in the entitiesAt method.

2. I don't even know if this is even possible, but do you have any code anywhere else that might have perhaps set a strange property to nil on an entity or maybe even somehow set an entity in the engine's entity list to nil (instead of calling destroy() on it say)?

Is it possible that your save game that causes this error was created before an upgrade and you are now playing it after the upgrade? Perhaps the upgrade process has broken the script (but starting a new game with a new party with the new version is fine).

All pretty unlikely suggestions I know - but I'm just trying to think what could be causing such a strange discrepancy!

JohnWordsworth wrote:you might want to consider exporting a full new copy of Grimrock from Steam and importing it on your laptop

I may try that later, but I think it's low on the likely list

JohnWordsworth wrote:1. Is it consistent? If you start the level again do you always get 198 (instead of the expected 465) when comparing allEntities to the for loops with entitiesAt? I wonder if something weird is going on like, after you have played the level for a bit an entity has died. That 'nil' entity might be breaking the allEntities loop but is tidied up in the entitiesAt method.

good idea to check this. So I first check my current save game, testing all the OTHER levels, they all have the same number of entities when I check with allEntities or entitiesAt. It's only level 5 right now with the problem/discrepancy. So I went back to a previous save when I was on level 4 at first and hadn't jumped in any pits, so no setting foot on level 5 yet at all. At that point the numbers for level 5 are THE SAME. So it seems apparent that something during my playing level 5 (or later parts of 4 perhaps - dropping something odd into a pit maybe) is what causes the problems

JohnWordsworth wrote:2. I don't even know if this is even possible, but do you have any code anywhere else that might have perhaps set a strange property to nil on an entity or maybe even somehow set an entity in the engine's entity list to nil (instead of calling destroy() on it say)?

I suppose its possible, I may have to review things (it's been a long time since I built this section). My current idea is to replay the level and periodically check the entity counts to see when they diverge and what I just did to cause it. Hopefully that will help discover the cause.

JohnWordsworth wrote:Is it possible that your save game that causes this error was created before an upgrade and you are now playing it after the upgrade? Perhaps the upgrade process has broken the script (but starting a new game with a new party with the new version is fine).

No, all playtesting is with a new game/new party, started yesterday

Thanks a lot for the ideas and feedback, now I have a couple things to try and hopefully get to the bottom of it!

I suggest that you add a high frequency timer that checks the entities all the time and play until the check fails. You should be able to find the culprit this way.

That's a good suggestion and I may try it again, but I already just caught up to where I was (after giving my champions a ton of health and strength to make it all very easy) and the counts always remained the same and are still the same - the bug didn't show up this time.

Of course I didn't do all the exact same things with every item and every piece of inventory and every this and every that, but went through all the rooms and did all the puzzles, etc. So I dunno.

I'd hate to have to replace all my dungeon allEntities with the 0,31 0,31 entitiesAt loops, I probably have a dozen or more. But maybe I'll do it just to be on the safe side since this bug scares me.

As you can read here: viewtopic.php?f=14&t=3884&start=470
the bug Rayeth was having with Grimwold's room seems to be this case too (at least, this is what happens if I load his savegame on my computer).

whoa, that's quite an interesting development!

- Is allEntities buggy and unreliable then?
- How exactly does it work?
- And why would entitiesAt in every cell work where allEntities fails?

Hi Komag. I wrote this iterator:

Edit: Don't use this one Edit: Then I wrote this iterator: Putting this in your script will redefine the global function allEntities. The redefined version will loop over each square of the level and do an entitiesAt that square. You should be able to copy/paste this into your code somewhere that it executes before your first call to allEntities. A couple of notes about this redefinition of allEntities:

1. It will generally return the entities in a different order than the original allEntities function did. This shouldn't really be an issue, since you shouldn't expect allEntities to iterate over the entities in any particular order anyway.

2. This iterator will not handle the situation that there are two ( or more ) entities in a level that are the same object in lua. I don't think this can happen in Grimrock, as each entity seems always to be a distinct lua object. If it does happen, the iterator will enter an infinite loop. This seems to cause the editor to hang. I've never tried intentionally exporting a script with an infinite loop and starting it in the actual game. I imagine it would also hang.
Edit: The updated iterator doesn't have this problem.

3. This would all be a lot easier to code if Grimrock lua supported upvalues <--> supported closures. Have the developers said why it doesn't? Have they said if there are or aren't any plans to support upvalues/closures/proper lexical scoping in the future?

That's very interesting and promising. I've tried it a couple ways but not sure how to tell if it's working. I tried just adding that code (your updated version) to my init.lua file within my dungeon folder. Seems fine, but I can't tell if it's actually having any effect or if my dungeon is still just using the allEntities normally. So I added a print command just before c = c + 1, but nothing ever printed.

So I tested by renaming it to tonsOJunk, then in the game I typed in the line:
local t={} for i in tonsOJunk(5) do table.insert(t,i) end print(#t)
but it spit out error "attempt to call global 'tonsOJunk' (a nil value)"

Then I tried just putting in a run-at-map-start script entity in the editor and doing the same thing, didn't work.

Then I tried it by using the script name in front, like this
local t={} for i in allEntScript.tonsOJunk(5) do table.insert(t,i) end print(#t)
and it works great, including the print command I stuck in there to see if anything was happening!

But I get the impression I shouldn't have to add the allEntScript. in front, is there something I'm missing?

Even if I have to add that part, it is a good solution, because I can just do a quick search/replace on the dungeon.lua file with notepad++, easy peasy

Thanks a bunch for the help so far!