[SOLVED] EOS Memory Leak

[high_octane]

Hello, I am here to report a memory leak in EOS. I've had my browser (Firefox) crash with an out of memory error logged to the console, during an EOS webtease. I actually ran out of system memory, so my computer was "locked up" for about 5 minutes. This is most likely to occur while playing one of the longer maze-like webteases.

It seems like once a resource is acquired (an image or sound file), it is never freed. In addition, duplicates of resources also become allocated, and not reused. I'm not a Javascript wizard, and I've only taken a brief look at the internals of the EOS engine, so I have no idea where this problem could be originating from.

In order to quickly reproduce this, play a webtease that allows you to choose a direction to move in, and keep going in opposite directions (go east, then west, then east, then west, etc). No matter how much time passes, the heap allocated memory continues to grow with each iteration.

I also tested this in Chromium and the same allocation behavior occurs.

[lolol2]

I have reported that in the EOS announcement thread too, waiting for feedback.

Really easy to reproduce, this test will crash chromium engine really fast because after 2,5 - 3 GB allocation the tease will stop working, not depending on how much system memory you have.

https://milovana.com/webteases/showteas ... c4a9b88b22

The latest update from firefox is very stable and will run until you will run out of memory.

[high_octane]

I have reported that in the EOS announcement thread too, waiting for feedback.

Whoops! I didn't see that.

I wish I were more well versed in Javascript, because then I'd have greater insight into what's going on. I know that in Javascript, you aren't allowed to explicitly manage memory. Deallocations are the job of the garbage collector. The only way for the garbage collector to free up heap allocated memory is for it to identify variables that are no longer needed, or no longer have references.

I know where the allocations are occurring. I believe it has to do with "createElement". grep returned 211 instances of it. I could try to fiddle around, but I'm not sure exactly how to go about freeing memory. If this were C, I could grep for something like "malloc" and make sure that there are the same number of "free" calls.

I did run a webtease under Chromium's performance devtool. It looks like that will be useful in detecting the exact cause of this issue, but I'm afraid that I'm too inexperienced. I guess I could try, though.

[kerkersklave]

I wish I were more well versed in Javascript, because then I'd have greater insight into what's going on.

As far as I remember, EOS javascript is generated by some compiler or optimizer, probably from typescript or something similar.Without the original source it is not much fun to debug that, and as far as I know the source is not available.
The GC can only free memory that is no longer referenced. So any kind of data structure that keeps growing can cause such a memory leak.
"createElement" is a method of the document API that allows to create new HTML elements in a document. But that does not mean that it has to be part of the visible document. If EOS keeps creating new elements and keeps them referenced somewhere the will remain in memory.

[seraph0x]

Just pushed a fix for this.

The issue was a bug in Eos Preloader which caused images that were referenced via wildcard to never be freed.

While I was at it, I also did some memory profiling and found some other much more minor memory leaks, which I also fixed.

Please note that when I open lolol2's test tease in Chrome now - the memory usage will still go up rapidly but this is just how Chrome manages its memory. All of the images that are loaded are marked garbage-collectable, so once Chrome runs out of memory, it cleans up and the memory usage drops back down. There should no longer be any crashes or frozen tabs.

Edit: I love that a tease that crashes your browser got a tease ID ending in 666.

[high_octane]

Just pushed a fix for this.

Awesome!

I just tested things out and it's working fine now; no more leak! Thanks for the quick fix, seraph0x!

[lolol2]

Just pushed a fix for this.

Thanks for the update, but when I test it with my 666 tease I still have a crash in chrome after 10sec with around 3gb memory usage.
Firefox still runs after around 2min into a freeze or extreme slow down too when system memory is full with 28gb in ram for the tease.

Is the updated version live?

[high_octane]

Just pushed a fix for this.

Thanks for the update, but when I test it with my 666 tease I still have a crash in chrome after 10sec with around 3gb memory usage.
Firefox still runs after around 2min into a freeze or extreme slow down too when system memory is full with 28gb in ram for the tease.

Is the updated version live?

I tested out your tease in FireFox. I'm not sure exactly how the underlying system works, but maybe your test tease is allocating resources faster than the garbage collector can free them? Of course, I have no clue. Javascript is not my expertise.

Ideally, there should be some way to restrict the number of resources that are allocated to a fixed buffer size, and once the end of the buffer is reached, the oldest indices are overwritten with the newest ones. I guess what I'm describing is a cache.

Under normal conditions, maybe the current code isn't an issue. I tested seraph0x's changes by playing one of the maze-like webteases for about 10 minutes. I rapidly switched from room to room, and the allocated space climbed a little, but was also freed quite quickly.

[lolol2]

Good point, just tried it with my tease too and same issue.
After around 150-200 pics chrome will use around 3gb and then things start to get buggy.
Buttons won't show up, text is missing or pictures won't show until the whole tease will stop working after a time.

The test tease from above has only <1gb files uploaded, so there must be something wrong when it will use 20gb+ memory.

Here is the same tease with a 1sec delay, it will take a bit longer but when chrome hits 3,5gb the same problem, no picture load.
https://milovana.com/webteases/showteas ... de8ac1bd51

Every page will claim 20-30mb in cache, so there must be something wrong with the preloading too.
This occurs only when you jump to a range of pages like "image-*" and the target pages have pictures which are also random from a galery.
Exactly what I'm using in my tease, so this is badly a real scenario which is breaking my tease for some users.

And if you have a tease which is crashed and you don't close this tab, all other tabs will also fail to load pictures too, so seems like it's also shared between sessions.

[seraph0x]

Thanks for the update, but when I test it with my 666 tease I still have a crash in chrome after 10sec with around 3gb memory usage.

Please test it in an incognito tab in the latest version of Chrome and refresh the page twice. (Eos client uses a service worker and refreshing the page should tell it to fetch the latest version of the client.) Make sure you do NOT have the developer tools open.

If you are still getting the issue, then I'm stumped. It's working fine for me now - I can watch the 666 tease for ten minutes in Chrome with no issues.

[lolol2]

Just tried it again and cleared the browser cache in chrome and firefox, but still the same issue. (also in inkognito mode)
So I just tried a fresh installation of chrome in a virtual machine with only 4GB memory.

Same issue, tease crashes after 2,1gb usage everytime and stops showing pictures.
CPU consumption is still there but no pics.

No developer tools or any addons installed.
Not sure what I'm doing wrong , can please someone else just start the tease and check the ram usage and if the tease is working over a longer time.
https://milovana.com/webteases/showteas ... c4a9b88b22

Tried it also with 8GB and 16GB memory in the VM but always at 2,1GB usage it stopps working... don't know why my computer with 32GB will last until 3,5GB before stopping.
Latest Win 10 1909 build just btw.

123.png (390.77 KiB) Viewed 2352 times

[boundupone]

Just tried it again and cleared the browser cache in chrome and firefox, but still the same issue. (also in inkognito mode)
So I just tried a fresh installation of chrome in a virtual machine with only 4GB memory.

Same issue, tease crashes after 2,1gb usage everytime and stops showing pictures.
CPU consumption is still there but no pics.

No developer tools or any addons installed.
Not sure what I'm doing wrong , can please someone else just start the tease and check the ram usage and if the tease is working over a longer time.
https://milovana.com/webteases/showteas ... c4a9b88b22

Tried it also with 8GB and 16GB memory in the VM but always at 2,1GB usage it stopps working... don't know why my computer with 32GB will last until 3,5GB before stopping.
Latest Win 10 1909 build just btw.


123.png

Hi, within less than a minute my ram and memory were way up trying that tease link you gave. Had to close the window to prevent a crash.

[tommarr]

Just tried it again and cleared the browser cache in chrome and firefox, but still the same issue. (also in inkognito mode)
So I just tried a fresh installation of chrome in a virtual machine with only 4GB memory.

Same issue, tease crashes after 2,1gb usage everytime and stops showing pictures.
CPU consumption is still there but no pics.

No developer tools or any addons installed.
Not sure what I'm doing wrong , can please someone else just start the tease and check the ram usage and if the tease is working over a longer time.
https://milovana.com/webteases/showteas ... c4a9b88b22

Tried it also with 8GB and 16GB memory in the VM but always at 2,1GB usage it stopps working... don't know why my computer with 32GB will last until 3,5GB before stopping.
Latest Win 10 1909 build just btw.


123.png

I tested this and I wasn't able to crash my browser. What I noticed is that Chrome tends to dump the stored data from ram to disk ie. to appdata. My memory usage was growing all the time fast but every 2-3GB it was being freed from the memory and moved to chrome appdata. I managed to user 50GB+ of my hdd this way. It also looked like memory usage would eventually get higher and higher as it was not able to dump data from memory to disk fast enough but the difference was so small that didn't test further for now

Are you maybe running out of disk space in your setup and is this difference between behaviour you see? Anyways its sufficient to say there is something wrong with the memory handling here if I'm able to use 50GB+ appdata and 3GB memory with single tease (yes, its a test but still). Clearly no data is being freed before closing the tease and this causes issues with longer teases

[RemiHiyama]

It's a little nuts how zealous the preloader is. I tried watching the network monitor for a while while lolol2's test tease was running, and it loads images enormously faster than how many were actually displaying. And how fast it switched images wasn't consistent. If the speed is supposed to be consistent... well, that might be another performance issue that's being introduced by the preloader being the way it is.

And okay, I can understand why, but it might be time to provide options to rein that thing in some, it doesn't seem to be a good fit for certain kinds of elaborate teases.

[fapnip]

I'm curious if people can reproduce this memory leak issue on my FapJack tease.

I noticed the the pre-loader tries to also pre-load images in pages linked via goto action. So if I had a page the was reloaded over and over again, pages it linked via goto actions nested in IFs would also have their images pre-loaded, over and over again.

Instead, I changed all my goto actions to pages.goto(...) evals, and made sure the game pages I continuously re-used track the image that's currently loaded and then only call (and return from) a different page to load the image I'm after for a given round.

Seemed to help in my testing, but I never heard back from those that were experiencing memory leaks if it actually solved the issue for them.