Out of Memory Errors

[wiwichips]

Issue type

Bug

How did you install PythonMonkey?

Installed from pip

OS platform and distribution

Ubuntu 22.04.4 LTS x86_64

Python version (python --version)

3.10.12

PythonMonkey version (pip show pythonmonkey)

0.8.0

Bug Description

I notice PythonMonkey will OOM in weird circumstances, and looking at ps -p $pid -o rss= , the rss memory usage (measured in kb) only hangs around the 200mb range (would be hilarious if I was off by a factor of 10 --- double check my work)

---

One thing to note, maybe this is expected? If it is, it would be sad - so hoping it's a bug!

Standalone code to reproduce the issue

#!/usr/bin/env python3
"""
Works at 1020000 elements, OOMs at 1030000 elements...
"""

import pythonmonkey as pm

oomer = pm.eval("""
function oomer()
{
  let arr_size; // change array size to different values
  arr_size = 1020000; // success!
  arr_size = 1030000; // OOMs (ON MY SYSTEM! - maybe you'll have to up it to OOM on yours!)

  const bigArray = [];
  for (let i = 0; i < arr_size; i++)
    bigArray.push(i + 0.1 / (i + 0.123456)); // randomish floats, fattened up by json.stringify A LOT later


  //let seed = 1; bigArray.sort(() => (Math.sin(seed++) * 10000) % 1 - 0.5); // TODO unnecessary, remove later


  // these initial values don't really matter per se, it's more just about how they're serialized
  console.log(`Array length: ${bigArray.length}`);
  console.log(`Array bytes : ${bigArray.length * 8}`); // 8 bytes per js number??? 
  console.log(`Array MB    : ${bigArray.length * 8 / 1000000}`);

  // The following code is baed off of encodeJobValueList in dcp-client/job/index.js
  const jsonedElementsArray = [];
  for (let i = 0; i < bigArray.length; i++)
  {
    jsonedElementsArray.push(JSON.stringify(bigArray[i]));

    // logging
    if (i % 10000 === 0 && i > 600000) // skip first 600000 then only print every 10000 elements
      console.log(i, ' -- ', bigArray[i]);
  }

  // now we calculate the total length of all the strings in the array and see how much memory they use 
  console.log(`JSONed Array length: ${jsonedElementsArray.length}`);
  console.log(`JSONed Array bytes : ${jsonedElementsArray.reduce((acc, str) => acc + str.length, 0) * 2}`); // 2 bytes per character
  console.log(`JSONed Array MB    : ${jsonedElementsArray.reduce((acc, str) => acc + str.length, 0) * 2 / 1000000}`);
}
oomer
""")

oomer()

Relevant log output or backtrace

Traceback (most recent call last):
  File "/home/will/git/oomy/./py-memory-bizarreness.py", line 63, in <module>
    oomer()
pythonmonkey.SpiderMonkeyError: uncaught exception: out of memory

Additional info if applicable

I started this investigation when Dan reported an OOM during dcp job deploy for large job arguments, reproring that I found PythonMonkey would allocate more memory than nodejs and take far longer to execute the same amount of code

Refer to my slack messages:
- https://dstrbtv.slack.com/archives/C02FS7B9PCP/p1724702701635389
- https://dstrbtv.slack.com/archives/C02FS7B9PCP/p1724773152533229

---

Another similar issue is described here, it could be related! https://github.com/Distributive-Network/PythonMonkey/issues/420#issuecomment-2313497379

What branch of PythonMonkey were you developing on? (If applicable)

No response

[wiwichips]

It is 10000% possible I made mistakes in my measurements etc, please check all of my work and ensure it is valid

[wiwichips]

Deleted Spam comments [1] [2]

[wiwichips]

NOTE ----

Side note: I was experimenting to see how much makes PM OOM by running this:

import pythonmonkey as pm
pm.eval("""
try {
  const chunks = [];
  let size = 1e6; // Start with 1MB chunks

  while (true) {
    chunks.push(new Array(size).fill(0)); // Allocate memory
    console.log(`Allocated ${chunks.length * size} bytes`);
  }
} catch (e) {
  console.error('Out of memory:', e.message);
}
""")

But it seemed to freeze at around 2GB of allocated memory, despite this, my system was using about 20gb of memory across my RAM and swap...BUT I never actually got an OOM error running this code, unlike the example in the above ticket description

[philippedistributive]

one more example we are looking at:

#!/usr/bin/env python3
import pythonmonkey as pm
dcp_client = pm.require('dcp-client')

lst1 = []
lst2 = []

#arr_size = 2500
arr_size = 5000
arr_size = int(arr_size)
print(arr_size)

for i in range(arr_size):
lst2.append([])
for j in range(arr_size):
lst2[i].append(i + j + (j + 0.1) / (i + j + 0.123456))
lst1.append(i * i)

args2 = [lst1, lst2]

data = [1, 2, 3, 4, 5]

work_function = """
function myEpicWorkFn(datum, arg1, arg2)
{
progress();
return arg1.length + arg2.length; + datum
}
"""

async def pringle():
dcp = pm.globalThis.dcp
# define the compute workload (job)
job = dcp.compute['for'](data, work_function, args2)

# add event listeners for debug logs
job.on('readystatechange', print)
job.on('accepted', lambda: print(job.id))
job.on('result', lambda x: print(f'New result for slice "{x.sliceNumber}" is "{x.result}"'))
#job.computeGroups = [{ 'joinKey': 'joseph', 'joinSecret': 'pringle' }]

# deploy the compute workload to the DCP network
results = await job.exec()
print(results)

dcp_client'init'

Latest error output:

exec
init
preauth
deploying
Exception in callback None()
handle:
Traceback (most recent call last):
File "/home/philippe/.pyenv/versions/3.11.7/lib/python3.11/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
pythonmonkey.SpiderMonkeyError: uncaught exception: out of memory

Unhandled error in exception handler
context: {'message': 'Future exception was never retrieved', 'exception': SpiderMonkeyError('out of memory'), 'future': <Future finished exception=SpiderMonkeyError('out of memory')>}
Traceback (most recent call last):
File "/home/philippe/.pyenv/versions/3.11.7/lib/python3.11/asyncio/base_events.py", line 1808, in call_exception_handler
self._exception_handler(self, context)
File "/home/philippe/Sources/PythonMonkey/python/pythonmonkey/helpers.py", line 53, in simpleUncaughtExceptionHandler
pm.eval("(err) => console.error('Uncaught', err)")(error)
pythonmonkey.SpiderMonkeyError: uncaught exception: out of memory

Unhandled error in exception handler
context: {'message': 'Exception in callback None()', 'exception': SpiderMonkeyError('uncaught exception: out of memory\n'), 'handle': }
Traceback (most recent call last):
File "/home/philippe/.pyenv/versions/3.11.7/lib/python3.11/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
pythonmonkey.SpiderMonkeyError: uncaught exception: out of memory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/philippe/.pyenv/versions/3.11.7/lib/python3.11/asyncio/base_events.py", line 1808, in call_exception_handler
self._exception_handler(self, context)
File "/home/philippe/Sources/PythonMonkey/python/pythonmonkey/helpers.py", line 53, in simpleUncaughtExceptionHandler
pm.eval("(err) => console.error('Uncaught', err)")(error)
pythonmonkey.SpiderMonkeyError: uncaught exception: out of memory

makes it look like something
@Xmader
has handled before, see
Future exception was never retrieved
and
Unhandled error in exception handler

[philippedistributive]

for the main code sample, in pm, there is no stack associated with the JS exception, only the message
uncaught exception: out of memory

[philippedistributive]

Looking at https://discourse.mozilla.org/t/how-does-spidermonkey-deal-with-memory-limitations/123670/2
We need to expand the maximum GC heap

we need to call
JS_SetGCParameter(GLOBAL_CX, JSGC_MAX_BYTES, (uint32_t)-1);
in the initial setup

with this the main example is fixed

[philippedistributive]

alternate example now getting different result -> onto its own ticket: #423

[philippedistributive]

The main example with arr_size = 1030000 now works with
arr_size = 100990000, more than 100 time more
Pretty convincing! The default max heap size was way too low it seems
10:55
the lower number recorded RSS at 100MB
the higher number recorded at 4GB

[philippedistributive]

we can achieve the same max heap increase by calling
JS_NewContext((uint32_t)-1);
instead of
JS_NewContext(JS::DefaultHeapMaxBytes);
which sets the initial maximum