Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caleb/fix/this #227

Merged
merged 229 commits into from
Apr 3, 2024
Merged

Caleb/fix/this #227

merged 229 commits into from
Apr 3, 2024

Conversation

zollqir
Copy link
Collaborator

@zollqir zollqir commented Jan 31, 2024

This PR:

fixes the this bug by:
implementing JSFunctionProxy to proxy JSFunctions
implementing JSMethodProxy to act as a method wrapper for JSFunctions (the python user must create a JSMethodProxy manually and add it to an object since it is not possible to programatically determine if the user intended for a given JSFunction call to act as a function or method on the object)
treating JSMethodProxys passed to JavaScript as bound functions

doing above revealed GC bugs related to storing JSObjects on the heap incorrectly, causing the program to crash, so decided to fix those as well in this PR by:

removing memoizePyTypeAndGCThing, PyTypeToGCThing, and handleSharedPythonMonkeyMemory, replaced by:
implementing JSStringProxy, JSFunctionProxy, JSMethodProxy
rooting JSStrings, JSFunctions, and JSObjects directly in the proxying PyObject
PyString refcounts handled with a map referenced by JSExternalStringCallbacks
PyFunction refcounts handled by a FinalizationRegistry
PyDict/List/other Object refcounts handled by Py(Dict/List/Object)ProxyHandler's finalize method (merged-in prior work)

other:
implemented PyObjectProxyHandler to proxy any type of python object not already handled, including user-defined custom types

addresses (but does not fix) #151

closes #124
closes #172
closes #252
closes #253

@philippedistributive
Copy link
Collaborator

philippedistributive commented Jan 31, 2024

Can we refactor to take into account the common code and concept between PyObjectProxyHandler and PyDictProxyHandler?

@zollqir zollqir self-assigned this Feb 1, 2024
@philippedistributive
Copy link
Collaborator

philippedistributive commented Feb 1, 2024

This pr reopens #58
and the initially listed example in #172 is also no longer fixed

I didn't put tests in for those as I thought you would keep them fixed...let's make sure to have tests for those.

I just added a test for #58, feel free to improve it of course

Please also add a test for the #172 initial issue

@philippedistributive
Copy link
Collaborator

philippedistributive commented Feb 1, 2024

I can't get the finalization callbacks called even with explicit gc calls

Indeed there is no guarantee that it gets called at all:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/FinalizationRegistry#notes_on_cleanup_callbacks

Seems to put into question the FinalizationRegistry solution

BUT: it seems we are registering stack-allocated objects instead of Heap ones

@philippedistributive
Copy link
Collaborator

philippedistributive commented Feb 1, 2024

JSMethodProxyType is currently not used, never instantiated except in a test

Can you document explicitly somehow a good example of where and how to do this?

src/PyObjectProxyHandler.cc Outdated Show resolved Hide resolved
src/JSFunctionProxy.cc Outdated Show resolved Hide resolved
src/JSFunctionProxy.cc Show resolved Hide resolved
src/JSFunctionProxy.cc Outdated Show resolved Hide resolved
src/JSFunctionProxy.cc Outdated Show resolved Hide resolved
src/jsTypeFactory.cc Show resolved Hide resolved
src/pyTypeFactory.cc Show resolved Hide resolved
registerArgs[0].setObject(*jsFuncObject);
registerArgs[1].setPrivate(object);
JS::RootedValue ignoredOutVal(GLOBAL_CX);
JS_CallFunctionName(GLOBAL_CX, *jsFunctionRegistry, "register", registerArgs, &ignoredOutVal);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it will never be called back because the target to be reclaimed is stack-allocated: jsFuncObject

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, had to call JS::SetHostCleanupFinalizationRegistryCallback to set the callback that calls the callbacks ... 😵‍💫

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It still looks like we are registering a stack-allocated object, jsFuncObject

src/JSFunctionProxy.cc Show resolved Hide resolved
tests/python/test_functions_this.py Outdated Show resolved Hide resolved
@philippedistributive
Copy link
Collaborator

Let's have the PR description document in good detail the following:

"doing above revealed GC bugs"

@@ -210,57 +246,14 @@ PyTypeObject JSObjectItemsProxyType = {
};

static void cleanup() {
delete jsFunctionRegistry;
Copy link
Collaborator

@philippedistributive philippedistributive Feb 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this block below at line 368 seems outdated or needing update:

// TODO: Find a better way to destroy the root when necessary (when the returned Python object is GCed).
js::ESClass cls = js::ESClass::Other; // placeholder if rval is not a JSObject
if (rval.isObject()) {
JS::GetBuiltinClass(GLOBAL_CX, JS::RootedObject(GLOBAL_CX, &rval.toObject()), &cls);
if (JS_ObjectIsBoundFunction(&rval.toObject())) {
cls = js::ESClass::Function; // In SpiderMonkey 115 ESR, bound function is no longer a JSFunction but a js::BoundFunctionObject.
}
}
bool rvalIsFunction = cls == js::ESClass::Function; // function object
bool rvalIsString = rval->isString() || cls == js::ESClass::String; // string primitive or boxed String object
if (!(rvalIsFunction || rvalIsString)) { // rval may be a JS function or string which must be kept alive.
delete rval;
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done for functions, for strings still not handled but thats not within the scope of this MR

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's please do it now that we are in-context

@philippedistributive
Copy link
Collaborator

philippedistributive commented Feb 2, 2024

Did you want to uncomment and add several tests from test_arrays.py that I had put in commented-out?

@philippedistributive
Copy link
Collaborator

When it comes to:
Can you think of a way to reconcile both of these issues simultaneously in a clean way? As far as I am aware, there is not a way for a python function to know whether it's being called as a method or as a plain function

How about we always assume the this can be used and make sure it's there?

philippedistributive and others added 26 commits March 20, 2024 17:07
…ive-Network/PythonMonkey into philippe/pyTypeFactory-cleanup
…ory-cleanup

Several fixes including: allocate RootedValue for pyTypeFactory call on the stack instead of the heap
@philippedistributive philippedistributive merged commit e9195bc into main Apr 3, 2024
5 checks passed
@philippedistributive philippedistributive deleted the caleb/fix/this branch April 3, 2024 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants