Skip to content

Commit 2e5825f

Browse files
itholicHyukjinKwon
authored andcommitted
[SPARK-47858][PYTHON][FOLLOWUP] Excluding Python magic methods from error context target
### What changes were proposed in this pull request? This PR followups for apache#46063 to exclude Python magic methods from error context target ### Why are the changes needed? We only need to logging the Apache Spark Column API functions for DataFrameQueryContext, but not Python native functions. ### Does this PR introduce _any_ user-facing change? No API changes, but the error message only contain Column APIs that are supported by Apache Spark ### How was this patch tested? The existing CI should pass. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#46215 from itholic/47858-followup. Authored-by: Haejoon Lee <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent a715007 commit 2e5825f

File tree

1 file changed

+9
-1
lines changed

1 file changed

+9
-1
lines changed

python/pyspark/errors/utils.py

+9-1
Original file line numberDiff line numberDiff line change
@@ -197,6 +197,14 @@ def with_origin_to_class(cls: Type[T]) -> Type[T]:
197197
"""
198198
if os.environ.get("PYSPARK_PIN_THREAD", "true").lower() == "true":
199199
for name, method in cls.__dict__.items():
200-
if callable(method) and name != "__init__":
200+
# Excluding Python magic methods that do not utilize JVM functions.
201+
if callable(method) and name not in (
202+
"__init__",
203+
"__new__",
204+
"__iter__",
205+
"__nonzero__",
206+
"__repr__",
207+
"__bool__",
208+
):
201209
setattr(cls, name, _with_origin(method))
202210
return cls

0 commit comments

Comments
 (0)