Graal js 22.3.1 broken on armhf big-endian platform

[extensia]

We have a Java application running on an embedded Linux armhf platform that uses GraalJS on a stock JDK (Liberica Standard JDK 17.0.6+10 arm 32 for Linux). We had been using the 21.2.0 release without issues, but decided to upgrade to 22.3.1 since we recently upgrade from Java 8 to Java 17.

Following the upgrade we found that our JS scripts/modules are now failing without providing any useful diagnostics. In particular we have been seeing the following error. The same application and JS scripts/modules run perfectly on Windows 10 and Ubuntu 20 machines with 64-bit Intel processors.

org.graalvm.polyglot.PolyglotException: java.lang.NoClassDefFoundError: Could not initialize class com.oracle.truffle.js.lang.JavaScriptLanguage
    at app//com.oracle.truffle.js.lang.JavaScriptLanguageProvider.create(JavaScriptLanguageProvider.java:51)
    at app//com.oracle.truffle.polyglot.LanguageCache.loadLanguage(LanguageCache.java:548)
    at app//com.oracle.truffle.polyglot.PolyglotLanguageInstance.<init>(PolyglotLanguageInstance.java:100)
    at app//com.oracle.truffle.polyglot.PolyglotLanguage.getInitLanguage(PolyglotLanguage.java:152)
    at app//com.oracle.truffle.polyglot.PolyglotLanguage.ensureInitialized(PolyglotLanguage.java:142)
    at app//com.oracle.truffle.polyglot.PolyglotLanguage.getOptionsInternal(PolyglotLanguage.java:134)
    at app//com.oracle.truffle.polyglot.PolyglotLanguage.getOptionValues(PolyglotLanguage.java:202)
    at app//com.oracle.truffle.polyglot.PolyglotContextConfig.getLanguageOptionValues(PolyglotContextConfig.java:407)
    at app//com.oracle.truffle.polyglot.PolyglotSharingLayer.collectLanguageOptions(PolyglotSharingLayer.java:577)
    at app//com.oracle.truffle.polyglot.PolyglotSharingLayer.claimLayerForContext(PolyglotSharingLayer.java:166)
    at app//com.oracle.truffle.polyglot.PolyglotEngineImpl.claimSharingLayer(PolyglotEngineImpl.java:370)
    at app//com.oracle.truffle.polyglot.PolyglotContextImpl.claimSharingLayer(PolyglotContextImpl.java:615)
    at app//com.oracle.truffle.polyglot.PolyglotLanguageContext.ensureCreated(PolyglotLanguageContext.java:576)
    at app//com.oracle.truffle.polyglot.PolyglotLanguageContext.ensureInitialized(PolyglotLanguageContext.java:691)
    at app//com.oracle.truffle.polyglot.PolyglotContextImpl.eval(PolyglotContextImpl.java:1468)
    at app//com.oracle.truffle.polyglot.PolyglotContextDispatch.eval(PolyglotContextDispatch.java:63)
    at app//org.graalvm.polyglot.Context.eval(Context.java:399)
    at app//org.graalvm.polyglot.Context.eval(Context.java:425)

While troubleshooting the problem we also found that GraalJS exception messages are garbled and contain what look to be Chinese characters. After a bit of debugging it appears that the problem is due to a byte ordering problem since reversing the order of each byte pair in the debugger results in ASCII text.

The garbled message problem is readily reproduced using the following snippet of code that intentionally triggers a coercion error:

        Context context = Context.create(JavaScriptLanguage.ID);
        Value result = context.eval(JavaScriptLanguage.ID, "new ArrayBuffer(10);");
        result.asString();

Here is the message produced on the armhf platform:

java.lang.ClassCastException: Cannot convert '牁慲䉹晵敦筲'(language: JavaScript, type: ArrayBuffer) to Java type 'java.lang.String' using Value.asString(): Invalid coercion. You can ensure that the value can be converted using Value.isString().

And the expected error message on Windows 10:

java.lang.ClassCastException: Cannot convert 'ArrayBuffer{}'(language: JavaScript, type: ArrayBuffer) to Java type 'java.lang.String' using Value.asString(): Invalid coercion. You can ensure that the value can be converted using Value.isString().
We are still investigating the source of the NoClassDefFoundError exception, but believe it is likely related to the apparent byte ordering problem that is causing the garbled exception messages.

[extensia]

The issue with garbled exception messages was introduced in Version 22.1.0. That's the release that moved the internal string representation to the new TruffleString type.

It appears Release 22.0.0 works correctly. We are still working through regression tests to confirm all is well with that release.

[oubidar-Abderrahim]

Hi, Thank you for sharing this, We currently do not support Linux armhf arm32, the currently supported platforms for GraalVM are:

• Linux amd64
• Linux arm64
• darwin amd64
• darwin arm64
• Windows amd64

[extensia]

True, but it worked fine up until the recent releases. I believe the issue affects big endian platforms in general. I imagine that will be a problem for users running on AArch64 which is becoming increasing popular in cloud environments.

[oubidar-Abderrahim]

could you please share a small reproducer that we can test with and verify this issue? Thanks

[jpovixwm]

I believe I'm hitting the same issue when trying to run graaljs in an OpenJ9 JVM. Versions 22.1.0 through 22.3.2 produce unexpected results when manipulating strings.
Version 22.0.0.2 appears to work normally. Version 23.0.0 is a bit curious: the issue is not reproducible when using the artifacts from Maven, but can be reproduced if I build the 23.0.0 tag on my own.
I've set up a repo with a reproduction of this problem: https://github.com/jpovixwm/graal-js-garbled-strings-issue
Here's what I get with ext.graalVersion = '22.3.2' in build.gradle:

$ ../temurin17-jdk/bin/java --version
openjdk 17.0.7 2023-04-18
OpenJDK Runtime Environment Temurin-17.0.7+7 (build 17.0.7+7)
OpenJDK 64-Bit Server VM Temurin-17.0.7+7 (build 17.0.7+7, mixed mode, sharing)

$ ./gradlew -Dorg.gradle.java.home="../temurin17-jdk/" clean run

> Task :run
PASS

BUILD SUCCESSFUL in 1s
3 actionable tasks: 3 executed

$ ../semeru17-jdk/bin/java --version
openjdk 17.0.7 2023-04-18
IBM Semeru Runtime Open Edition 17.0.7.0 (build 17.0.7+7)
Eclipse OpenJ9 VM 17.0.7.0 (build openj9-0.38.0, JRE 17 Windows 10 amd64-64-Bit Compressed References 20230418_439 (JIT enabled, AOT enabled)
OpenJ9   - d57d05932
OMR      - 855813495
JCL      - 9d7a231edbc based on jdk-17.0.7+7)

$ ./gradlew -Dorg.gradle.java.home="../semeru17-jdk/" clean run

> Task :run
FAIL: Unexpected result: "??". (bytes: 41414141).

BUILD SUCCESSFUL in 1s
3 actionable tasks: 3 executed

FWIW, I did a git bisect between vm-22.0.0.2 and vm-22.1.0 and I ended up with commit 9967bd3

[iamstolis]

IBM Semeru has disabled compact strings by default. There was a bug in TruffleString library that caused it to produce malformed strings when it was used on JVM with disabled compact strings. This bug was fixed in master (by oracle/graal@2e942e6) and the fix was backported into GraalVM 23.0.x release branch (branch release/graal-vm/23.0) by oracle/graal@2e30d3b. You can also work around it using -XX:+CompactStrings JVM command line option.

[jpovixwm]

Ah, so it's in fact unrelated to the original report. Thank you for the explanation, I appreciate it.
Out of curiosity: would the fix be backported also to the upcoming 22.3.3 release?

[iamstolis]

Out of curiosity: would the fix be backported also to the upcoming 22.3.3 release?

I am not aware of the backport. So, probably not. Honestly, while I understand that it is an ugly bug when you hit it, it is quite a corner case. All the distributions that we provide have compact strings enabled and I don't see a reason to disabled them. Who would want their strings to consume more memory? I wonder why the JDK from IBM have them disabled by default.