Do not enable __ARM_NEON in recent compilers #5637

claudioandre-br · 2024-12-23T13:56:54Z

We can change autoconf to deal with it, but maybe the gcc version check is a better way. The autoconf change would be invasive.

Closes #5631.

It works fine (but the result is really odd):

Target CPU ......................................... arm, 64-bit LE
Target OS .......................................... linux-gnueabi
Cross compiling .................................... no
Legacy arch header ................................. arm64le.h
Optional libraries/features found:
Memory map (share/page large files) ................ yes
Fork support ....................................... yes
OpenMP support ..................................... yes (not for fast formats)
OpenCL support ..................................... no
Generic crypt(3) format ............................ yes
OpenSSL (many additional formats) .................. yes
libgmp (PRINCE mode and faster SRP formats) ........ yes
128-bit integer (faster PRINCE mode) ............... yes
libz (7z, pkzip and some other formats) ............ yes
libbz2 (7z and gpg2john bz2 support) ............... no
libpcap (vncpcap2john and SIPdump) ................. no
Non-free unrar code (complete RAR support) ......... yes
librexgen (regex mode, see doc/README.librexgen) ... no
OpenMPI support (default disabled) ................. no
Experimental code (default disabled) ............... no
ZTEX USB-FPGA module 1.15y support ................. no
Install missing libraries to get any needed features that were omitted.
Configure finished.  Now "make -s clean && make -sj4" to compile.
make with flags: -sj4
/usr/bin/ar: creating poly1305-donna.a
/usr/bin/ar: creating aes.a
/usr/bin/ar: creating ed25519-donna.a
/usr/bin/ar: creating secp256k1.a
[...]

Version: 1.9.0-jumbo-1+bleeding-d287408b37 2024-12-23 10:42:28 -0300
Build: linux-gnueabi 64-bit arm  AC OMP
SIMD: SSE2, interleaving: MD4:2 MD5:2 SHA1:1 SHA256:1 SHA512:1
AES hardware acceleration: AES-NI
$JOHN is ../run/
Format interface version: 14
Max. number of reported tunable costs: 4
Rec file version: REC4, charset file version: CHR3
CHARSET_MIN: 1 (0x01), CHARSET_MAX: 255 (0xff), CHARSET_LENGTH: 24
SALT_HASH_SIZE: 1048576
SINGLE_IDX_MAX: 32768, SINGLE_BUF_MAX: 4294967295
Effective limit: Max. KPC 32768
Markov mode max. level: 400, length 30
gcc version: 11.4.0
GNU libc version: 2.35 (loaded: 2.35)
Crypto library: OpenSSL 3.0.2 15 Mar 2022
GMP library version: 6.2.1
File locking: fcntl()
fseek(): fseek, ftell(): ftell, fopen(): fopen, memmem(): System's
times(2) sysconf(_SC_CLK_TCK) is 100
Using times(2) for timers, resolution 10 ms
HR timer: clock_gettime(), latency 42 ns
Total physical host memory: 15981 MiB
Available physical host memory: 15063 MiB
Terminal locale string: en_US.UTF-8
Parsed terminal locale: UTF-8

[...]
All 429 formats passed self-tests
[...]
1 2g 0:00:00:21 DONE 2/3 (2024-12-23 13:52) 0.09430g/s 2968Kp/s 2968Kc/s 4135KC/s buddy323d..243999d
Waiting for 1 child to terminate
2 1g 0:00:00:21 DONE 2/3 (2024-12-23 13:52) 0.04713g/s 3005Kp/s 3005Kc/s 3695KC/s jcorolla5..jasif007
[...]
1g 0:00:00:00  12.50g/s 1300p/s 1300c/s 1300C/s joan..joqq
Use the "--show" option to display all of the cracked passwords reliably
[...]
-------------------------------------------
###  Performed 14 tests in 388 seconds  ###
-------------------------------------------

We can change autoconf to deal with it, but maybe the gcc version check is a better way. The autoconf change would be invasive. Signed-off-by: Claudio André <[email protected]>

magnumripper · 2024-12-23T22:22:06Z

That is beyond weird and makes me even more curious what would happen if you give it a --enable-native-march. Will it detect NEON or SSE2? lol

claudioandre-br · 2024-12-24T12:11:46Z

If you don't --disable-native-tests, it will detect whatever exists, such as AVX2 or AVX512.

But john itself could do better in this scenario. Note what used to happen when using arm32le.h.

No SIMD:

Target CPU ......................................... arm, 64-bit LE
Target OS .......................................... linux-gnueabi
Cross compiling .................................... no
Legacy arch header ................................. arm32le.h
Optional libraries/features found:
Memory map (share/page large files) ................ yes
Fork support ....................................... yes
OpenMP support ..................................... yes (not for fast formats)
OpenCL support ..................................... no
Generic crypt(3) format ............................ yes
OpenSSL (many additional formats) .................. yes
libgmp (PRINCE mode and faster SRP formats) ........ yes
128-bit integer (faster PRINCE mode) ............... yes
libz (7z, pkzip and some other formats) ............ yes
libbz2 (7z and gpg2john bz2 support) ............... no
libpcap (vncpcap2john and SIPdump) ................. no
Non-free unrar code (complete RAR support) ......... yes
librexgen (regex mode, see doc/README.librexgen) ... no
OpenMPI support (default disabled) ................. no
Experimental code (default disabled) ............... no
ZTEX USB-FPGA module 1.15y support ................. no

Version: 1.9.0-jumbo-1+bleeding-f3517213a5 2024-12-22 16:54:19 +0000
Build: linux-gnueabi 64-bit arm  AC OMP
AES hardware acceleration: AES-NI
$JOHN is ../run/
Format interface version: 14
Max. number of reported tunable costs: 4
Rec file version: REC4, charset file version: CHR3
CHARSET_MIN: 1 (0x01), CHARSET_MAX: 255 (0xff), CHARSET_LENGTH: 24
SALT_HASH_SIZE: 1048576
SINGLE_IDX_MAX: 32768, SINGLE_BUF_MAX: 4294967295
Effective limit: Max. KPC 32768
Markov mode max. level: 400, length 30
gcc version: 11.4.0
GNU libc version: 2.35 (loaded: 2.35)
Crypto library: OpenSSL 3.0.2 15 Mar 2022
GMP library version: 6.2.1
File locking: fcntl()
fseek(): fseek, ftell(): ftell, fopen(): fopen, memmem(): System's
times(2) sysconf(_SC_CLK_TCK) is 100
Using times(2) for timers, resolution 10 ms
HR timer: clock_gettime(), latency 29 ns
Total physical host memory: 15981 MiB
Available physical host memory: 15076 MiB
Terminal locale string: en_US.UTF-8
Parsed terminal locale: UTF-8

magnumripper · 2024-12-24T12:47:30Z

Note what used to happen when using arm32le.h

So no SIMD but still AES-NI, which should be AES-CE if anything. Just weird.

solardiz · 2024-12-25T05:14:24Z

Those build-infos are indeed really weird. I wonder if those binaries pass tests, do they? And are they even ARM at all?

solardiz

I don't mind this change, so approving.

Do we know this is unneeded specifically starting with gcc 6 (not e.g. 5 or 7)? How do we know?

Rather than add two more lines, we could write:

#if !defined(__ARM_NEON) && __GNUC__ < 6

Oh, and how do we want this to behave when __GNUC__ is not defined?

claudioandre-br · 2024-12-25T11:07:04Z

Do we know this is unneeded specifically starting with gcc 6

No. But there is another check < 6 in the source code: let's say it's a kind of minimal version of GCC we trust.

it can work with gcc < 6 but we consider it problematic (see source code comment).
if it's gcc and elder, we activate __ARM_NEON.

claudioandre-br · 2024-12-25T11:14:26Z

I don't mind this change.

A suitable fix would be to let configure detect the environment and set a minimum version of gcc if older versions don't work on ARM. I mean, trust configure's result (gcc or whatever) and don't enable __ARM_NEON ourselves.

I'll let you guys merge this (or change the way configure works in this scenario).

claudioandre-br · 2024-12-25T13:27:00Z

I'm creating a Christmas release of the test packages (only); I need this branch. So, closing.

The patch itself remains saved in d287408.

solardiz · 2024-12-25T16:13:08Z

there is another check < 6 in the source code: let's say it's a kind of minimal version of GCC we trust.

This was (bad) news to me. Now that I check, we recently got this in mbedtls/aesce.c:

#elif defined(__GNUC__)
#   if __GNUC__ < 6
#       error "Minimum version of GCC for MBEDTLS_AESCE_C is 6.0."

I wonder what actually happens when we try to build for ARM with older gcc. Maybe we need to make changes to disable AES-CE on older gcc, but let them build proceed.

maint: not enable __ARM_NEON in recent compilers

d287408

We can change autoconf to deal with it, but maybe the gcc version check is a better way. The autoconf change would be invasive. Signed-off-by: Claudio André <[email protected]>

solardiz approved these changes Dec 25, 2024

View reviewed changes

claudioandre-br closed this Dec 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Do not enable __ARM_NEON in recent compilers #5637

Do not enable __ARM_NEON in recent compilers #5637

claudioandre-br commented Dec 23, 2024 •

edited

Loading

magnumripper commented Dec 23, 2024

claudioandre-br commented Dec 24, 2024

magnumripper commented Dec 24, 2024

solardiz commented Dec 25, 2024

solardiz left a comment

claudioandre-br commented Dec 25, 2024

claudioandre-br commented Dec 25, 2024 •

edited

Loading

claudioandre-br commented Dec 25, 2024 •

edited

Loading

solardiz commented Dec 25, 2024

Do not enable __ARM_NEON in recent compilers #5637

Do not enable __ARM_NEON in recent compilers #5637

Conversation

claudioandre-br commented Dec 23, 2024 • edited Loading

magnumripper commented Dec 23, 2024

claudioandre-br commented Dec 24, 2024

magnumripper commented Dec 24, 2024

solardiz commented Dec 25, 2024

solardiz left a comment

Choose a reason for hiding this comment

claudioandre-br commented Dec 25, 2024

claudioandre-br commented Dec 25, 2024 • edited Loading

claudioandre-br commented Dec 25, 2024 • edited Loading

solardiz commented Dec 25, 2024

claudioandre-br commented Dec 23, 2024 •

edited

Loading

claudioandre-br commented Dec 25, 2024 •

edited

Loading

claudioandre-br commented Dec 25, 2024 •

edited

Loading