DoS: Arithmetic Operation Causes 100% CPU Usage and Unbounded Virtual Memory Growth

[sha0coder]

Bug report

Bug description:

There is a bug in the PyLongObject implementation during x_mul operations with certain values, triggering a Denial of Service (DoS) by causing 100% CPU usage and unbounded virtual memory growth.

 static PyLongObject *  x_mul(PyLongObject *a, PyLongObject *b)

First I triggered allocation errors in python3.10, and I also verified it with git version.

# dmesg

[1094342.309776] __vm_enough_memory: pid: 1137590, comm: python3, bytes: 156909473792 not enough memory for the allocation
[1094342.309781] __vm_enough_memory: pid: 1137590, comm: python3, bytes: 156909486080 not enough memory for the allocation
[1094342.309784] __vm_enough_memory: pid: 1137590, comm: python3, bytes: 156909961216 not enough memory for the allocation

Reproducing the issue is simple, here I triggered it in latest git verision main branch:

~/s/cpython ❯❯❯ ./python                                                                                                                                            
Python 3.15.0a0 (heads/main:9258f3da917, Jun  5 2025, 16:55:39) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> a = 241; b = 137; a = b << a; a = b**a;

With other operations I got a more convenient exceptions like:

OverflowError: too many digits in integer

But with this operation, the CPU starts consuming 100% and virtual memory keeps increasing.

/home/sha0/soft/cpython/Objects/longobject.c:3875

   3875             while (pa < paend) {
   3876                 carry += *pz + *pa++ * f;
   3877                 *pz++ = (digit)(carry & PyLong_MASK);
   3878                 carry >>= PyLong_SHIFT;
   3879                 assert(carry <= (PyLong_MASK << 1));
   3880             }

pwndbg> x/x pa
0x555555e089a0:	0x18f00998
pwndbg> x/x paend
0x555555e089c0:	0x00000001

paend pointer is expected that points after pa pointer.
And this causes a brk() syscall loop:

brk(0x5d96a1b3c000)                     = 0x5d96a1b3c000
brk(0x5d96a1b61000)                     = 0x5d96a1b61000
brk(0x5d96a1b85000)                     = 0x5d96a1b85000
brk(0x5d96a1ba9000)                     = 0x5d96a1ba9000
brk(0x5d96a1bca000)                     = 0x5d96a1bca000
brk(0x5d96a199f000)                     = 0x5d96a199f000
brk(0x5d96a19e0000)                     = 0x5d96a19e0000
brk(0x5d96a1a61000)                     = 0x5d96a1a61000
brk(0x5d96a1aa2000)                     = 0x5d96a1aa2000
brk(0x5d96a1ac7000)                     = 0x5d96a1ac7000
brk(0x5d96a1af3000)                     = 0x5d96a1af3000
brk(0x5d96a1b18000)                     = 0x5d96a1b18000
brk(0x5d96a189b000)                     = 0x5d96a189b000
brk(0x5d96a191d000)                     = 0x5d96a191d000
brk(0x5d96a195e000)                     = 0x5d96a195e000
brk(0x5d96a1982000)                     = 0x5d96a1982000
brk(0x5d96a19a3000)                     = 0x5d96a19a3000
brk(0x5d96a19cf000)                     = 0x5d96a19cf000
brk(0x5d96a19f0000)                     = 0x5d96a19f0000
brk(0x5d96a1a14000)                     = 0x5d96a1a14000
brk(0x5d96a1a35000)                     = 0x5d96a1a35000
brk(0x5d96a1a61000)                     = 0x5d96a1a61000
brk(0x5d96a1613000)                     = 0x5d96a1613000
brk(0x5d96a1695000)                     = 0x5d96a1695000
brk(0x5d96a1716000)                     = 0x5d96a1716000
...

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

[sha0coder]

The problem is not that loop itself, is comparing the pointer not the values.

[skirpichev]

I doubt it's a bug.

You tried something like 137 ** 484116095749277306305823515203559673336730587743919594525320099154177818624.

What you expect, computing enormous power with arbitrary precision arithmetic?

@ZeroIntensity, could you argue your decisions?

[sha0coder]

math.h pow() wouldnt do this ;) but being objects is logic, but maybe an exception

[skirpichev]

math.h pow() wouldnt do this ;)

math.h is not for arbitrary precision integer arithmetic.

[ZeroIntensity]

@ZeroIntensity, could you argue your decisions?

I didn't mean to remove pending, GitHub did it automatically when I applied other labels. I agree with you that using massive exponents will take... very long, to say the least.

math.h pow() wouldnt do this ;) but being objects is logic, but maybe an exception

What exception do you expect?

[sha0coder]

    a << a
    ~~^^~~
OverflowError: too many digits in integer

but with pow there is no error control and is keep allocating all the time