#include <sys/ptrace.h>
long int ptrace(enum __ptrace_request request, pid_t pid,
void * addr, void * data)request decides what to be done Pid - Pid of the traced process OR thread Addr offset in the user space data will be written to, if instructed to do so (depending on request)
This is what we’re using: A parent fork()’s and the resulting child calls
ptrace(PTRACE_TRACEME, ..) to initiate being traced by the parent.
whenever ptrace is called it first locks the kernel (I assume to mutex other ptrace calls or anything else compromising the trace attempt), just before returning it unlocks the kernel. Depending on the request different things happen in between the locks. What follows is an overview of the requests behaviour
Any signals (except SIGKILL) either delivered from outside or from exec calls made by the process, causes it to stop and lets the parent decide how to proceed. This request only sets a flag that grants permission to trace the process. Important insight: All the parameters other than *request* are ignored!
This is pretty handy, you can make the current process the tracer of the target
process, “become the parent of that process by attaching yourself to it”. But getpid()
will return the child’s real parent!
- Note: the init process is a big deal - nobody is allowed to trace or control it! A process is also not allowed to trace itself.
What actually happens:
- In the background all the cases are handled (1) is it the init (2) itself aka current process (3) already traced process, before permission to attach is granted.
- “the child is removed from the task queue and its parent process field is changed (…). It is put to the queue again in such a position that init comes next to it”
- (!)Interesting: finally a =SIGSTOP= signal is delivered to it. This could be useful to attach to the process in the moment when inspecting the process memory is of interest!
The inverse of Ptrace_Attach: (1) reset the Ptrace flag, (2) move child to original
position of the task queue (note task=process in kernel lingo) and finally (3) the pid
of the real parent is written in the parent field. “The single-step bit which might
have been set is reset”.
addr is ignored
data arg the parent can send an exit code for the child
to continue child execution
will also continue execution of the child, but will stop on every syscall invocation the child issues. This can be used to implement “strace”.
like PTRACE_SYSCALL except we stop here on every instruction!
fetches registers of the process such as: general purpose, floating point.
data will store these, furthermore it expects a struct user_regs_struct variable!
Read a word from the process’ memory. apparently they’re identical on Linux because “Linux does not have separate text and data address spaces”. addr is the byte offset from which to read the word[fn:3] data is ignored
Read a word from the tracee’s USER area (register + open files, etc. see below) addr is the byte offset from which to read the word data is ignored
- if PTRACE_PEEK* calls are unsuccessful they return zero!
- “On error, all request return -1, and errno is set appropriately (…)”
Reads all the registers of the tracee’s USER area and puts it in the given struct of
type user_regs_struct
data expects a pointer to user_regs_struct, so pass by reference, and fills it with
the registers of the tracee
addr is ignored
The inverse of the above: write the given word at the offset (in bytes) specified in the data or user area of the tracee. data word to write to tracee addr byte offset where to write to
In a multithreaded application a single ptrace() call can only attach to a single
thread at a time, never the whole process!
This means that if we ptrace() a thread of a process. We might peekdata a particular address and at the same time the memory pointed to could be changed by another thread that shares it! This is where we want to know all the threads of a process, so we can either just SIGSTOP them all, or see who share the same memory regions and might be responsible for some changes
To get a visual overview use pstree It will show you a tree of processes by pids
where the {curly brackets leaves} indicate the child threads of the nodes process!
/proc/pid/task has entries for all the threads in the process. The name of each directory is even the tid (thread ID) for each process. The thread directories have much in common with the layout of the proc/pid. You can inspect then use the files herein to see which resources are indeed shared between the threads.
Making it visual again:
This means that if we take a node from the pstree output and inspect the associated
pid proc/pid then the task/ dir will be the successor nodes/leaves of the
process/thread!
top -H -p <pid>
Is most useful as it shows all the threads of the process and, with their tid and
there status and periodically updating!
Since we can only ptrace particular threads, we might find the subsequent calls to peekdata will yield changing values on some addresses. This is due to some other thread that is still running and has access to the same memory segment and changing it.
We might then either send SIGSTOPs to all the threads of a process or we can find the culprit thread (in an attempt to gain more knowledge about the program itself) by inspecting the /status file of each task. We might find some threads are sleeping and hence innocent of memory tampering.
Given a multithreaded application with the threads, who all share the same address-space foo-thread bar-thread qux-thread
And the process running our ptrace calls (in this case a single threaded process): tracer-thread
If we ptrace(PTRACE_ATTACH ..) to foo-thread, we make tracer-thread another parent of foo-thread. In accordance with ptrace’ documentation, a SIGSTOP is send, that stops the thread. But this SIGSTOP send by tracer-thread only stops foo-thread not any other thread!
But if we send a SIGSTOP to foo-thread directly, i.e. not through the means of the attach ptrace call, then the whole process, and its threads foo- and bar-thread are stopped.
This is an instance where the peculiarity of ptrace becoming a makeshift foster home for threads has to be taken into account.
When a process wants to invoke a system call, it puts the arguments to system calls in
registers rdi rsi rdx rcx r8 r9 -in that order- and calls the soft interrupt[fn:1] 0x80 (TODO:
0x80 on i386). This code is put in the =rax=[fn:2] register on x86_64 architecture machines.
Can be found in /usr/include/asm-generic/unistd.h !
Signals can be send to a process using a command like kill <signal> <pid>
(see man 7 signal)
Each signal has a default dispositions that determine how a process will react when
the signal is delivered to it. (change disposition with function like sigaction())
Some default dispositions:
| Term | Default action is to terminate the process |
| Core | terminate process and dump core |
| stop | stop the process |
| con | continue process if it is currently stopped |
“Disposition is a per-process attribute (…) In a multithreaded application the disposition of the signal is the same for all threads in the process””
A signal may be generated for a process as a whole or for a specific thread. For example SIGSEGV (invalid memory reference) is thread specific
The following is on x86, not sure if it still applies to x86_64
First every process contains
- the same kernel space
- its own user mode space
Of the roughly 4GB addressable by 32-bit x86, the 1GB can be kernel space and 3GB user space.
Via page tables each address is mapped to physical memory. Since each process has to share its virtual address space with the same fixed kernel space, whenever a user mode program tries to touch it a page fault is signalled for example.
But this also means that “kernel code is always addressable, ready to handle interrupts or system calls at any time”. So finally to understand the fixed kernel space: whenever a process switch happens, only the user space is what changes!
The User Mode Space consists of the following distinct memory segments, listed in address order from highest to lowest (btw “growing downwards” means new elements append to this space at the one end with the lower address)
Finally these can be examined by looking at /proc/<pid>/maps!
- Stack (grows downwards) Contains the pageframes of functions, their environment and whenever you call nested functions the stack grows adding a new frame, and the stack is destroyed when the function returns
- Memory Mapping Segment (grows downwards)
This is where the mapping of shared objects, loaded dynamic libraries and the
contents of files when needed reside. This portion is filled by using the
mmap()function. Also big object runtime object (by default about 128kb) get dumped here. For example if you do hugemalloc()the area might be mapped here instead.Btw it is also possible to create an anonymous memory mapping meaning the the area mapped doesn’t correspond to any actual file, for example when using program data or perhaps creating new buffer in Emacs before saving it to an actual file.
(sbcl note: when you load libc.so to create bindings to it with cffi, you can see that it has been indeed mapped into the memory segment by looking up /proc/<pidof sbcl>/maps because it occurs right under the [stack:] portion of addresses !!!)
- the Heap (grows upwards to the Memory Mapping segment!)
This is where runtime object live! Whenever you create an object with the
newoperator or simply to a cmalloc(). (So this is where we will focus our ptrace process manipulation efforts!)If the Heap runs out of space, it can be enlarged with
brk()system call (so the kernel must get involved at this point. Heaps are complex to implement having to deliver efficiency and speed and avoid fragmentation - BSS segment
stores the contents declared uninitialised static global variables (static int
x;). These might be useful!
(BSS stands for “Basic Service Set” if you must know!)
- Data segment stores the initialized i.e. with definition of declared static global variables (meaning in the source code it already is initialized: static int = 22;). This also means it is not anonymous mapping to the values in the program’s binary image!! This is also explains why you will find entices static variables in the executable binary file with a hex-editor.
- Text segment stores the executable with its machine code instructions and String literals. Hm, writes to this area cause segmentation faults to avoid pointer bugs. (This might mean that Lisps image based development is a Heap/mmap() Memory Mapping magic show.)
“Data Segment”, people sometimes refer to the Heap + BSS + Data Segment as the simple “data segment”
nm <obj-file.o>
will list symbols from an object files and also to which memory segment they belong!
The above memory segments used to start at exactly the same virtual addresses. This made them vulnerable to exploits. So nowadays modern operation systems employ ASLR, adding a random offset to these address.
From what I could gather ASLR is only applied at process initialisation, so far, and doesn’t apply at run-time or at process switching. There was a paper on ASLR Re-Randomization, but a paper is for the future..
It can be easily disabled see (below) but it might be not necessary if it only occurs once the process is created since we always will try to ptrace it once its created.
So apparently /proc is the user space interface to process information, and should be used to read out process information. Functions like find_task_by_vpid() and basically all not well documentation syscalls are as a rule of thumb not for user space.
So we will parse /proc/<pid>/maps directly!
- program counter
- all the CPU’s registers
- process stack (containing temporary data) such as:
- routine parameters
- return addresses
- saved variables
is a region of the process’ memory that contains information about:
- open files
- current directory
- signal action
- “accounting information”
Is basically the problem we’re trying to solve. `find-match-address-partial’[fn:4] tries to fix this problem.
The real world example involves an address range of 200 Million. Just `peekdata’ over such a range, byte-wise, takes a long time
Ideas (from easy to hard implementation):
- use (ptrace …)directly as it is an order of magnitude faster than (peekdata ..) faster. Test done: using (ptrace ) directly is roughly _6.3 times faster
- Compare with raw C ptrace scans, if just peekdata is already an overhead than try to readout the whole process memory range to, disc/array with “pread” or something like that
- try multithreading it (bordeaux-thread). This would be the hardest to implement but it would be great to finally have a real reason to use it
The crucial point is to finding out `ldb’ existed and `integer-length’ Also just ldb the whole bits in one go, and not byte-wise… which was also off-by-one wrong.
This is from using TIME and slime-profile-package slime-profile-report
Btw if you’re done profiling use slime-UNprofile-all !
;; profiling data: ;; from-addr to to-addr is a range of 1-Million addresses. So only about 0.5% of ;; the real world problem size we will face with this system. ;; 5.3 Seconds is too long. (* 5.3 200) ;; (time (length (find-match-address-partial #xb7 from-addr to-addr))) ;; ;;*** with integer->bit-vector
;; seconds | gc | consed | calls | sec/call | name ;; --------------------------------------------------------------- ;; 3.444 | 0.119 | 1,502,503,360 | 2,000,002 | 0.000002 | INTEGER->BIT-VECTOR ;; 0.834 | 0.034 | 648,617,776 | 1,000,001 | 0.000001 | BIT-MASK-PADDING ;; 0.363 | 0.000 | 16,675,984 | 1,000,001 | 0.000000 | PTRACE ;; 0.320 | 0.004 | 33,613,056 | 1,000,001 | 0.000000 | ENDS-WITH-BYTES? ;; 0.213 | 0.000 | 14,320,080 | 1,000,001 | 0.000000 | PEEKDATA ;; 0.044 | 0.000 | 0 | 1,000,001 | 0.000000 | PTRACE-SUCCESSFUL? ;; 0.000 | 0.000 | 848 | 1 | 0.000000 | FIND-MATCH-ADDRESS-PARTIAL ;; --------------------------------------------------------------- ;; 5.218 | 0.157 | 2,215,731,104 | 7,000,008 | | Total ;; ;; estimated total profiling overhead: 4.82 seconds ;; overhead estimation parameters: ;; 1.2e-8s/call, 6.88e-7s total profiling, 3.22e-7s internal profiling
;; *** using `NEO-ends-with-bytes?’ !!!! [WINNER] ;; seconds | gc | consed | calls | sec/call | name ;; ------------------------------------------------------------- ;; 0.318 | 0.003 | 16,285,168 | 1,000,001 | 0.000000 | PEEKDATA ;; 0.289 | 0.004 | 15,955,120 | 1,000,001 | 0.000000 | PTRACE ;; 0.247 | 0.014 | 144,133,200 | 1,000,001 | 0.000000 | BITS-IN-NUMBER ;; 0.117 | 0.000 | 0 | 1,000,001 | 0.000000 | BYTES-IN-NUMBER ;; 0.007 | 0.000 | 0 | 1,000,001 | 0.000000 | PTRACE-SUCCESSFUL? ;; 0.000 | 0.000 | 32,768 | 1 | 0.000000 | FIND-MATCH-ADDRESS-PARTIAL ;; 0.000 | 0.000 | 15,168,704 | 1,000,001 | 0.000000 | NEO-ENDS-WITH-BYTES? ;; ------------------------------------------------------------- ;; 0.978 | 0.021 | 191,574,960 | 6,000,007 | | Total ;; ;; estimated total profiling overhead: 4.13 seconds ;; overhead estimation parameters: ;; 1.2e-8s/call, 6.88e-7s total profiling, 3.22e-7s internal profiling
;;*** with lispforum-integer->bit-vector ;; seconds | gc | consed | calls | sec/call | name ;; --------------------------------------------------------------- ;; 2.077 | 0.100 | 1,047,853,824 | 1,000,001 | 0.000002 | LISPFORUM-INTEGER->BIT-VECTOR ;; 1.039 | 0.043 | 657,010,064 | 1,000,001 | 0.000001 | INTEGER->BIT-VECTOR ;; 1.021 | 0.025 | 655,109,376 | 1,000,001 | 0.000001 | BIT-MASK-PADDING ;; 0.270 | 0.000 | 16,348,576 | 1,000,001 | 0.000000 | PTRACE ;; 0.218 | 0.000 | 13,468,528 | 1,000,001 | 0.000000 | PEEKDATA ;; 0.201 | 0.000 | 29,877,824 | 1,000,001 | 0.000000 | ENDS-WITH-BYTES? ;; 0.041 | 0.000 | 0 | 1,000,001 | 0.000000 | PTRACE-SUCCESSFUL? ;; 0.000 | 0.000 | 240 | 1 | 0.000000 | FIND-MATCH-ADDRESS-PARTIAL ;; --------------------------------------------------------------- ;; 4.867 | 0.168 | 2,419,668,432 | 7,000,008 | | Total
;;Evaluation took: just `ends-with-bytes?’ ;; 4.836 seconds of real time ;; 4.846667 seconds of total run time (4.600000 user, 0.246667 system) ;; [ Run times consist of 0.107 seconds GC time, and 4.740 seconds non-GC time. ] ;; 100.23% CPU ;; 16,478,953,784 processor cycles ;; 2,247,740,896 bytes consed
;; Evaluation took: ;; 0.309 seconds of real time ;; 0.310000 seconds of total run time (0.080000 user, 0.230000 system) ;; [ Run times consist of 0.003 seconds GC time, and 0.307 seconds non-GC time. ] ;; 100.32% CPU ;; 1,052,773,168 processor cycles ;; 31,844,240 bytes consed
;; FINAL COMPARISON
;; Evaluation took: `ends-with-bytes’ ;; 8.732 seconds of real time ;; 8.743333 seconds of total run time (6.223333 user, 2.520000 system) ;; [ Run times consist of 0.153 seconds GC time, and 8.591 seconds non-GC time. ] ;; 100.13% CPU ;; 29,758,796,854 processor cycles ;; 2,247,718,016 bytes consed
;; 17 ;; seconds | gc | consed | calls | sec/call | name ;; --------------------------------------------------------------- ;; 3.737 | 0.092 | 1,533,066,304 | 2,000,002 | 0.000002 | INTEGER->BIT-VECTOR ;; 0.982 | 0.061 | 655,445,312 | 1,000,001 | 0.000001 | BIT-MASK-PADDING ;; 0.208 | 0.000 | 16,021,792 | 1,000,001 | 0.000000 | PTRACE ;; 0.148 | 0.000 | 27,750,256 | 1,000,001 | 0.000000 | ENDS-WITH-BYTES? ;; 0.000 | 0.000 | 15,434,352 | 1 | 0.000000 | FIND-MATCH-ADDRESS-PARTIAL ;; --------------------------------------------------------------- ;; 5.075 | 0.153 | 2,247,718,016 | 5,000,006 | | Total
;; estimated total profiling overhead: 3.37 seconds ;; overhead estimation parameters: ;; 6.e-9s/call, 6.7400003e-7s total profiling, 3.3199998e-7s internal profiling
;; Evaluation took: `NEO-ends-with-bytes’ ;; using spec function `integer-length’ and `ldb’ ;; 1.862 seconds of real time ;; 1.863332 seconds of total run time (0.773333 user, 1.089999 system) ;; [ Run times consist of 0.007 seconds GC time, and 1.857 seconds non-GC time. ] ;; 100.05% CPU ;; 6,345,773,994 processor cycles ;; 31,785,104 bytes consed
;; 17 ;; seconds | gc | consed | calls | sec/call | name ;; ------------------------------------------------------------ ;; 0.291 | 0.000 | 16,452,384 | 1 | 0.290999 | FIND-MATCH-ADDRESS-PARTIAL ;; 0.235 | 0.007 | 15,332,720 | 1,000,001 | 0.000000 | PTRACE ;; 0.000 | 0.000 | 0 | 1,000,001 | 0.000000 | NEO-ENDS-WITH-BYTES? ;; ------------------------------------------------------------ ;; 0.526 | 0.007 | 31,785,104 | 2,000,003 | | Total
;; estimated total profiling overhead: 1.35 seconds ;; overhead estimation parameters: ;; 6.e-9s/call, 6.7400003e-7s total profiling, 3.3199998e-7s internal profiling
[fn:1] a soft interrupt is to be contrasted by a hardware interrupt. Both are like a function that takes highest priority and interrupts every other non-interrupt execution of instruction. A hardware interrupt is issued by hardware, a soft interrupt is issued by the program code. Examples - hardware interrupt:
- telling the CPU when I/O components are available - instead of a spin lock solution
- watchdog circuits - tell the CPU if some hardware component is not working properly these are interrupts are highly critical. Interrupts have a priority hierarchy scheme so that these kind of interrupts can get handled before others.
Examples - software interrupt:
- system calls!
- Programming language Exception system is handled through software interrupts! Such as division-by-zero
[fn:2] on i386 this is the eax register - a 32bit register
[fn:3] the size of a word is architecture and OS dependent: on x86_64 Linux it is 64bit, on 32Bit Linux OS, the word size is 32 bits
[fn:4] name might be subject to change, this function takes a value like #xabcd and searches for it over an address range that is used as index to memory with `peekdata’
“with this option the compiler and linker will generate and retain symbol information
in the executable itself”
For example loading the executable with gdb allows to associate the source-code files
with the produces assembly. (gdb) info line 12 for example, will show at what address
in the process memory those are mapped to.
Though since we can read out the instruction pointer we can pass that address to gdb
and get the assembly pointed at !
For example:
(gdb) disassemble 0x400500
On Linux:
echo 0 > /proc/sys/kernel/randomize_va_space
Now if you have a C program with a malloc()’d pointer try to print its address. By default, meaning with enabled ASLR, the address pointed to will be different on each execution of the program. But if you disable ASLR, like done above, you will see the the address will always be the same!!
Now if you run the address lookup in an endless loop, you will see that regardless of ASLR the address will stay the same during runtime, meaning that ASLR is only applied once, on process creation!
To find the header file for a function use the cmd line man
For example, in what header file is sbrk(), try
man 2 sbrkIn the SYNOPSIS section, one of the first lines, it will show #include <unistd.h>. Great!
If you look at man man you will find that the second number argument can be
used to look up different section of the manual. For example “3” refers to Library calls.
So for abs(num), man 2 abs won’t find anything, since abs() is not a system call
but man 3 abs will be more successful. This is especially useful to know when a name
can refer to two different manual entries such as man 1 kill and man 2 kill
| man <num> | manual section |
| 1 | Executable programs or shell commands |
| 2 | syscalls |
| 3 | library calls |
| non given | will look in all section in the order 1 8 3 0 2 5 4 9 6 7 |
Finally when you call man ptrace for example the header will read: PTRACE(2). The 2
refers to the manual section 2, as ptrace is indeed a syscall!
“A file containing the currently mapped memory regions and their access permissions”
from man proc. shows the virtual address space of a process and its protection
(read,write,executable). See the function in util.c called find_readable_memory to
find the regions corresponding with ‘r’ as in ‘r’eadable For example the first row of
/proc/pid/maps:
| start-end of virtual | permission | offset | dev | inode | pathname |
| address | last one is p=private | ||||
| or s=shared | |||||
| 00400000-00401000 | r-xp | 00000000 | 103:03 | 8529909 | /path/to/exe |
permission, can be changed using the mprotect syscall
When a process violates its memory access then a SIGSEGV (segmentation fault) is issued by
the kernel.
TODO: not sure if permission hinders PTRACE_POKE* in any sense, as it didn’t hinder it on
‘w’ lacking regions.. might be because I was using a sudo (root privilege sudo command)
tracer process to do so..?
dev, pathname and inode, help us find the file on the disc that this process was fed data from (I think this is the executable)
offset, is the offset into the file we wrote into memory and mapped with virtual addresses
pathname,
this one is useful for example “[stack]” might be the area containing the dynamic object,
so if you want to ptrace() a particular runtime object, this is where we
might find them
you can peek_data the instructions and the search them in the executable, they’re usually in order. But very important when you search the address you have to consider if you’re machine uses big- or little-endian. I think most machines use little-endian. Because now a peekdata might return: 200b5b058901c083 but the address is laid out in memory differently PEEKDATA: 200b 5b05 8901 c083 in memory: 83c0 0189 055b 0b20
in the hex-editor a word is in fact a halfword it seems, just look at the last entry in the PEEKDATA row is c083 and in memory it is the leading chunk, we start with the least significant portion of a datum (little endian). Because we have broken down the datum in in 4x4hex value. We get 4 halfword because 0xffff can represent up to 16⁴ or 2¹⁶ aka 16 bit (halfword).
The definition of a “word” is not very rigid, but in the literature I use, a word is 32bit, double word 64bit and consequently a halfword is 16bit. Alas in ptrace()’s man page the “word” is used and annotated as being architecture dependent.
| print <var> | print value of var |
| print &<var> | print address of var! useful for verifying with peekdata |
x86 assembly has two main syntax branches:
- Intel syntax - prevalent in the Windows world and
- AT&T syntax - prevalent in the Linux world, hence the one
gdbuses
| AT&T | Intel | comment | |
| Sigils | $<immediate value>, %register | automatic recognition | |
| parameter order | mov <src>, <destination> | mov <destination>, <src> | sigh |
| parameter size | mnemonic suffixes e.g. addl | register used imply size! | |
| rax,eax,ax,al are q,l,w,b | |||
| addresses | disp(base,index,scale) | all arithmetic expressions | |
| are written in [brackets] | |||
| movl mem_location(%ebx, %ecx,4), %eax | mov eax, [ebx + ecx*4 + | ||
| mem_location] |
Even prints the machine code assembly mapping!
objdump -d <program>
# assemble, link and execute:
# Important: the option -O0 disables all optimzations! And the assembly gets
# translated 1:1, such that
# mov rax, 0x01 ; doesn't turn into
# mov eax, 0x01 ; !!
nasm -O0 -f elf64 -o hello.o hello.asm && ld hello.o -o hello && ./hellouse gdb’s disas /r _start to disassemble at the assembly label “_start”!
gdb ./hello
(gdb) set disassembly-flavor intel
(gdb) disas /r _start| byte | 8bit |
| word | 16 bit |
| double word | 32 bit |
| quadword (qword) | 64 bits |
| double quadword | 128 bits |
We have the general purpose registers, that we know from user-regs-struct
First 8 general purpose Registers: RAX, RBX, RCX … RSP
| RAX | access full 64bits |
| EAX | access first 32bits |
| AX | first 16 bits |
| AL | of the AX access the lower byte (aka first 8 bits |
| AH | the higher, 2nd, byte of AX |
you can see this in action when you disassemble 0x400500 the bin/spam binary by replacing the x type accordingly
| int x | add $0x1 %eax | |
| long long x | add $0x1 %rax | |
| short x or char x | add $0x1 %eax | hmm the compiler doesn’t care ! |
The new registers can be accessed in a similar manner R8 through R15
| R8 | quadword (64bits) | |
| R8D | lower dword (32bits) | |
| R8H | higher dword | |
| R8W | lowest word | |
| R8B | lowest byte | this is “MASM” (Microsoft) style, “Intel style” |
| is R8L, note there is no R8H |
RIP, points at the next instruction this might be important to remember when you want immediate changes while single-stepping!
RSP, points to last item pushed onto stack, which grows toward lower addresses Used to store return values of function in high-level languages (C etc.)
RFLAGS, (formed from x86 32bit register EFLAGS, so EFLAGS value can’t be used directly..?), contains stores flags used for results of operations and for controller the CPU. Some of the most useful flags include information on:
- operation generated a carry or borrow
- last byte was even number of 1’s
- result was zero
- most significant bit of result is 1
- Overflow on signed operation
FPU - floating point unit, contains the eight registers FPR0-FPR7, status and control register. The FPR0 to FPR7 registers share space with the 64-bit MMX registers.
SIMD Architecture, instruction execute a single command on up to 8 pieces of data. Which might be troublesome when finding the instruction, or data, we wish to change.
Also some “extensions” such as SSE2 SSE3 etc. include operations for_pre-fetching memory (for performance reasons) might proof tricky. How can they be recognised? Is this immediately appellant in the instruction name?
a short overview of some command sets and which registers they operate on This is helpful, knowing that the general purpose registers are never used for this.
| Technology | Register size/type | items in parallel |
| MMX | 64 MMX | 8, 4, 2, 1 |
| SSE2/SSE3/SSSE3 | 64 MMX | 8, 4, 2, 1 |
These are not all, nor fixed. For example just “SSE” may operate on 64 MMX and 128 XXM!
Addressing Modes The usual: immediate (literal numbers),direct (content of registers), indirect(use addresses of registers).
Some Opcodes:
| Opcode | Meaning |
| CMOV | conditional move |
| JE | jump equal |
| JC | jump carry |
| LOOP | loop with ECX |
| NOP | No operation |
LOOPing, instruction is used by decrementing RCX, ECX or CX depending on range JNE if the result is 0. (testing this with a very simple for-loop did not use *CX!)
floating point opcodes, usually start with F
Although 64bit system allow addressing 64 bytes, no current CPUs can implement accessing all 16 exabytes 18.446.744.073.709.551.616, how weak is that!
The AMD architecture only uses the lower 48bits of an address, while the bits 48 through 63 must be a copy of bit 47 or the CPU raises an interrupt.
Thus the address range is in effect 0 to 00007fff’ffffffff
(!) which explains why the stack pointer of user-regs-struct is close to that limit (remember the stack pointer grows downwards!): 7fff9338e0e0 rsp of this Emacs session of the time of writing 7fffffffffff highest address
(gdb) disas /r main Shows not only the mnemonics but also the associated process data!!
returns the disassembly of the main function. The striking observation is the way it
is formatted regarding the spacing of addresses. It is important to note that different
instruction make up a smaller or bigger instruction size!! (You can tell by the <+xyz>
column and the process data after
Example from (dspm) 0x00000000004004ed <+0>: 55 push %rbp 0x00000000004004ee <+1>: 48 89 e5 mov %rsp,%rbp 0x00000000004004f1 <+4>: 89 7d fc mov %edi,-0x4(%rbp) 0x00000000004004f4 <+7>: 48 89 75 f0 mov %rsi,-0x10(%rbp) 0x00000000004004f8 <+11>: eb 0f jmp 0x400509 <main+28> 0x00000000004004fa <+13>: 8b 05 64 0b 20 00 mov 0x200b64(%rip),%eax # 0x601064 <x> 0x0000000000400500 <+19>: 83 e8 01 sub $0x1,%eax 0x0000000000400503 <+22>: 89 05 5b 0b 20 00 mov %eax,0x200b5b(%rip) # 0x601064 <x> 0x0000000000400509 <+28>: 8b 05 4d 0b 20 00 mov 0x200b4d(%rip),%eax # 0x60105c <flag> 0x000000000040050f <+34>: 3d cd ab 00 00 cmp $0xabcd,%eax 0x0000000000400514 <+39>: 74 e4 je 0x4004fa <main+13> 0x0000000000400516 <+41>: 8b 05 48 0b 20 00 mov 0x200b48(%rip),%eax # 0x601064 <x> 0x000000000040051c <+47>: 5d pop %rbp 0x000000000040051d <+48>: c3 retq
The instruction size is thus variable, from a minium of 1 byte upto 15 byte according to Intel’s Instruction set Reference.
As you might notice the first 3 MOV instruction don’t seem to have the same opcode. Well first of all there ARE different opcode for MOV instructions (see “Instruction Set Reference”), but in this case the “89” is the opcode for MOV and in the <+1> and <+7> line the leading 48 is in fact a “rex prefix” for 64 bit operands and registers (%rbp and %rsp are GPRs), that means the the MOV in the <+4> line is a 32bit MOV!
The rex prefix is a prefix for the opcode byte and indicates that 64 bit operands or GPRs or FPRs registers will be used.
Consider the output:
0x0000000000400558 <+43>: 74 e4 je 0x40053e <main+17> 0x000000000040055a <+45>: 8b 05 fc 0a 20 00 mov 0x200afc(%rip),%eax # 0x60105c <flag> 0x0000000000400560 <+51>: 89 c6 mov %eax,%esi
If we want to change the value of the variable flag, gdb’s disas output helpfully puts it there in clear text. Referring to it is a bit tricky. It is moved to EAX in line <+45>. It is referred to with the disp(base,index,scale) address notation. Since it is just disp(base) though it is simply:
0x200afc+%rip = &flag
But, then, what value is %rip? The obvious answer is wrong, it is not 0x40055a like the address of the current instruction beginning. But rather rip points to the next instruction So whenever you read assembly, that wants to use the content of %rip, you need to remember that it refers to the next instruction, after the one it is referred to.
This changes `flag’ for every subsequent instruction! As we change it directly in its memory location!
if (peekdata #x400500) is 200B5B058901C083 and the next instruction is #x400503, then the instruction is only #x400500 + 3 bytes long! One byte can be represented by two hex numbers so of the 200B5B058901C083, only 6 places count, and given endian representation, we have to read instruction in backwards “380C109850B5B002”? No, not quite! the tricky thing to know, is that little endian pertains byte order so if 1024 is represented as #x400 then in memory it will be laid out as 0 0 0 4 The first two zeros are the first byte 4 and 0 is the 2nd byte. We start with the least significant of the #400 namely “00” and the then we move to the next byte “04”
So lets go back to our example 200B5B058901C083 represented as bytes 20 0B 5B 05 89 01 C0 83 first we can make a surefire cut though! Because #x400503 is the next instruction and it is only 6 hex numbers apart, calling peekdata should return some overlapping data! (peekdata #x400503) ==> 58B00200B5B0589
lets compare the two 200B5B058901C083 58B00200B5B0589 <- overlap!
Oh! they overlap in a different way I imagined they would.. Now even more surprising is: (peekdata #x400500) ==> 200B5B058901C083 (peekdata #x400501) ==> 200B5B058901C0 (peekdata #x400502) ==> -74FFDFF4A4FA76FF ;; what the..? (peekdata #x400503) ==> 58B00200B5B0589
let’s keep that (peekdata #x400502) output in mind. It might be an issue of representing a number as twos-complement. It should be avoidable by only issuing peekdata at portions the instruction pointer rip will point to!
UPDATE: Whenever the leading byte’s most significant bit is ‘1’ the number gets interpreted as a negative number because of the two’s complement representation. This can be avoided by requesting the literal bit representation.
Moving along, only 6 bytes make up the instruction. But which 6? It should be obvious now that those must be the bytes that went missing with increasing hex-number calls so: 01 C0 83 And because we use :little-endian, it is in fact 83 C0 01 Now which instruction is this?? Using (gdb) bin/spam and (gdb) disas /r main we get: 83 C0 01 add $01, %eax
Feeding the above into another disassembler online reveals that C0 = EAX and 01 = $01 The differing order of arguments in the gdb and the one-to-one order from the online disassembler is due to the syntax style of assembly used. See the “intel syntax” and “at&t syntax” in a table above, with all the other significant differences.
<k-stz> hey asm newb here, trying to find an api/library that will let me disassemble machine code. (x86_64 littleendian). so that foo(83 c0 01) => add $01 %eax <anon0> RTFM, its only <anon1> k-stz: https://github.com/jonwil/unlinker/tree/master/udis86 [00:41] <k-stz> anon1: thank you looks good [00:42] <k-stz> Furthermore I’m attempting to read out the machine code out of a live process, using the instruction pointer. I wonder if i might run into some pitfalls where the same bytes will translate to something different depending on say context..? [00:46] <anon2> k-stz: the only context you need is the operating mode (16/32/64 bits) [00:48] <anon2> and you need to start from the instruction pointer of course <k-stz> anon: thanks
TODO: look into intel operating mode (16/32/64)
https://github.com/jonwil/unlinker/tree/master/udis86 I read that the format needs to be XML somewhere.. so its looking bad already
What is the operating mode and how can it affect how machine code will be interpreted by the CPU? Finally how to tell in which operating mode a process is, and do I have to perhaps change it?
Given the spam.c, which generates an endless loop, peekdata of a particular instructions from the instruction pointer is:
8be4740000abcd3d
From what I gather instructions are of the form: opcode 1.Operand 2.operande …
Reading the above word, byte-wise, from right to left (ascending addresses), the
opcode byte is 3d.
Now we can look it up, by using a resource such as: http://ref.x86asm.net/coder64.html#x3D
Which tells us its a CMP (compare) instructions. Whose first opearand is rAX and the 2nd operand is “imm16/32”. Which means a comparison with an immediate value.
Looking at the text-data of the small loop we get:
#x55F6B5BF765B 0 L=6 002009f3058b MOV r16/32/64 r/m16/32/64 #x55F6B5BF7661 + 6 L=3 01c083 ADD r/m16/32/64 imm8 (RAX=0xc5dec3ba -> 0xc5dec3bb) #x55F6B5BF7664 + 3 L=6 002009ea0589 MOV r/m16/32/64 r/m16/32/64 #x55F6B5BF766A + 6 L=6 002009dc058b MOV r16/32/64 r/m16/32/64 #x55F6B5BF7670 + 6 L=5 0000abcd3d CMP rAX imm16/32 #x55F6B5BF7675 + 5 L=2 2009cf058be474 JZ/JE re18
With the help of (gdb) disassembly, we get the assembly instructions. Very useful: you
can change the assembly style to intel with (gdb) set disassembly-flavor intel !
Address OpC Mne Operands 0x000000000000065b <+17>: 8b mov eax,DWORD PTR [rip+0x2009f3] # 0x201054 <x> 002009f3058b 0x0000000000000661 <+23>: 83 add eax,0x1 01c083 0x0000000000000664 <+26>: 89 mov DWORD PTR [rip+0x2009ea],eax # 0x201054 <x> 002009ea0589 0x000000000000066a <+32>: 8b mov eax,DWORD PTR [rip+0x2009dc] # 0x20104c <flag> 002009dc058b 0x0000000000000670 <+38>: 3d cmp eax,0xabcd 0000abcd3d 0x0000000000000675 <+43>: 74 je 0x65b <main+17> 2009cf058be474
is the “operand-identifier” byte It can determine what register, or if a memory address is used, in the byte from left to right the consequtive bits form the three values:
Mod: Bits 7 to 6 Reg/Opcode: Bits 5 to 3 R/M: Bits 2 to 0
In the ‘geek’ table Instruction (http://ref.x86asm.net/geek64.html#x3D) like ‘3D’ has an operand code which is understood as follows: ‘Ivds’ I=Immediate, v= ord or double word (imm16/32), ds=doubeword sign-extended
- Operand: “r/m16/32/64”, geek64: Evqp (bold face means operand gets modified by instruction.
‘vqp’or ‘16/32/64’ means word or dw depending on operand-size attribute, or qw promoted by “REX.W” in 64 bit mode.
E or ‘r/m’ means that the ModR/M byte follows the opcode, and specifies if the operand is a register or memory address.
If it is a memory address, then it is calculated from a segment register and any of the following: base register, index register, scalling factor, or a displacment
- Operand: geek64 Gvqp
The r/m field of a ModR/M byte selects a general register for exaple #b000=Ax
‘J’ means the instruction contains a relative offset to be added to rip! ‘bs’ byte sign extended, to the size of the destination operand. In this case the destination operand is probably an address so 64 bit. Such that the operand ‘e4’ translates to: 0xffffffffffffffe4 The many 0xff…s because 0xe4 has a leading 1 ==> #b11100100, and gets sign extended.
As we can see ‘x’ and ‘flag’ are addressed relative to RIP!
If we have an instruction like:
8b 05 00200b20 mov eax,DWORD PTR [rip+0x200b20]
The important thing here is which value is used? Answer, the one of the next instruction, that is the address of the first byte after the instruction 8b 05 00200b20 <– after the ‘00200b20’.
As we step through the code we can peek the next instruction with RIP, but to calculate the RIP relative adress we then need to use (+ (rip-address) (instruction-size-rip-points-to)) = relative rip address used during said instruction execution!
This is quite helpful http://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-instruction-set-reference-manual-325383.pdf
“Bindings [to C] are written in Lisp. They can be created at-will by Lisp programs. Lisp programmers can write new bindings and add them to the image (…) The only foreign library to load is the one being wrapped- the one with pure C interface; no C or other no-Lisp compilation is required.”
Because CFFI can only deal with .so files, a summary follows:
shortly put .a are static linked libraries and .so dynamic linked libraries
.a statically linked: This means when a program is linked against it, and the library changed, the program needs to be compiled again for it to use the new library. Because the linking only takes place once, statically, during compilation If the .a static library changes the program needs to be compiled again, against it for the changes to be available to the program
.so dynamically linked: only .so libraries can by dynamically linked when compiling a program. That means that once the compilation is complete and the .so library changes in the future, the changes will be immediately available to the program because the linking is dynamically taking place at runtime of the application
Because only shared object files “*.so” can by dynamically loaded into the Lisp image for binding creation
Only differs from regular compilation in the command line switches
gcc -shared -fPIC -g test.c -o bin/libtest.so
where -shared -fPIC suffices
I.e. a shared libraries (and also .a files) can be created from any source files
All we need is an .so file, and here is how it works:
- First we describe how to load a shared library into the Lisp image with
define-foreign-library
(define-foreign-library libc-definition-name
;; where to fetch the .so files on :unix systems:
(:unix (:or ;"libc-2.19"
"libc.so"
"/lib/x86_64-linux-gnu/libc-2.19"))
;; default search location - if all fails
(t (:default "/lib/x86_64-linux-gnu/libc-2.19"
"/usr/lib/libc.so")))
- Then we actually load the described library into the Lisp image
(use-foreign-library libc-definition-name)- And now we can create the bindings to the functions we care about
(defcfun ("abs" absoluto) :int (flags :int))
;; "abs" is the name of the clib function where `absoluto' refers to the name we can
;; invoke it with in the lisp code (as `abs' is already a function in Common Lisp!)
;; :int = return-valueThe binary *.so file indeed contains all the metadata necessary to refer to the functions name to the machine code therein!
The command line tool nm lets us see the symbols of an object file, the output of,
for example, nm libtest.so returns among others:
w _Jv_RegisterClasses
00000000000006e5 T main
U printf@@GLIBC_2.2.5
0000000000000630 t register_tm_clones
0000000000000711 T returnsTwo
For example “T” in the 2nd column means that the particular symbol is in the text/code section
There might be some useful info in there pertaining address offset of particular functions. (And where to find functions of interest in the process’ memory via ptrace!)
with-foreign-object is sometimes called dynamic allocation in Lisp,
because the object created (allocated) therein only has dynamic extent. Dynamic extent
in Lisp means that the object lives only in the extend of the (with-foreign-object
<body>) body. Much like the let bindings of special variables
foreign-alloc on the other hand is what C calls “dynamic” allocation. The object
get allocated to the heap and stay there unless freed by foreign-free !!
In C these kind of objects are created with malloc() and friends.
We should always keep in mind that C imposes the semantic on all data that it is really just all arrays. Pointers are represented as arrays as well.
In Lisp we will dereference pointers with mem-aref and mem-ref !! Just like
AREF those are SETFable places!!
“To decide which one to use, consider whether you would use the array index operator [n] or the pointer dereference * in C; use mem-aref for array indexing and mem-ref for pointer dereferencing.”
C code:
int x = 77;
printf("%d\n", x); // => 77
passByReference(&x); // sets x = 2
printf("%d\n", x); // => 2Common Lisp translation (but with just Lisp’s dynamic allocation):
(with-foreign-pointer (x-ptr 4) ;; 4 = size of int
(setf (mem-ref x-ptr :int) 77) ;; x = 77
(print (mem-ref x-ptr :int))
(passbyreference x-ptr)
(print (mem-ref x-ptr :int)))Now because ptrace() is a system call it requires root privilege. To successfully call it in lisp requires for lisp to be run as root. There is no other way, you need to start the lisp program with root privilege. For securities sake you can use another system call to turn off root privilege at runtime and turn it back on again. *This only works when the process was started as root to begin with!!
The system call to toggle root privilege is, seteuid()
;; This only works if you run the lisp program as root to begin with!
;; turn on root privilege
(sb-posix:seteuid 0)
;; do your thang
(ptrace-or-another-root-privileged-call)
;; become root again
(sb-posix:seteuid 1000)Way of laying out a datum that needs more than one byte to be represented. Consider the C integer 33, which needs 32 bits or 4 bytes to be represented
Big Endian example:
0000 0000 0010 0001 1st 2nd 3rd 4th byte (in ascending address order, from left to right)
In what order can we write these 4 bytes in memory? Well there are two schemes the above is big-endian because the most significant byte (big) is the first and the least significant the last.
With little endian we put the least significant byte first, so the above integer would be represented like so
0001 0010 0000 0000 1st 2nd 3rd 4th byte
each byte has an own address and here 0001 has the first(smaller) address than 0010.
(peekdata hex-value) => 0x200B17058901C083 in what way is the above laid out in memory, out of the following:
(1) 200B 1705 8901 C083 (2) 20 0b 17 05 89 01 c0 83 (3) 83 c0 01 89 05 17 0b 20
If you chose 3 and thought that 1 is just silly than you’re goddamn right. Remember that two hex make a byte #xff => 255 => #b11111111
the following has been only tested for bin/spam You can actually find the instructions of bin/spam that you fetched with (peekdata…) if you look at bin/spam with a hex-editor. If the peeked address is #400500 then you will find the byte order in question at byte 500 and the following!
From experience so far the code block is usually the first entry in /proc/pid/maps
cl-ptrace can be used to hack games just like a cheat engine. There are some methods that can be used. In the following I will list what works, or what I want to try next.
Just like scanning a savefile for the “player money” value, we can simply scan the memory of the process by doing (find-value-address <current-money>) we can filter all addresses that might store the money value. Then we can resume the game, get some money, or lose it, and search again but this time over the sieved out values.
Now if those values match we have a strong inkling that those addresses might store our money value and we can either do more tests, narrowing down the addresses or go ahead and try to inject the money we want with (pokedata ..) and see if it changes ingame.
- The Address-space of modern Computer is huge (expt 2 64), simply looping over such, huge space, without executing any execution, takes unfeasible amount of time. (multiple years)
We can narrow down the address range because if we take memory segments into consideration. For example most runtime data is allocated on the heap, who’s memory range can be read out from the /proc/<pid>/maps file. (some, because mmap will write bigger allocation, by default, to the memory mapping segment) This already reduced the problem size by plenty orders of magnitude.
Searching for a player health value like “300 Hp” we need to consider that if the machine uses :little-endian, then the bytes will be laid out in reverse order: 300 = #x12c, hence memory: 0c 12
That’s the smaller issue, the bigger is that the value might be represented implicitly through other variables. Like instead of the programmer writing int health = 300; The code might be, String health = “300”; and on calling the string is parsed and added to. This might look strange, but it is a actual method used by programmers to hack-proof there code!
Now we also might consider the implementation of the game itself, where the player,
and his stats are static variables, that are only declared on execution. Then
The values would reside in the BSS-Segment, which can’t just be readout from
/proc/<pid>/maps file directly. We have to guess which memory range it might contain.
Also it could be in the data segment, if the Hero starts out with static int health =
10; health.
This is a bit involved, we will save the entire address region values. Then continue the game and without triggering a money change. Then stop the game again and filter out all addresses that have changed. We can repeat this, and eventually introduce some money changes. We could then check if the the address values now changed by the proper magnitude: If the player loses 10 health points, we might scan if the absolute of the (- old-value new-value) is 10. This might still fail, as programmers might store the value as an binary inverse and together with a factor. The solution might then just be to either keep on trail and error (perhaps with ASLR enabled etc) The idea is if we find a needle in the haystack, neighbour values might be part of the data structure representing the player stats, money, items etc.
Finally, I haven’t tried if this is feasible. We could singlestep through a portion
of the program where a certain change takes place to a value we care about.
For example the player is 2 pixels apart from falling on spikes. We now know
that somewhere along the way there is going to be an instruction that reads
out the health value from the process memory and puts it in a register. To
perform the arithmetic. We can then use the memory-address used to find the
address in question!
This would require to get at the disassembly easily, either by disassembling
on the fly, or using the instruction pointer to find the functions referred to
in the executable binary! Perhaps some nm .. function value is being referred
to
Some programs might try to guard against code injection by various methods, of which some may be too involved for the cracker/hacker to bother trying to bypass it.
For example, the program might checksum its core, or parts of it, with a separate thread, every now and then, and guard against code injection this way (for example crashing the program on a registered failed checksum).
The Hacker might find the checksum routine and feed it some bogus values to satisfy it, or perhaps directly crop out the checksum routine itself. To guard against that the checksum system might then fetch its values from some cloud server, that raises the effort to hack the program.
mmap() allows to map a file (or anything represented by file descriptor) to process
memory. Now you can access the contents of the file using pointer arithmetic, without
the open(), close() shenanigans. Even cooler: it is even possible to make it so that
writing to this memory region changes the mapped file!
#include <sys/mman.h>
void *mmap(void *addr, size_t len, int prot,
int flags, int fildes, off_t off);| Parameter | Description | SBCL |
| addr | Address where to map the file into. If ’0’ is | address 0 is |
| provided, then the OS decides it for you. | NIL or | |
| But you can’t choose it entirely freely either: | (make-pointer 0) | |
| it needs to be a multiple of the | UPDATE: osicat-posix:MMAP doesn’t | |
| virtual memory page size | accept NIL use a null ptr instead | |
| len | length of the file. Again if the length modulo | use (FILE-LENGTH stream) |
| virtual-memory-page-size then it gets rounded | where stream is from | |
| up filling the extra bytes with 0s. | (WITH-OPEN-FILE (stream <path>..)) | |
| prot | this is the ‘rwxp’ part of /proc/<pid>/maps. | sb-posix:PROT-READ and |
| Enums: PROT_READ, .._WRITE, | -WRITE, -EXEC respectively | |
| flags | this sets the p and s in ‘rwxp’. Where ‘p’ | sb-posix:MAP-PRIVATE |
| is MAP_PRIVATE and ‘s’ is MAP_SHARED. Private: | and sb-posix:MAP-SHARED | |
| get a copy of memory region. Shared: changing | ||
| the region, changes it for other processes. TODO: | ||
| does it change the underlying (meaning: the file | ||
| represented by the fd represented) file? | ||
| fildes | the FILe DEScriptor, the FD | (sb-impl::fd-stream-fd stream) |
| offset | offset in the file represented by fildes, BUT: | off_t is signed integer |
| must also be multiple of virtmem. page size! |
to get at the FD, file descriptor, use
(with-open-file (stream "/path/to/file" :direction :IO :if-exists :append)
;; to get the file-descriptor, finally do:
(sb-impl::fd-stream-fd stream)
..)we set :direction :IO because in case we want to use the shared flag!
mmap() returns the address(pointer) to the beginning of the file otherwise -1 setting errno accordingly
If the shared flag is set in the for the mapped memory region with a permission string like: rw-s, then the following be behavior takes place.
- changing the file, also changes what the SAP changes points to
- writing to the sap address changes the file
It helps to imagine that the SHARED flag used in mmap-region, would helps to implementing a text-editor.
problem size: 2.187.264 on laptop
looping and reading out address, no output:
| READ-PROC-MEM-BYTE | 103 seconds |
| (mem-ref readv-c-array) | 0.041 seconds! |
| (ldb (byte 8 0) (peekdata ..)) | 2.216 seconds |
==> don’t use read-proc-mem-byte, or implement it in terms of (ldb (byte 8 0) (peekdata ..))
==> the syscall process_vm_readv trumps all