Gradual Notes

Recent Writing

Recent Notes

Side Exits in JIT Compilation

6 min read

Side exits are pre-compiled escape routes from JIT-compiled code that restore interpreter state when runtime assumptions break. They are the mechanism that makes speculative optimization safe - allowing aggressive optimization while maintaining correctness.

What is a Side Exit?

A side exit is a code path inserted by the JIT compiler that:

  1. Detects assumption violations (via type guards or other checks)
  2. Jumps to a pre-defined exit point (the “side” exit, off the fast path)
  3. Restores interpreter state (stack pointer, instruction pointer, registers)
  4. Returns control to the interpreter to continue execution correctly
Execution Flow with Side Exit:
┌─────────────────┐
│ JIT compiled    │
│ native code     │
└────────┬────────┘
         │
         ▼
    ┌────────────┐
    │ Type guard │
    │ or check   │
    └─────┬──────┘
          │
     ┌────┴────┐
     │         │
   Pass?    Fail?
     │         │
     ▼         ▼
 ┌───────┐  ┌─────────────┐
 │Fast   │  │ SIDE EXIT:  │
 │path   │  │ - Restore   │
 │       │  │ - Deopt     │
 └───────┘  │ - Interpret │
            └─────────────┘

The name “side exit” reflects that it’s an alternative path to the side of the main optimized execution path.

Why Side Exits Exist

YJIT execution mechanics relies on making assumptions about types, values, and program behavior:

def calculate(x)
  x + 10
end
 
# YJIT observes: x is always Integer
# Generates optimized code assuming Integer
# But Ruby allows any type!

Without side exits, the JIT would have two bad choices:

  1. Conservative: Never assume types → slow code
  2. Unsafe: Always assume types → wrong results

Side exits provide a third option:

  1. Speculative: Assume types, but verify with guards, exit to interpreter if wrong → fast AND correct

How Side Exits Work

The Assembly-Level View

; Generated JIT code for: x + 10
 
; GUARD: Check if x is Integer (Fixnum)
test    rdi, 0x1          ; Test least significant bit
jz      side_exit_1       ; If zero, not a Fixnum - take side exit
 
; FAST PATH: Optimized integer addition
lea     rax, [rdi + 20]   ; Add 10 (encoded as 20 for Fixnum)
ret
 
; SIDE EXIT: Restore and interpret
side_exit_1:
    push    rbp
    mov     rbp, rsp
    ; Restore VM stack pointer
    mov     [r13 + vm_sp_offset], r14
    ; Restore instruction pointer to current bytecode
    mov     [r13 + vm_pc_offset], current_pc
    ; Clear jit_entry to prevent re-entry
    mov     qword [iseq + jit_entry_offset], 0
    ; Jump back to interpreter
    jmp     vm_exec_core

The side exit code:

  • Preserves the program’s logical state
  • Makes it appear as if interpretation never stopped
  • Clears the JIT entry to trigger de-optimization

The State Restoration Process

When a side exit triggers, the VM must restore:

  1. Stack pointer (SP): Where the Ruby stack currently is
  2. Program counter (PC): Which bytecode instruction to execute next
  3. Environment pointer (EP): For variable lookups (see frame parent-child relationships)
  4. Registers: Any temporary values the interpreter expects

This restoration ensures the interpreter can continue exactly where the JIT left off.

Side Exits vs De-optimization

Side exits and de-optimization are related but distinct:

Side Exit:

  • The mechanism - the actual code path and state restoration
  • Happens during execution when a guard fails
  • A micro-level operation (nanoseconds)

De-optimization:

  • The consequence - invalidating the JIT-compiled code
  • Happens after a side exit
  • A macro-level operation (clearing jit_entry, resetting counters)
Side Exit → De-optimization → Re-profiling → Potential Re-compilation

Guard fails  Clear jit_entry  Gather new     Maybe compile
             pointer          type data      with new info

Every side exit triggers de-optimization, but not every de-optimization requires a side exit (e.g., TracePoint activation proactively invalidates code).

Types of Side Exits

Type Guard Side Exits

The most common - guard type assumptions:

def process(x)
  x * 2  # Assumes Integer
end
 
# Side exit if x is Float, String, etc.

Range Check Side Exits

Guard value ranges:

def safe_index(arr, i)
  arr[i]  # Assumes i is within bounds
end
 
# Side exit if i < 0 or i >= arr.length

Null/Nil Check Side Exits

Guard against nil values:

def get_name(user)
  user.name  # Assumes user is not nil
end
 
# Side exit if user is nil

Class Identity Side Exits

Guard object class:

def area(shape)
  shape.width * shape.height  # Assumes Rectangle class
end
 
# Side exit if shape is Circle, Triangle, etc.

Side Exit Performance Implications

When Side Exits Are Cheap

If a side exit never fires, it’s essentially free:

  • Guard check: 1-2 CPU cycles (branch prediction works)
  • Fast path continues unimpeded
  • Side exit code isn’t even loaded into CPU cache

This is the happy path - stable types, predictable behavior.

When Side Exits Are Expensive

If a side exit fires frequently:

  • Guard check fails: ~5-10 cycles (branch misprediction penalty)
  • State restoration: ~20-50 cycles
  • Return to interpreter: hundreds of cycles per operation
  • De-optimization overhead: microseconds
  • Potential re-compilation: milliseconds

This is the pathological case - polymorphic code, unstable types.

The 95% Rule

Side exits are profitable when guards pass >95% of the time:

If guard passes 99% of time:
  100 calls = 99 fast (optimized) + 1 slow (side exit)
  Net win: ~10-100x speedup

If guard passes 50% of time:
  100 calls = 50 fast + 50 slow
  Net loss: Compilation overhead not worth it

Side Exits in Other VMs

Side exits are a universal pattern in JIT compilation:

JavaScript V8:

  • “Deoptimization trampolines” (their term for side exits)
  • Guards on object shapes (“hidden classes”)
  • Exits to baseline or interpreter tier

Java HotSpot:

  • “Uncommon traps” (their term)
  • Guards on class hierarchy assumptions
  • Exits from optimized to interpreted code

PyPy (Python):

  • “Guards” with explicit exit points
  • Trace-based compilation with guard failures
  • Exits abort traces and return to interpreter

The terminology varies, but the concept is identical: speculate, guard, exit if wrong.

Monitoring Side Exits

# Enable YJIT statistics
ruby --yjit --yjit-stats script.rb
 
# Look for:
# - Side exit count (total exits taken)
# - Exit reasons (type mismatch, nil check, etc.)
# - Hot exits (which exits fire most often)

Red flags:

  • Same side exit firing repeatedly: Wrong type assumption
  • High exit rate (>5%): Polymorphic code
  • Exits in hot loops: Performance killer

Optimizing for Fewer Side Exits

Type Stability

# Bad: Polymorphic input → frequent side exits
def calculate(value)
  value * 2
end
 
calculate(5)      # Integer
calculate(3.14)   # Float - SIDE EXIT
calculate(5)      # Integer - SIDE EXIT (different from Float)
 
# Good: Consistent types → no side exits
def calculate_int(n)
  n * 2  # Always Integer
end
 
def calculate_float(f)
  f * 2  # Always Float
end

Guard Hoisting

Move guards out of loops:

# Bad: Guard fires every iteration
def sum_doubled(arr)
  total = 0
  arr.each do |x|
    total += x * 2  # Type guard on x every iteration
  end
end
 
# Better: Guard once before loop
def sum_doubled(arr)
  # YJIT can hoist guard: check arr contains Integers
  # Then optimize entire loop without per-iteration guards
  arr.sum { |x| x * 2 }
end

Side exits make JIT compilation safe in dynamic languages. They’re the escape hatches that prevent wrong execution while allowing aggressive optimization. Understanding side exits reveals why type-stable code runs faster: fewer side exits mean more time in the fast path.

Recent Writing

Recent Notes

Graph View