Type Guards in JIT Compilation

Type guards are runtime checks inserted into JIT-compiled code that verify type assumptions made during compilation. They enable speculative optimization by allowing the compiler to generate fast specialized code while maintaining correctness when assumptions break.

The Fundamental Problem

Dynamic languages like Ruby allow types to change at runtime:

def double(x)
  x * 2
end
 
double(5)      # Integer
double(3.14)   # Float
double("hi")   # String (via String#*)

The JIT compiler wants to generate optimal native code, but optimal code depends on knowing types:

; If x is Integer (Fixnum):
lea rax, [rdi + rdi]  ; Fast: bit shift for *2
 
; If x is Float:
mulsd xmm0, xmm1      ; Floating point multiply
 
; If x is String:
call rb_str_times     ; Method call to C function

Type guards solve this: assume the common type, verify the assumption, fall back if wrong.

How Type Guards Work

A type guard checks a value’s type before executing type-specific optimized code:

def calculate(x)
  x + 10
end
 
# After profiling, YJIT assumes x is Integer
# Generated native code:

; Type guard - verify x is Integer
test    rdi, 0x1          ; Check if Fixnum (LSB = 1 in Ruby)
jz      side_exit         ; If not, take side exit
 
; Fast path - native integer addition
lea     rax, [rdi + 20]   ; x + 10 (Fixnum encoding: value << 1 | 1)
ret
 
side_exit:
; Side exit - restore interpreter state and fall back
mov     r12, rdi          ; Save argument
; Restore stack pointer, instruction pointer
call    vm_exec_core      ; Return to interpreter
ret

The type guard is just a few instructions:

Test - Check type tag bits
Branch - Take side exit if wrong type
Fast path - Execute optimized code if right type

Side exits are pre-defined exit points that restore interpreter state (stack, instruction pointer) and return control to the VM when assumptions break.

Type Representation in Ruby

Ruby uses tagged pointers to encode type information efficiently:

Integer (Fixnum) encoding:
┌────────────────────────────────┬─┐
│ Value (shifted left 1 bit)     │1│  ← LSB = 1 indicates Fixnum
└────────────────────────────────┴─┘

Symbol encoding:
┌────────────────────────────┬───┐
│ Symbol ID                  │110│  ← Last 3 bits = 0b110
└────────────────────────────┴───┘

Object pointer (aligned to 8 bytes):
┌────────────────────────────────┬─┐
│ Memory address                 │0│  ← LSB = 0 indicates pointer
└────────────────────────────────┴─┘
  └─ Points to object header with class/type info

Type guards exploit this encoding for fast checks:

; Check if Fixnum
test    rdi, 0x1          ; Test LSB
jz      not_fixnum        ; Zero = not Fixnum
 
; Check if Symbol
and     r11, rdi, 0x7     ; Mask last 3 bits
cmp     r11, 0x6          ; Compare to 0b110
jne     not_symbol
 
; Check class of heap object
mov     r11, [rdi]        ; Load object header
cmp     r11, String_class ; Compare class pointer
jne     not_string

These checks are extremely fast - typically 2-3 CPU cycles. Much faster than Ruby-level type checking (x.is_a?(Integer)).

Guard Placement Strategies

YJIT places guards strategically to minimize overhead while ensuring correctness:

Entry Guards

Check types at method entry before executing any optimized code:

def process(x, y)
  x + y
end
 
# Guard at entry:
# 1. Verify x is Integer
# 2. Verify y is Integer
# 3. Execute fast integer addition

Advantage: Single check covers entire method Disadvantage: Might guard types never actually used

Operation Guards

Check types immediately before type-specific operations:

def complex_method(x, y, z)
  a = some_operation(x)
  b = another_operation(y)
  a + b  # Guard here: verify a and b are Integers
end
 
# Guards only when needed:
# - Don't guard x, y (not directly used in typed operation)
# - Guard a, b before addition

Advantage: Only guard when necessary Disadvantage: Multiple guards can accumulate overhead

Conditional Guards

Guard only on specific code paths:

def conditional_process(x, flag)
  if flag
    x * 2  # Guard x here (only if flag is true)
  else
    "skipped"
  end
end

YJIT analyzes control flow to place guards on paths that need them.

Multi-Type Guards: Handling Polymorphism

When code sees multiple types, YJIT can generate polymorphic inline caches with multiple guard clauses:

def stringify(value)
  value.to_s
end
 
# Sees Integer and String during profiling
# Generated code:

; First guard: Integer?
test    rdi, 0x1
jnz     handle_integer
 
; Second guard: String?
mov     r11, [rdi]
cmp     r11, String_class
je      handle_string
 
; Neither: deoptimize
jmp     deoptimize
 
handle_integer:
call    rb_int_to_s
ret
 
handle_string:
mov     rax, rdi          ; String to_s returns self
ret

This creates a guard chain - check each known type, fall back if none match.

Diminishing returns: Each guard adds overhead. After 3-4 types, the guard chain becomes slower than just calling the method normally.

Guard Failure: Side Exits and De-optimization

When a guard fails, the code takes a side exit to de-optimization:

Guard Failure Flow:
┌─────────────────┐
│ Execute guard   │
└────────┬────────┘
         │
    ┌────┴────┐
    │         │
  Pass?    Fail?
    │         │
    ▼         ▼
┌────────┐  ┌──────────────────┐
│Continue│  │ Side Exit:       │
│fast    │  │ 1. Jump to exit  │
│path    │  │ 2. Restore state │
└────────┘  │ 3. Clear jit ptr │
            │ 4. Interpret     │
            │ 5. Maybe         │
            │    re-compile    │
            └──────────────────┘

A side exit is a pre-compiled escape route from JIT code. When a guard fails:

Jump to the side exit code
Restore interpreter state (stack pointer, instruction pointer, etc.)
Clear the jit_entry pointer in the instruction sequence
Return control to the interpreter

Side exits preserve correctness - the interpreter handles the unexpected type correctly, even if slowly.

Guard Strength vs Performance Tradeoff

Stronger guards = more safety, but more overhead:

Weak Guards (Fast but Limited)

; Just check Fixnum bit
test rdi, 0x1

Cost: 1-2 cycles
Coverage: Only distinguishes Fixnum from others
Risk: Treats all non-Fixnums the same

Medium Guards (Balanced)

; Check class of heap object
mov  r11, [rdi]
cmp  r11, expected_class

Cost: 3-5 cycles
Coverage: Identifies specific class
Risk: Misses subclasses

Strong Guards (Thorough but Slow)

; Check class and flags
mov  r11, [rdi]
cmp  r11, expected_class
jne  fail
test [r11+flags_offset], expected_flags
jne  fail

Cost: 6-10 cycles
Coverage: Class + frozen state + singleton methods
Risk: Minimal

YJIT chooses guard strength based on profiled behavior and optimization goals.

Eliminating Guards: The Ultimate Optimization

The fastest guard is no guard at all. YJIT can eliminate guards when types are provably stable:

Constant Folding

def calculate
  x = 42          # Compiler knows x is Integer
  y = 10          # Compiler knows y is Integer
  x + y           # No guard needed!
end

The compiler tracks concrete values - no runtime check needed.

Type Flow Analysis

def process(input)
  x = input.to_i  # .to_i always returns Integer
  x * 2           # No guard needed - x is Integer
end

Even with dynamic input, type flow analysis proves x must be Integer.

Closed-World Optimization

class FinalClass
  def calculate(x)
    x + 1
  end
end
 
# If FinalClass is never subclassed:
# - x.class is provably FinalClass
# - No guard needed for self

When classes are sealed or final, more optimizations become safe.

Monitoring Guard Performance

Guards add overhead. Too many guards negate JIT benefits:

# Enable YJIT stats
ruby --yjit --yjit-stats script.rb
 
# Output includes:
# - Guard hit rate (how often guards pass)
# - Guard failure locations (where de-optimization occurs)
# - Polymorphism degree (how many types per guard)

Red flags:

Guard hit rate < 95%: Frequent failures, unstable types
High polymorphism (4+ types): Guard chains too long
Same guard failing repeatedly: Wrong type assumption

Writing Guard-Friendly Code

To maximize JIT performance, write code that needs fewer guards:

Type Stability

# Bad: Polymorphic return type
def get_value(flag)
  flag ? 42 : "forty-two"
end
 
# Good: Consistent return type
def get_value(flag)
  flag ? 42 : -1
end

Avoid Type Coercion in Loops

# Bad: Guard fires every iteration
def sum_array(arr)
  total = 0
  arr.each do |x|
    total += x.to_i  # .to_i might return different types!
  end
end
 
# Good: Type-stable iteration
def sum_integers(arr)
  total = 0
  arr.each do |x|
    total += x  # Assume all elements are Integer
  end
end

Separate Paths for Different Types

# Bad: One method, multiple types
def format(value)
  value.is_a?(Integer) ? value.to_s : value.upcase
end
 
# Good: Separate methods
def format_integer(n)
  n.to_s
end
 
def format_string(s)
  s.upcase
end

Comparison to Other Dynamic Languages

Type guards are common across JIT-compiled dynamic languages:

JavaScript V8:

Uses “hidden classes” for object shape guards
Polymorphic inline caches with guard chains
Speculative type tracking

Python (PyPy):

Guards on type and attribute existence
Trace-based compilation with guard chaining
Invalidation when classes mutate

Java (HotSpot):

Guards on class hierarchy (instanceof checks)
Null checks as guards
Aggressive inlining with guards

The pattern is universal: assume, guard, optimize, fall back.

Type guards are the linchpin of modern JIT compilation - they enable aggressive optimization while maintaining the correctness guarantees of dynamic languages. Understanding guards reveals why type-stable code runs faster: fewer guards mean more direct execution, less overhead, and better CPU pipeline utilization.

The Guard Paradox

Type guards embody a fundamental paradox in dynamic language optimization:

Dynamic flexibility requires static assumptions.

To run fast, dynamic languages must temporarily pretend to be static. Guards make this pretense safe - they let the compiler optimize as if types never change, while quietly checking if that assumption holds.

The art of JIT compilation is knowing when to trust types enough to optimize, but verify enough to stay correct.

Recent Writing

Recent Notes