JIT Compilation Threshold

source

The JIT compilation threshold determines when Ruby’s JIT compiler transforms interpreted YARV bytecode into native machine code. This threshold represents a critical tradeoff: compile too early and waste effort on cold code; compile too late and miss optimization opportunities.

The Fundamental Tradeoff

By default, Ruby compiles source code into instruction sequences (YARV bytecode) that are executed by the Ruby VM. This interpretation happens for all code initially - Ruby parses source code, generates the instruction sequences and VM executes them directly.

At some point, JIT compilation occurs when code execution becomes “hot” (executed frequently to warrant optimization). How is this determined? JIT compilers must balance:

• Compilation overhead: Time and memory to generate native code
• Execution benefit: Speedup from native vs interpreted execution
• Memory pressure: Compiled code memory and GC impact
• Code stability: Risk of de-optimization invalidating effort

The threshold embodies this balance - it’s the point where expected benefits outweigh costs.

MJIT (Ruby 2.6)

Triggered after a method gets called 10,000 times.

YJIT: Two-Phase Compilation (Ruby 3.1+)

YJIT uses a sophisticated two-phase approach that separates profiling from compilation. This enables better speculative optimization by gathering data before making optimization decisions.

Phase 1: Profiling Threshold (25 calls)

After 25 calls, YJIT begins observing the method:

• Tracks actual runtime types
• Records which code paths execute
• Collects data for type assumptions
• Does NOT generate machine code yet

Phase 2: Compilation Threshold (30 calls)

After 30 total calls, YJIT compiles to native code:

• Analyzes profiled type information
• Generates optimized machine code
• Inserts type guards to verify assumptions
• Updates ISEQ jit_entry pointer

This adaptive compilation strategy improves on MJIT by compiling much earlier (30 vs 10,000 calls) while still making informed optimization decisions.

Dynamic Thresholds:

YJIT switches between small and large thresholds based on application size:

• Small Call Threshold: 30 calls (default)
• Large Call Threshold: 120 calls (apps with >40,000 ISEQs)
• Cold Threshold: 200,000 calls (prevents compiling rarely-called methods)

Applications with more than 40,000 ISEQs are considered “large” and use the 120-call threshold yjit/src/options.rs. This prevents wasting compilation effort in massive codebases.

See YJIT execution mechanics for detailed exploration of how this compilation process works.

ZJIT compilation thresholds upstreamed

In contrast, ZJIT focuses on profiling before compilation. zjit/src/options.rs.

The profile threshold determines when to start profiling instruction sequences, while the call threshold determines when to compile. How it’s done: The profiling process replaces regular ISEQ with special zjit versions that collect runtime information. Once these profiled ISEQ reach call thresholds(2 calls by default), ZJIT compiles the ISEQ and then performs cleanup/disables the profiling.

Threshold Design Patterns

The evolution of JIT thresholds reveals important patterns:

1. Profile Before Compiling

YJIT’s two-phase approach (profile at 25, compile at 30) is more sophisticated than MJIT’s single threshold (compile at 10,000). Gathering data before optimization leads to better code generation.

2. Adaptive Thresholds

YJIT adjusts thresholds based on application size. Large codebases use higher thresholds to avoid compilation explosion. This context-aware compilation improves on fixed thresholds.

3. Multiple Tiers

Modern JIT compilers use tiered compilation:

• Tier 0: Interpret (no compilation)
• Tier 1: Quick compile with simple optimizations
• Tier 2: Optimizing compile with profiling data

YJIT’s phases hint at this direction - future versions may add more tiers.

4. Compile What Matters

The cold threshold (200,000 calls) recognizes that not all hot code is worth compiling. Some methods are called often but execute so quickly that compilation overhead exceeds benefits.

Threshold and Call Counter Mechanics

The threshold works through the ISEQ’s call counter:

// Simplified execution logic
if (iseq->jit_entry != NULL) {
    // Already compiled - execute native code
    result = iseq->jit_entry(args);
} else {
    // Interpret bytecode
    result = vm_exec_core(iseq);
 
    // Increment counter and check threshold
    if (++iseq->call_counter == YJIT_PROFILE_THRESHOLD) {
        yjit_start_profiling(iseq);
    } else if (iseq->call_counter == YJIT_COMPILE_THRESHOLD) {
        yjit_compile(iseq);
    }
}

Each ISEQ tracks its own calls independently - a frequently-called small method will JIT compile even if the overall program has millions of ISEQs.

Impact on Performance

Understanding thresholds clarifies performance characteristics:

Warmup time: Lower thresholds mean faster warmup - YJIT reaches peak performance after ~30 calls vs MJIT’s 10,000.

Memory usage: Lower thresholds compile more code, increasing memory pressure. The adaptive threshold (120 for large apps) mitigates this.

De-optimization cost: If de-optimization occurs after compilation, the effort is wasted. Higher thresholds reduce this risk by observing more behavior first.

Peak performance: Lower thresholds don’t guarantee better peak performance - they just reach it faster. Quality of compiled code matters more than compilation speed.

Tuning Thresholds

Ruby exposes threshold tuning for advanced use cases:

# YJIT with custom thresholds (Ruby 3.3+)
RUBY_YJIT_CALL_THRESHOLD=50 ruby script.rb
 
# Disable JIT entirely
ruby --yjit=false script.rb

When to tune:

• Lower threshold: Short-running scripts that need quick warmup
• Higher threshold: Long-running servers where compilation overhead matters
• Disable JIT: Debugging, profiling, or when de-optimization is frequent

Most applications should use defaults - the Ruby team has optimized thresholds based on extensive benchmarking.

The threshold is a deceptively simple concept - just a number determining when to compile. But it embodies deep tradeoffs about warmup time, memory usage, compilation quality, and peak performance. Understanding thresholds reveals why modern JIT compilers like YJIT are adaptive systems that balance multiple competing goals.