Sidekiq Middleware System

Sidekiq uses middleware chains inspired by Rack to implement cross-cutting concerns like metrics, logging, and error tracking without modifying job code. Client middleware wraps job pushing; server middleware wraps job execution. This architectural pattern provides elegant extensibility while maintaining the single responsibility principle.

Architecture Pattern

Middleware follows the nested function pattern where each middleware wraps the next in the chain:

class TimingMiddleware
  def call(worker, job, queue)
    start = Time.now
    yield  # Continue to next middleware or job execution
    duration = Time.now - start
    logger.info "#{job['class']} took #{duration}s"
  end
end

The yield keyword creates the nesting—control flows down through the middleware stack to job execution, then back up as each middleware completes. This enables before/after behavior naturally:

Request → M1 before → M2 before → Job → M2 after → M1 after → Response

Client vs Server Middleware

Client middleware runs in the application process when enqueueing jobs. It can:

• Validate or transform arguments
• Add metadata to the job hash
• Prevent job creation entirely by not yielding
• Log job creation for audit trails

class ArgumentSanitizer
  def call(worker_class, job, queue, redis_pool)
    job['args'].map! { |arg| sanitize(arg) }
    yield
  end
end

Server middleware runs in the Sidekiq server process when executing jobs. It can:

• Measure job duration
• Handle exceptions and retry logic
• Manage resources (database connections, locks)
• Copy thread-local state

class TransactionMiddleware
  def call(worker, job, queue)
    ActiveRecord::Base.transaction do
      yield
    end
  end
end

Configuration

Middleware is configured globally and applied to all jobs:

Sidekiq.configure_client do |config|
  config.client_middleware do |chain|
    chain.add ArgumentSanitizer
  end
end
 
Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add TransactionMiddleware
    chain.add TimingMiddleware
  end
end

Order matters—middleware executes in the order added. Early middleware can prevent later middleware from running by not yielding.

Built-In Middleware

Sidekiq includes several critical middleware out of the box:

JobLogger: Logs job start and completion with timing information. Structured logging includes job class, JID, queue, and arguments.

RetryJobs: Implements exponential backoff for failed jobs. Catches exceptions, calculates next retry time, and updates the retry count in Redis.

CurrentAttributes: Copies Rails CurrentAttributes from enqueuing thread to executing thread. This preserves request context like current user or tenant ID across the async boundary.

# In controller
Current.user = @user
MyJob.perform_async  # Enqueues with Current.user preserved
 
# In job execution
class MyJob
  def perform
    Current.user  # Same user object available!
  end
end

Exception Handling

Middleware can customize exception handling without modifying the retry logic:

class SentryMiddleware
  def call(worker, job, queue)
    yield
  rescue => exception
    Sentry.capture_exception(exception, extra: job)
    raise  # Re-raise so retry logic still works
  end
end

By re-raising the exception, we send to Sentry while letting Sidekiq’s retry middleware handle scheduling. This separation of concerns keeps error reporting independent from retry logic.

Resource Management

Middleware excels at resource acquisition and cleanup patterns:

class ConnectionPoolMiddleware
  def call(worker, job, queue)
    external_api_pool.with do |client|
      Thread.current[:api_client] = client
      yield
    ensure
      Thread.current[:api_client] = nil
    end
  end
end

The middleware ensures the resource is available during job execution and cleaned up afterward, even if exceptions occur. This pattern eliminates boilerplate in every job class.

Performance Considerations

Each middleware adds overhead—a function call and potential memory allocations. For high-throughput systems, minimize middleware or optimize hot paths:

class ConditionalMiddleware
  def call(worker, job, queue)
    if job['expensive_feature']
      do_expensive_thing
    end
    yield
  end
end

Avoid expensive operations for every job. Conditionally execute based on job metadata or worker class.

Thread Safety

Middleware must be thread-safe since one instance is shared across all processor threads. Avoid instance variables for per-job state:

# BAD: instance variable shared across threads
class BrokenMiddleware
  def call(worker, job, queue)
    @job = job  # Thread-unsafe!
    yield
  end
end
 
# GOOD: local variable or thread-local storage
class SafeMiddleware
  def call(worker, job, queue)
    job_id = job['jid']  # Local variable, thread-safe
    yield
  end
end

For per-job state that must survive the yield, use thread-local storage: Thread.current[:state] = value.

See Sidekiq Architecture for how middleware fits into the overall job execution flow.