What is an Execution Engine?

A workflow execution engine is software that reads your WDL file and actually runs it. The engine handles the complex behind-the-scenes work of determining which tasks can run in parallel, managing where computations happen, tracking inputs and outputs, and dealing with failures. Essentially, it’s the interpreter that transforms your workflow into real computational work.

This guide covers the three major WDL execution engines (Cromwell, Sprocket, miniWDL) and how to use them, with specific guidance for Fred Hutch researchers.

New to WDL? Start with our WDL Workflow guide to learn about the language itself before diving into execution engines.

Cromwell

Cromwell is a production-grade workflow execution engine developed by the Broad Institute. It powers platforms like Terra and is the engine behind Fred Hutch’s PROOF platform.

Cromwell at Fred Hutch

Key features for HPC execution:

  • Call caching - reuses results from previous identical tasks
  • Server mode - manages multiple workflows simultaneously via PROOF
  • SLURM integration - optimized for Fred Hutch’s Gizmo cluster
  • Production-tested - used by major research institutions worldwide

When to use Cromwell:

  • Running workflows on Fred Hutch’s HPC cluster via PROOF
  • Large-scale production pipelines requiring call caching
  • Advanced users who need to run their own Cromwell server

At Fred Hutch, Cromwell is the primary execution engine for WDL workflows on our HPC infrastructure. We provide custom configurations and tools to make it easier to use.

Running Cromwell with PROOF

PROOF provides a user-friendly web interface for submitting WDL workflows to our cluster:

  1. Connect to the Fred Hutch network
  2. Navigate to proof.fredhutch.org
  3. Log in with your Hutch credentials
  4. Start a PROOF server with a click of a button
  5. Upload your WDL file and inputs JSON
  6. Submit and monitor your workflow through the dashboard

PROOF handles all Cromwell server setup, configuration, and job scheduling automatically - the “server” is just a background process that coordinates your workflow execution.

It is possible to set up and run your own Cromwell server on Fred Hutch’s Gizmo cluster, however this requires more advanced configuration and maintenance. We recommend using PROOF for most users, but for more details on manually running your own Cromwell server, see the diy-cromwell-server for instructions and corresponding configuration files.

What Cromwell Does Behind the Scenes

When you submit a workflow via PROOF, Cromwell handles several important tasks automatically:

Call caching - Cromwell remembers previously completed tasks and skips re-running them if inputs haven’t changed. Enable this in PROOF using write_to_cache and read_from_cache in your options JSON. This saves time when restarting workflows or testing parameter changes.

Automatic retries - Failed tasks can be automatically retried by adding maxRetries to your WDL runtime section. Combined with call caching, you can restart workflows without losing progress on completed tasks.

Scratch space management - PROOF automatically uses /fh/scratch for intermediate files, keeping final outputs in your specified location via final_workflow_outputs_dir. This is essential for genomics workflows that generate large temporary files.

Cromwell Documentation

Sprocket

Sprocket is a modern WDL execution engine developed at St. Jude Children’s Research Hospital. It’s designed for ease of use with a focus on local execution and cloud platforms, with active ongoing development.

Why Choose Sprocket?

  • Simple installation: Single binary download, no dependencies
  • Clear error messages: Helpful, actionable feedback when things go wrong
  • Active development: Regularly updated with new features and improvements from St. Jude
  • Cloud-native design: Good AWS/Azure integration

Getting Started with Sprocket

Sprocket is distributed as a single binary with no dependencies. For installation instructions, see the Sprocket Installation Guide.

# Run a workflow
sprocket workflow.wdl inputs.json --entrypoint workflow_name

# Lint a WDL file
sprocket lint workflow.wdl

For more details on Sprocket commands and options, see the Sprocket Guided Tour in their documentation.

Sprocket at Fred Hutch

While PROOF uses Cromwell in the Fred Hutch HPC environment, Sprocket is our recommended tool for local workflow development:

  • Test workflows locally before submitting to PROOF
  • Develop new tasks with quick iteration cycles
  • Learn WDL with clear error messages and fast feedback

All WILDS WDL Library components are tested with Sprocket and work identically on PROOF/Cromwell.

Sprocket Documentation

miniWDL

miniWDL is a lightweight WDL execution engine developed by the Chan Zuckerberg Initiative. It’s an alternative option for local development and testing.

Why Choose miniWDL?

  • Simple setup: Install with pip install miniwdl
  • Fast execution: Minimal overhead for local workflows
  • Local-first: Designed for local development and testing

Getting Started with miniWDL

# Install via pip
pip install miniwdl

# Run a workflow
miniwdl run workflow.wdl -i inputs.json

# Validate WDL without running
miniwdl check workflow.wdl

For more installation options, see the miniWDL Installation Guide. For more details on WDL script execution options with miniWDL, see miniWDL’s documentation on running workflows.

miniWDL Documentation

Summary

Feature Cromwell Sprocket miniWDL
Best For Shared HPC systems, production pipelines Local execution, development and testing Alternative for local testing
Ease of Setup Complex (server mode recommended) Simple (single binary) Simple (single command install)
Key Features Call caching, server mode, extensive backend support Lightweight, modern architecture, active development Fast local execution, good error messages
Fred Hutch Support Full (PROOF, dedicated configs) Community support Community support
Learning Curve Steeper Gentle Gentle
When to Use Running on Gizmo, AWS Batch; production workflows Local testing and development; actively maintained Local testing when Sprocket isn’t available

For Fred Hutch researchers running on Gizmo or AWS Batch:

  • Use PROOF (Cromwell backend) for workflow submission with all features managed for you
  • Use Sprocket or miniWDL locally to test before scaling up

For local workflow development and testing:

  • Use Sprocket for easy installation, clear error messages, and actively maintained modern execution
  • Use miniWDL as an alternative if Sprocket doesn’t meet your needs

All WDLs in the WILDS WDL Library have been tested on Cromwell, miniWDL, and Sprocket. We recommend starting with these workflows to get up to speed with WDL execution via any of these engines.

Getting Help

Fred Hutch Support

Community Resources

Next Steps

  1. Learn WDL basics - See WDL Workflows guide
  2. Try local execution - Install Sprocket or miniWDL and run a WILDS WDL vignette
  3. Scale to HPC - Use PROOF to run on Fred Hutch infrastructure
  4. Join the community - Connect in #workflow-managers Slack

Updated: