WILDS WDL Library

The WILDS WDL Library is a collection of workflow description language (WDL) scripts that provide reusable, well-tested bioinformatics tasks that can be combined into pipelines for research. All components are validated with real-world bioinformatics datasets to ensure they perform correctly in production environments. The library eliminates the need to write WDL workflows from scratch, ensures reproducibility through standardized containerization, and reduces debugging time with pre-tested, validated components.

You can:

• Build workflows faster from reusable, tested modules without reinventing common tasks
• Ensure reproducibility with versioned Docker containers and standardized components
• Execute workflows confidently on multiple platforms including PROOF, Cromwell, miniWDL, and Sprocket
• Learn WDL best practices through well-documented examples and pipelines

Library Organization

The library is organized into two complementary levels:

Modules

Tool-specific collections of reusable WDL tasks with comprehensive testing.

• Purpose: Foundational building blocks for larger workflows
• Content: Individual bioinformatics tools (STAR, BWA, GATK, etc.)
• Testing: Unit tests ensure each task functions correctly over time
• Usage: Import tasks into custom workflows or run demonstration workflows

Pipelines

Complete analysis workflows that combine multiple modules, ranging from simple educational examples to comprehensive production-ready analyses.

• Purpose: End-to-end analyses and templates for common bioinformatics patterns
• Content: Workflows combining multiple modules with varying complexity levels
• Testing: Integration and end-to-end tests verify modules work together seamlessly
• Usage: Run as production workflows or use as templates for custom analyses

Pipeline Complexity Levels

Level	Modules	Typical Runtime	Description
Basic	2-3	< 30 minutes	Simple integrations ideal for learning
Intermediate	4-6	1-4 hours	Multi-step analyses for common use cases
Advanced	10+	> 4 hours	Comprehensive production pipelines

Testing & Validation

All WILDS WDLs undergo rigorous testing with real-world bioinformatics data to ensure production readiness:

Automated Testing

• Cross-platform validation: All modules are automatically tested using Cromwell, miniWDL, and Sprocket executors
• Continuous integration: Every code change triggers automated test runs before merging
• Real data requirements: Tests use authentic bioinformatics datasets from the ww-testdata module

Validation by Component Type

• Modules: Unit tests with real sequencing files (FASTQ, BAM, VCF) verify each task functions correctly
• Pipelines: Integration and end-to-end tests confirm modules work together using representative analysis datasets suitable for publication-quality analyses

This testing approach ensures that WILDS WDLs perform reliably from individual tasks through complete analytical pipelines.

Getting Started

We recommend users start their exploration with a basic-level pipeline to learn how tasks can be imported and combined in a WDL workflow. From there, you can dive into task details at the module level or explore more complex pipelines. You can customize existing pipelines as necessary or create your own from scratch!

Downloading Pipelines

No git clone required! You can download any pipeline directly from GitHub:

1. Navigate to the pipeline you want in the pipelines folder
2. Click on the .wdl file, then click the download button (or “Raw” and save the page)
3. Do the same for inputs.json
4. Modify inputs.json as needed for your data

This works because all pipelines import modules using GitHub URLs, so your WDL executor fetches dependencies automatically.

Alternatively, if you prefer to have the full repository locally, you can clone it:

git clone https://github.com/getwilds/wilds-wdl-library.git

Running Pipelines

Once you have a WDL that you like, you can run it several ways:

Fred Hutch Users

You can use PROOF to submit WDL workflows to the cluster through a user-friendly interface:

1. Ensure you have access to the data you plan to analyze via the cluster
2. Make sure you are connected to the FH network and navigate to proof.fredhutch.org
3. Log in with your Fred Hutch credentials
4. Upload a WDL and its accompanying JSON files
5. Submit and monitor your job through the PROOF dashboard

Command Line

You can run WDLs locally if you have a WDL executor and Docker/Apptainer installed:

• Basic WDL executor options: miniWDL, sprocket
• Docker Desktop for container execution

For example, you can run WILDS WDLs from the terminal using Sprocket like so:

sprocket run ww-sra-star.wdl --inputs inputs.json

Importing into Your Workflows

You can import a WILDS WDL into your own WDL script:

version 1.0

import "https://raw.githubusercontent.com/getwilds/wilds-wdl-library/refs/heads/main/modules/ww-sra/ww-sra.wdl" as sra_tasks

workflow my_analysis {
  input {
    String sra_id
  }
  call sra_tasks.fastqdump { input: sra_id = sra_id }
  output {
    File fastq = fastqdump.fastq
  }
}

Note: Fred Hutch users must use WDL version 1.0 to run on PROOF

Then you can provide custom inputs using an inputs.json file:

{
  "sra_star.sra_id_list": ["SRR12345678"]
}

Available WDLs

The library now includes dozens of modules and pipelines, growing regularly as new tools and workflows are contributed. Rather than duplicate that list here (and have it go stale), browse the current set directly on GitHub:

• Modules - tool-specific, reusable WDL tasks for alignment (BWA, STAR), variant calling (GATK, BCFtools), structural variants (Manta, DELLY), and more
• Pipelines - complete workflows combining multiple modules, from basic educational examples to advanced production pipelines

Each module and pipeline folder includes its own README with usage details, container information, and testing status.

If there’s a tool you’d like to see or a task you want written, you can file an issue, reach out to us directly (see below), or make a contribution.

Key Features of the WILDS WDL Library

Quality Assurance

• Automated Testing: all scripts automatically execute full test runs before merging to main
• Multi-Executor Support: Compatible with Cromwell, miniWDL, and Sprocket
• Real Data Validation: Tested with realistic bioinformatics datasets

Different research institutions use different WDL execution engines (e.g., St. Jude’s uses Sprocket, CZI uses miniWDL, Broad Institute uses Cromwell). While these engines follow the same WDL specification, they have subtle differences in how they handle file paths and other execution details. The WILDS WDL Library is designed and tested to work seamlessly across all three major engines, ensuring your workflows remain portable regardless of which platform you or your collaborators use. This is particularly valuable as the WDL ecosystem continues to evolve, with institutions gradually migrating from Cromwell to newer engines like Sprocket and miniWDL.

Container Management

• Versioned, tested Docker images from the WILDS Docker Library
• Consistent container standards across all modules

Standards and Best Practices

• Standardized structure across all components
• Comprehensive documentation and inline comments
• WDL 1.0 specification compliance

Frequently Asked Questions

Can I use these workflows on my own compute infrastructure?

Yes! All workflows are designed to be portable and work with any WDL executor (Cromwell, miniWDL, Sprocket). Be sure you have one of these WDL executors (and Docker) installed.

Do I need to manually download Docker containers locally?

No, Docker will take care of pulling the necessary containers automatically when you run a workflow. You’ll need Docker installed but you don’t have to worry about pulling individual containers.

How can I contribute to the library?

We are very open to contributions from the Fred Hutch community and beyond! Our Contributing Guidelines describe these processes in detail, but feel free to reach out to us at wilds@fredhutch.org if you have questions.

Are these workflows production-ready?

Yes! All components undergo rigorous testing:

• Modules: Automated unit tests run on every update using Cromwell, miniWDL, and Sprocket to ensure cross-platform compatibility. Each module is validated with authentic bioinformatics datasets.
• Pipelines: Integration and end-to-end tests verify that modules work together seamlessly. Pipelines undergo comprehensive validation with realistic datasets and are suitable for research publications.

All testing is run through our continuous integration system. Users can reproduce these tests locally using the repository Makefile and test data from the ww-testdata module.

How do I get help with a specific workflow?

Contact the WILDS team at wilds@fredhutch.org, schedule a Data House Call with us, post on the #workflow-managers channel in FH-Data Slack, or open an issue on GitHub.

Release Notes

For detailed release notes, see the WILDS WDL Library releases page on GitHub.

Resources

Get Started

• Browse Available Modules
• Explore Pipelines
• Use PROOF for Easy Execution

Learn More

• WDL Best Practices Guide
• WILDS Docker Library
• Contributing Guidelines

Get Support

• Issues and Bug Reports: GitHub Issues
• General Questions: Email wilds@fredhutch.org
• Data House Calls: Schedule a 30-minute consultation session with us
• Slack: Join the #workflow-managers channel in FH-Data Slack