diff --git a/README.md b/README.md index 0dc571c..175cdb3 100644 --- a/README.md +++ b/README.md @@ -1,71 +1,232 @@ # Workflow template -This repository contains a Nextflow workflow template and associated Docker -container build. The workflow also supports using conda environments as an -alternative software isolation method to Docker. +This repository contains a [nextflow](https://www.nextflow.io/) workflow +template that can be used as the basis for creating new workflows. + +> This workflow is not intended to be used by end users. + ## Quickstart -### Building the container +The workflow uses [nextflow](https://www.nextflow.io/) to manage compute and +software resources, as such nextflow will need to be installed before attempting +to run the workflow. -> This step is not necessary if you intend to run the workflow using -> conda environments, or are not interesting in developing of modifying -> the workflow. The current release version of the container is located -> on dockerhub with the tag ontresearch/wf-template. +The workflow can currently be run using either +[Docker](https://www.docker.com/products/docker-desktop) or +[conda](https://docs.conda.io/en/latest/miniconda.html) to provide isolation of +the required software. Both methods are automated out-of-the-box provided +either docker of conda is installed. -The Docker container image can be built with the following command: +> See the sections below for installation of these prerequisites in various scenarios. +> It is not required to clone or download the git repository in order to run the workflow. -```bash -CONTAINER_TAG=ontresearch/wf-template -docker build \ - -t ${CONTAINER_TAG} -f Dockerfile \ - --build-arg BASEIMAGE=ontresearch/base-workflow-image:v0.1.0 \ - . +**Workflow options** + +To obtain the workflow, having installed `nextflow`, users can run: + +``` +nextflow run epi2me-labs/wf-template --help ``` -The `BASEIMAGE` argument here can be changed to use an alternative image. +to see the options for the workflow. -### Running the workflow +**Workflow outputs** -The template includes a simple workflow that outputs a file with the lengths -of sequences contained in a .fastq.gz file. +The primary outputs of the workflow include: -**Running the workflow with Docker containers** +* a simple text file providing a summary of sequencing reads, +* an HTML report document detailing the primary findings of the workflow. + + +### Supported installations and GridION devices + +Installation of the software on a GridION can be performed using the command + +`sudo apt install ont-nextflow` + +This will install a java runtime, Nextflow and docker. If *docker* has not already been +configured the command below can be used to provide user access to the *docker* +services. Please logout of your computer after this command has been typed. + +`sudo usermod -aG docker $USER` + +### Installation on Ubuntu devices + +For hardware running Ubuntu the following instructions should suffice to install +Nextflow and Docker in order to run the workflow. + +1. Install a Jva runtime environment (JRE): + + ```sudo apt install default-jre``` + +2. Download and install Nextflow may be downloaded from https://www.nextflow.io: + + ```curl -s https://get.nextflow.io | bash``` + + This will place a `nextflow` binary in the current working directory, you + may wish to move this to a location where it is always accessible, e.g: + + ```sudo mv nextflow /usr/local/bin``` + +3. Install docker and add the current user to the docker group to enable access: + + ``` + sudo apt install docker.io + sudo usermod -aG docker $USER + ``` + +## Running the workflow + +The `wf-template` workflow can be controlled by the following parameters. The `fastq` parameter +is the most important parameter: it is required to identify the location of the +sequence files to be analysed. + +**Parameters:** + +- `fastq` specifies a *directory* path to FASTQ files (required) +- `out_dir` the path for the output (default: output) To run the workflow using Docker containers supply the `-profile standard` argument to `nextflow run`: +> The command below uses test data available from the [github repository](https://github.com/epi2me-labs/wf-template/tree/master/test_data) +> It can be obtained with `git clone https://github.com/epi2me-labs/wf-template`. + ``` -OUTPUT=workflow-output -nextflow run main.nf \ +# run the pipeline with the test data +OUTPUT=output +nextflow run epi2me-labs/wf-template \ -w ${OUTPUT}/workspace \ -profile standard \ --fastq test_data/reads.fq.gz \ --out_dir ${OUTPUT} ``` -The output of the pipeline will be found in `./workflow-output` for the above +The output of the pipeline will be found in `./output` for the above example. This directory contains the nextflow working directories alongside -the two primary outputs of the pipeline. +the two primary outputs of the pipeline: a `seqs.txt` file containing a summary +of all reads, and a `report.html` file summarising the workflows calculations. -**Using conda environments** +### Running the workflow with Conda -To run the workflow backed by conda environments, simply provide the -`-profile conda` argument to `nextflow run`. +To run the workflow using conda rather than docker, simply replace + + -profile standard + +with + + -profile conda + +in the command above. + +### Configuration and tuning + +> This section provides some minimal guidance for changing common options, see +> the [Nextflow documentation](https://www.nextflow.io/docs/latest/config.html) for further details. + +The default settings for the workflow are described in the configuration file `nextflow.config` +found within the git repository. The default configuration defines an *executor* that will +use a specified maximum CPU cores (four at the time of writing) and RAM (eight gigabytes). + +If the workflow is being run on a device other than a GridION, the available memory and +number of CPUs may be adjusted to the available number of CPU cores. This can be done by +creating a file `my_config.cfg` in the working directory with the following contents: ``` -# run the pipeline with the test data -OUTPUT=workflow-output -nextflow run main.nf \ - -w ${OUTPUT}/workspace \ - -profile conda \ - --fastq test_data/reads.fq.gz \ - --out_dir ${OUTPUT} +executor { + $local { + cpus = 4 + memory = "8 GB" + } +} ``` -This will create a conda environment with all required software within the -workspace directory. When running multiple analyses on distinct datasets -it may not be desirable to have Nextflow create a conda environment for each -analysis. To avoid the situation editing the file `nextflow.config` will -be necessary. Search for the term `cacheDir` and set this to a directory -where you wish the conda environment to be placed. +and running the workflow providing the `-c` (config) option, e.g.: + +``` +# run the pipeline with custom configuration +nextflow run epi2me-labs/wf-template \ + -c my_config.cfg \ + ... +``` + +The contents of the `my_config.cfg` file will override the contents of the default +configuration file. See the [Nextflow documentation](https://www.nextflow.io/docs/latest/config.html) +for more information concerning customized configuration. + +**Using a fixed conda environment** + +By default, Nextflow will attempt to create a fresh conda environment for any new +analysis (for reasons of reproducibility). This may be undesirable if many analyses +are being run. To avoid the situation a fixed conda environment can be used for all +analyses by creating a custom config with the following stanza: + +``` +profiles { + // profile using conda environments rather than docker + // containers + fixed_conda { + docker { + enabled = false + } + process { + withLabel:artic { + conda = "/path/to/my/conda/environment" + } + shell = ['/bin/bash', '-euo', 'pipefail'] + } + } +} +``` + +and running nextflow by setting the profile to `fixed_conda`: + +``` +nextflow run epi2me-labs/wf-template \ + -c my_config.cfg \ + -profile fixed_conda \ + ... +``` + +## Updating the workflow + +Periodically when running the workflow, users may find that a message is displayed +indicating that an update to the workflow is available. + +To update the workflow simply run: + + nextflow pull epi2me-labs/wf-template + +## Building the docker container from source + +The docker image used for running the `wf-template` workflow is available on +[dockerhub](https://hub.docker.com/repository/docker/ontresearch/wf-template). +The image is built from the Dockerfile present in the git repository. Users +wishing to modify and build the image can do so with: + +``` +CONTAINER_TAG=ontresearch/wf-template:latest + +git clone https://github.com/epi2me-labs/wf-template +cd wf-template + +docker build \ + -t ${CONTAINER_TAG} -f Dockerfile \ + --build-arg BASEIMAGE=ontresearch/base-workflow-image:v0.1.0 \ + . +``` + +In order to run the workflow with this new image it is required to give +`nextflow` the `--wfversion` parameter: + +``` +nextflow run epi2me-labs/wf-template \ + --wfversion latest +``` + +## Useful links + +* [nextflow](https://www.nextflow.io/) +* [docker](https://www.docker.com/products/docker-desktop) +* [conda](https://docs.conda.io/en/latest/miniconda.html)