kfukuta/wf-transcriptomes-v202
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'readme' into 'dev'

Browse Source 
Standardise readme

See merge request epi2melabs/workflow-containers/wf-template!3
This commit is contained in:
Chris Wright 2021-03-09 17:23:54 +00:00
commit e984ec361f
1 changed files with 201 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

241
README.md
View File

@ -1,71 +1,232 @@
# Workflow template
This repository contains a Nextflow workflow template and associated Docker
container build. The workflow also supports using conda environments as an
alternative software isolation method to Docker.
This repository contains a [nextflow](https://www.nextflow.io/) workflow
template that can be used as the basis for creating new workflows.
> This workflow is not intended to be used by end users.
## Quickstart
### Building the container
The workflow uses [nextflow](https://www.nextflow.io/) to manage compute and
software resources, as such nextflow will need to be installed before attempting
to run the workflow.
> This step is not necessary if you intend to run the workflow using
> conda environments, or are not interesting in developing of modifying
> the workflow. The current release version of the container is located
> on dockerhub with the tag ontresearch/wf-template.
The workflow can currently be run using either
[Docker](https://www.docker.com/products/docker-desktop) or
[conda](https://docs.conda.io/en/latest/miniconda.html) to provide isolation of
the required software. Both methods are automated out-of-the-box provided
either docker of conda is installed.
The Docker container image can be built with the following command:
> See the sections below for installation of these prerequisites in various scenarios.
> It is not required to clone or download the git repository in order to run the workflow.
```bash
CONTAINER_TAG=ontresearch/wf-template
docker build \
-t ${CONTAINER_TAG} -f Dockerfile \
--build-arg BASEIMAGE=ontresearch/base-workflow-image:v0.1.0 \
.
**Workflow options**
To obtain the workflow, having installed `nextflow`, users can run:
```
nextflow run epi2me-labs/wf-template --help
```
The `BASEIMAGE` argument here can be changed to use an alternative image.
to see the options for the workflow.
### Running the workflow
**Workflow outputs**
The template includes a simple workflow that outputs a file with the lengths
of sequences contained in a .fastq.gz file.
The primary outputs of the workflow include:
**Running the workflow with Docker containers**
* a simple text file providing a summary of sequencing reads,
* an HTML report document detailing the primary findings of the workflow.
### Supported installations and GridION devices
Installation of the software on a GridION can be performed using the command
`sudo apt install ont-nextflow`
This will install a java runtime, Nextflow and docker. If *docker* has not already been
configured the command below can be used to provide user access to the *docker*
services. Please logout of your computer after this command has been typed.
`sudo usermod -aG docker $USER`
### Installation on Ubuntu devices
For hardware running Ubuntu the following instructions should suffice to install
Nextflow and Docker in order to run the workflow.
1. Install a Jva runtime environment (JRE):
```sudo apt install default-jre```
2. Download and install Nextflow may be downloaded from https://www.nextflow.io:
```curl -s https://get.nextflow.io | bash```
This will place a `nextflow` binary in the current working directory, you
may wish to move this to a location where it is always accessible, e.g:
```sudo mv nextflow /usr/local/bin```
3. Install docker and add the current user to the docker group to enable access:
```
sudo apt install docker.io
sudo usermod -aG docker $USER
```
## Running the workflow
The `wf-template` workflow can be controlled by the following parameters. The `fastq` parameter
is the most important parameter: it is required to identify the location of the
sequence files to be analysed.
**Parameters:**
- `fastq` specifies a *directory* path to FASTQ files (required)
- `out_dir` the path for the output (default: output)
To run the workflow using Docker containers supply the `-profile standard`
argument to `nextflow run`:
> The command below uses test data available from the [github repository](https://github.com/epi2me-labs/wf-template/tree/master/test_data)
> It can be obtained with `git clone https://github.com/epi2me-labs/wf-template`.
```
OUTPUT=workflow-output
nextflow run main.nf \
# run the pipeline with the test data
OUTPUT=output
nextflow run epi2me-labs/wf-template \
-w ${OUTPUT}/workspace \
-profile standard \
--fastq test_data/reads.fq.gz \
--out_dir ${OUTPUT}
```
The output of the pipeline will be found in `./workflow-output` for the above
The output of the pipeline will be found in `./output` for the above
example. This directory contains the nextflow working directories alongside
the two primary outputs of the pipeline.
the two primary outputs of the pipeline: a `seqs.txt` file containing a summary
of all reads, and a `report.html` file summarising the workflows calculations.
**Using conda environments**
### Running the workflow with Conda
To run the workflow backed by conda environments, simply provide the
`-profile conda` argument to `nextflow run`.
To run the workflow using conda rather than docker, simply replace
-profile standard
with
-profile conda
in the command above.
### Configuration and tuning
> This section provides some minimal guidance for changing common options, see
> the [Nextflow documentation](https://www.nextflow.io/docs/latest/config.html) for further details.
The default settings for the workflow are described in the configuration file `nextflow.config`
found within the git repository. The default configuration defines an *executor* that will
use a specified maximum CPU cores (four at the time of writing) and RAM (eight gigabytes).
If the workflow is being run on a device other than a GridION, the available memory and
number of CPUs may be adjusted to the available number of CPU cores. This can be done by
creating a file `my_config.cfg` in the working directory with the following contents:
```
# run the pipeline with the test data
OUTPUT=workflow-output
nextflow run main.nf \
-w ${OUTPUT}/workspace \
-profile conda \
--fastq test_data/reads.fq.gz \
--out_dir ${OUTPUT}
executor {
$local {
cpus = 4
memory = "8 GB"
}
}
```
This will create a conda environment with all required software within the
workspace directory. When running multiple analyses on distinct datasets
it may not be desirable to have Nextflow create a conda environment for each
analysis. To avoid the situation editing the file `nextflow.config` will
be necessary. Search for the term `cacheDir` and set this to a directory
where you wish the conda environment to be placed.
and running the workflow providing the `-c` (config) option, e.g.:
```
# run the pipeline with custom configuration
nextflow run epi2me-labs/wf-template \
-c my_config.cfg \
...
```
The contents of the `my_config.cfg` file will override the contents of the default
configuration file. See the [Nextflow documentation](https://www.nextflow.io/docs/latest/config.html)
for more information concerning customized configuration.
**Using a fixed conda environment**
By default, Nextflow will attempt to create a fresh conda environment for any new
analysis (for reasons of reproducibility). This may be undesirable if many analyses
are being run. To avoid the situation a fixed conda environment can be used for all
analyses by creating a custom config with the following stanza:
```
profiles {
// profile using conda environments rather than docker
// containers
fixed_conda {
docker {
enabled = false
}
process {
withLabel:artic {
conda = "/path/to/my/conda/environment"
}
shell = ['/bin/bash', '-euo', 'pipefail']
}
}
}
```
and running nextflow by setting the profile to `fixed_conda`:
```
nextflow run epi2me-labs/wf-template \
-c my_config.cfg \
-profile fixed_conda \
...
```
## Updating the workflow
Periodically when running the workflow, users may find that a message is displayed
indicating that an update to the workflow is available.
To update the workflow simply run:
nextflow pull epi2me-labs/wf-template
## Building the docker container from source
The docker image used for running the `wf-template` workflow is available on
[dockerhub](https://hub.docker.com/repository/docker/ontresearch/wf-template).
The image is built from the Dockerfile present in the git repository. Users
wishing to modify and build the image can do so with:
```
CONTAINER_TAG=ontresearch/wf-template:latest
git clone https://github.com/epi2me-labs/wf-template
cd wf-template
docker build \
-t ${CONTAINER_TAG} -f Dockerfile \
--build-arg BASEIMAGE=ontresearch/base-workflow-image:v0.1.0 \
.
```
In order to run the workflow with this new image it is required to give
`nextflow` the `--wfversion` parameter:
```
nextflow run epi2me-labs/wf-template \
--wfversion latest
```
## Useful links
* [nextflow](https://www.nextflow.io/)
* [docker](https://www.docker.com/products/docker-desktop)
* [conda](https://docs.conda.io/en/latest/miniconda.html)