kfukuta/wf-transcriptomes-v202
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Standardise readme

Browse Source
This commit is contained in:
Chris Wright 2021-03-09 17:21:21 +00:00
parent b86690ea7d
commit 72ffce613c
1 changed files with 201 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

241
README.md
View File

@ -1,71 +1,232 @@
# Workflow template # Workflow template
This repository contains a Nextflow workflow template and associated Docker This repository contains a [nextflow](https://www.nextflow.io/) workflow
container build. The workflow also supports using conda environments as an template that can be used as the basis for creating new workflows.
alternative software isolation method to Docker.
> This workflow is not intended to be used by end users.
## Quickstart ## Quickstart
### Building the container The workflow uses [nextflow](https://www.nextflow.io/) to manage compute and
software resources, as such nextflow will need to be installed before attempting
to run the workflow.
> This step is not necessary if you intend to run the workflow using The workflow can currently be run using either
> conda environments, or are not interesting in developing of modifying [Docker](https://www.docker.com/products/docker-desktop) or
> the workflow. The current release version of the container is located [conda](https://docs.conda.io/en/latest/miniconda.html) to provide isolation of
> on dockerhub with the tag ontresearch/wf-template. the required software. Both methods are automated out-of-the-box provided
either docker of conda is installed.
The Docker container image can be built with the following command: > See the sections below for installation of these prerequisites in various scenarios.
> It is not required to clone or download the git repository in order to run the workflow.
```bash **Workflow options**
CONTAINER_TAG=ontresearch/wf-template
docker build \ To obtain the workflow, having installed `nextflow`, users can run:
-t ${CONTAINER_TAG} -f Dockerfile \
--build-arg BASEIMAGE=ontresearch/base-workflow-image:v0.1.0 \ ```
. nextflow run epi2me-labs/wf-template --help
``` ```
The `BASEIMAGE` argument here can be changed to use an alternative image. to see the options for the workflow.
### Running the workflow **Workflow outputs**
The template includes a simple workflow that outputs a file with the lengths The primary outputs of the workflow include:
of sequences contained in a .fastq.gz file.
**Running the workflow with Docker containers** * a simple text file providing a summary of sequencing reads,
* an HTML report document detailing the primary findings of the workflow.
### Supported installations and GridION devices
Installation of the software on a GridION can be performed using the command
`sudo apt install ont-nextflow`
This will install a java runtime, Nextflow and docker. If *docker* has not already been
configured the command below can be used to provide user access to the *docker*
services. Please logout of your computer after this command has been typed.
`sudo usermod -aG docker $USER`
### Installation on Ubuntu devices
For hardware running Ubuntu the following instructions should suffice to install
Nextflow and Docker in order to run the workflow.
1. Install a Jva runtime environment (JRE):
```sudo apt install default-jre```
2. Download and install Nextflow may be downloaded from https://www.nextflow.io:
```curl -s https://get.nextflow.io | bash```
This will place a `nextflow` binary in the current working directory, you
may wish to move this to a location where it is always accessible, e.g:
```sudo mv nextflow /usr/local/bin```
3. Install docker and add the current user to the docker group to enable access:
```
sudo apt install docker.io
sudo usermod -aG docker $USER
```
## Running the workflow
The `wf-template` workflow can be controlled by the following parameters. The `fastq` parameter
is the most important parameter: it is required to identify the location of the
sequence files to be analysed.
**Parameters:**
- `fastq` specifies a *directory* path to FASTQ files (required)
- `out_dir` the path for the output (default: output)
To run the workflow using Docker containers supply the `-profile standard` To run the workflow using Docker containers supply the `-profile standard`
argument to `nextflow run`: argument to `nextflow run`:
> The command below uses test data available from the [github repository](https://github.com/epi2me-labs/wf-template/tree/master/test_data)
> It can be obtained with `git clone https://github.com/epi2me-labs/wf-template`.
``` ```
OUTPUT=workflow-output # run the pipeline with the test data
nextflow run main.nf \ OUTPUT=output
nextflow run epi2me-labs/wf-template \
-w ${OUTPUT}/workspace \ -w ${OUTPUT}/workspace \
-profile standard \ -profile standard \
--fastq test_data/reads.fq.gz \ --fastq test_data/reads.fq.gz \
--out_dir ${OUTPUT} --out_dir ${OUTPUT}
``` ```
The output of the pipeline will be found in `./workflow-output` for the above The output of the pipeline will be found in `./output` for the above
example. This directory contains the nextflow working directories alongside example. This directory contains the nextflow working directories alongside
the two primary outputs of the pipeline. the two primary outputs of the pipeline: a `seqs.txt` file containing a summary
of all reads, and a `report.html` file summarising the workflows calculations.
**Using conda environments** ### Running the workflow with Conda
To run the workflow backed by conda environments, simply provide the To run the workflow using conda rather than docker, simply replace
`-profile conda` argument to `nextflow run`.
-profile standard
with
-profile conda
in the command above.
### Configuration and tuning
> This section provides some minimal guidance for changing common options, see
> the [Nextflow documentation](https://www.nextflow.io/docs/latest/config.html) for further details.
The default settings for the workflow are described in the configuration file `nextflow.config`
found within the git repository. The default configuration defines an *executor* that will
use a specified maximum CPU cores (four at the time of writing) and RAM (eight gigabytes).
If the workflow is being run on a device other than a GridION, the available memory and
number of CPUs may be adjusted to the available number of CPU cores. This can be done by
creating a file `my_config.cfg` in the working directory with the following contents:
``` ```
# run the pipeline with the test data executor {
OUTPUT=workflow-output $local {
nextflow run main.nf \ cpus = 4
-w ${OUTPUT}/workspace \ memory = "8 GB"
-profile conda \ }
--fastq test_data/reads.fq.gz \ }
--out_dir ${OUTPUT}
``` ```
This will create a conda environment with all required software within the and running the workflow providing the `-c` (config) option, e.g.:
workspace directory. When running multiple analyses on distinct datasets
it may not be desirable to have Nextflow create a conda environment for each ```
analysis. To avoid the situation editing the file `nextflow.config` will # run the pipeline with custom configuration
be necessary. Search for the term `cacheDir` and set this to a directory nextflow run epi2me-labs/wf-template \
where you wish the conda environment to be placed. -c my_config.cfg \
...
```
The contents of the `my_config.cfg` file will override the contents of the default
configuration file. See the [Nextflow documentation](https://www.nextflow.io/docs/latest/config.html)
for more information concerning customized configuration.
**Using a fixed conda environment**
By default, Nextflow will attempt to create a fresh conda environment for any new
analysis (for reasons of reproducibility). This may be undesirable if many analyses
are being run. To avoid the situation a fixed conda environment can be used for all
analyses by creating a custom config with the following stanza:
```
profiles {
// profile using conda environments rather than docker
// containers
fixed_conda {
docker {
enabled = false
}
process {
withLabel:artic {
conda = "/path/to/my/conda/environment"
}
shell = ['/bin/bash', '-euo', 'pipefail']
}
}
}
```
and running nextflow by setting the profile to `fixed_conda`:
```
nextflow run epi2me-labs/wf-template \
-c my_config.cfg \
-profile fixed_conda \
...
```
## Updating the workflow
Periodically when running the workflow, users may find that a message is displayed
indicating that an update to the workflow is available.
To update the workflow simply run:
nextflow pull epi2me-labs/wf-template
## Building the docker container from source
The docker image used for running the `wf-template` workflow is available on
[dockerhub](https://hub.docker.com/repository/docker/ontresearch/wf-template).
The image is built from the Dockerfile present in the git repository. Users
wishing to modify and build the image can do so with:
```
CONTAINER_TAG=ontresearch/wf-template:latest
git clone https://github.com/epi2me-labs/wf-template
cd wf-template
docker build \
-t ${CONTAINER_TAG} -f Dockerfile \
--build-arg BASEIMAGE=ontresearch/base-workflow-image:v0.1.0 \
.
```
In order to run the workflow with this new image it is required to give
`nextflow` the `--wfversion` parameter:
```
nextflow run epi2me-labs/wf-template \
--wfversion latest
```
## Useful links
* [nextflow](https://www.nextflow.io/)
* [docker](https://www.docker.com/products/docker-desktop)
* [conda](https://docs.conda.io/en/latest/miniconda.html)