DeepMAsED

Deep learning for Metagenome Assembly Error Detection (DeepMAsED)

“mased”

Middle English term: misled, bewildered, amazed, or perplexed

Citation

Mineeva, Olga, Mateo Rojas-Carulla, Ruth E. Ley, Bernhard Schölkopf, and Nicholas D. Youngblut. 2020. “DeepMAsED: Evaluating the Quality of Metagenomic Assemblies.” Bioinformatics , February.

Main Description

The tool is divided into two main parts:

DeepMAsED-SM
- A snakemake pipeline for generating DeepMAsED train/test datasets from reference genomes
DeepMAsED-DL
- A python package for misassembly detection via deep learning

Setup

Via the conda recipe

The simplest approach is to use the conda recipe:

conda create -n deepmased bioconda::deepmased

[alternative] The piecemeal setup

Dependency setup via conda

[If needed] Install miniconda (or anaconda)
See the conda create line in the .travis.yml file.
If just using DeepMAsED-SM:
- conda create -n snakemake conda-forge::pandas bioconda::snakemake

Testing the DeepMAsED package (optional)

pytest -s

Installing the DeepMAsED package into the conda environment

Via setup.py
- python setup.py install
Via pip
- pip install DeepMAsED

Usage

Example of classifying contig misassemblies

You need to have the following input:

fasta of metagenome assembly contigs (uncompressed)
BAM file of metagenome reads mapped to the contigs

Create table mapping BAM & fasta files

If multiple sets of contigs (eg., MAGs) and BAM files, then which contigs go with which BAM files?

Create a tab-delim table of: bam<tab>fasta (header required)

This will be your bam_fasta_table, which is need for creating the features.

Create feature table(s)

DeepMAsED features $bam_fasta_table

This generates >=1 feature table and a table listing all output files (the “feature_file_table”). This feature_file_table will be the input for predict

Predict misassemblies

DeepMAsED predict $feature_file_table

…where feature_filt_table is the path to a table that lists all feature files (see above).

--force-ovewrite forces the re-creation of the pkl files, which is a bit slower but can prevent issues.

Change --save-path to set the output directory. Use --cpu-only to just use CPUs instead of a GPU.

Third, inspect the output

By default, the predictions will be written to deepmased_predictions.tsv.

Example output

Collection     Contig  Deepmased_score
0       NODE_1156_length_5232_cov_4.046938      0.0007264018
0       NODE_1563_length_3868_cov_5.851298      0.03783685
0       NODE_4288_length_1225_cov_3.235897      0.070887744
1       k141_9081       8.8751316e-05
1       k141_2594       6.720424e-05
1       k141_4878       0.0015754104
2       NODE_5204_length_1290_cov_3.283401      0.00036007166
2       NODE_2848_length_2164_cov_2.982456      0.0005029738
2       NODE_446_length_6027_cov_5.812291       0.068261534

See Mineeva et al., 2020 to help decide what score cutoff is prudent for classifying misassembled contigs.

Creating training datasets with `DeepMAsED-SM`

This is useful for training DeepMAsED-DL with a custom train/test dataset (e.g., just biome-specific taxa).

Input

A table listing refernce genomes. Two possible formats:
- Genome-accession: <Taxon>\t<Accession>
  - “Taxon” = the species/strain name
  - “Accession” = the NCBI genbank genome accession
  - The genomes will be downloaded based on the accession
- Genome-fasta: <Taxon>\t<Fasta>
  - “Taxon” = the species/strain name of the genome
  - “Fasta” = the fasta of the genome sequence
  - Use this option if you already have the genome fasta files (uncompressed or gzip’ed)
The snakemake config file (e.g., config.yaml). This includes:
- Config params on MG communities
- Config params on assemblers & parameters

The column order for the tables doesn’t matter, but the column names must be exact.

Running locally

See the “Setup” section above for snakemake installation instructions.

cd ./DeepMAsED-SM/

Edit the config.yaml file as needed (eg., changing input & output paths)

snakemake --use-conda -j <NUMBER_OF_THREADS> --configfile <MY_CONFIG.yaml_FILE>

Running on SGE cluster

./snakemake_sge.sh <MY_CONFIG.yaml_FILE> cluster.json <PATH_FOR_SGE_LOGS> <NUMBER_OF_PARALLEL_JOBS> [additional snakemake options]

It should be rather easy to update the code to run on other cluster architectures. See the following resources for help:

Output

The output will the be same as for feature generation, but with extra directories:

./output/genomes/
- Reference genomes
./output/MGSIM/
- Simulated metagenomes
./output/assembly/
- Metagenome assemblies
./output/true_errors/
- Metagenome assembly errors determined by using the references
./output/map/
- Feature tables for each simulation

DeepMAsED-DL

Main interface: DeepMAsED -h

DeepMAsED [train|predict] can be run without GPUs, but the will be substantially slower.

Predicting with existing model

See DeepMAsED predict -h

Training a new model

See DeepMAsED train -h

Evaluating a model

See DeepMAsED evalulate -h

Creating features for `predict`

See DeepMAsED features -h

Features table

Basic info
- assembler
  - metagenome assembler used
- contig
  - contig ID
- position
  - position on the contig (bp)
- ref_base
  - nucleotide at that position on the contig
Extracted from the bam file
- num_query_A
  - number of reads mapping to that position with ‘A’
- num_query_C
  - number of reads mapping to that position with ‘C’
- num_query_G
  - number of reads mapping to that position with ‘G’
- num_query_T
  - number of reads mapping to that position with ‘T’
- num_SNPs
  - number of SNPs at that position
- coverage
  - number of reads mapping to that position
- num_discordant
  - discordant reads according to the read mapper definition
- num_supplementary
  - number of reads mapping to that position where the alignment is supplementary
  - see the samtools docs for more info
- num_secondary
  - number of reads mapping to that position where the alignment is secondary
  - see the samtools docs for more info
MetaQUAST info
- Extensive_misassembly
  - the “extensive misassembly” classification set by MetaQUAST