gsort

gsort is a tool to sort genomic files according to a genomefile.

For example, for some reason, you may want to sort your VCF to have order: X,Y,2,1,3,... and you want to keep the header at the top.

As a more likely example, you may want to sort your file to match GATK order (1 … X, Y, MT) which is not possible with any other sorting tool. With gsort one can simply place MT as the last chrom in the .genome file.

Given a genome file (lines of chrom\tlength) With this tool, you can sort a BED/VCF/GTF/… in the order dictated by that file with:

gsort --memory 1500 my.vcf.gz crazy.genome | bgzip -c > my.crazy-order.vcf.gz

where here, memory-use will be limited to 1500 megabytes.

We will use this to enforce chromosome ordering in ggd.

It will also be useful for getting your files ready for use in bedtools.

GFF parent

In GFF, the Parent attribute may refer to a row that would otherwise be sorted after it (based on the end position). But, some programs require that the row referenced in a Parent attribute be sorted first. If this is required, used the --parent flag introduced in version 0.0.6.

Performance

gsort can sort the 2 million variants in ESP in 15 seconds. It takes a few minutes to sort the ~10 million ExAC variants because of the huuuuge INFO strings in that file.

Usage

gsort will error if your genome file has ‘chr’ prefix and your file does not (or vice-versa).

It will write temporary files to your $TMPDIR (usually /tmp/) as needed to avoid using too much memory.

TODO

Specify a VCF for the genome file and pull order from the @SQ tags
Avoid temp file when everything can fit in memory. (more universally, last chunk can always be kept in memory).

API Documentation

– import “github.com/brentp/gsort”

Package gsort is a library for sorting a stream of tab-delimited lines ([]bytes) (from a reader) using the amount of memory requested.

Instead of using a compare function as most sorts do, this accepts a user-defined function with signature: func(line []byte) []int where the []ints are used to determine ordering. For example if we were sorting on 2 columns, one of months and another of day of months, the function would replace “Jan” with 1 and “Feb” with 2 for the first column and just return the Atoi of the 2nd column.

func Sort

func Sort(rdr io.Reader, wtr io.Writer, preprocess Processor, memMB int) error

Sort accepts a tab-delimited io.Reader and writes to wtr using prepocess to determine ordering

type Processor

type Processor func(line []byte) []int

Processor is a function that takes a line and return a slice of ints that determine ordering