Individual steps (Quick Tools)
DEMULTIPLEX
If data is multiplexed, the first step would be demultiplexing (using cutadapt (Martin 2011)). This is done based on the user specified indexes file, which includes molecular identifier sequences (so called indexes/tags/barcodes) per sample. Note that reverse complementary matches will also be searched.
demultiplexed_out
directory. Indexes are truncated from the sequences..R1
and .R2
read identifiers.Note
When using paired indexes, then sequences with any index combination will be outputted to ‘unnamed_index_combinations’ dir. That means, if, for example, your sample_1 is indexed with indexFwd_1-indexRev_1 and sample_2 with indexFwd_2-indexRev_2, then files with indexFwd_1-indexRev_2 and indexFwd_2-indexRev_1 are also written (although latter index combinations were not used in the lab to index any sample [i.e. represent tag-switches]). Simply remove those files if not needed or use to estimate tag-switching error if relevant.
Setting |
Tooltip |
---|---|
|
select your fasta formatted indexes file for demultiplexing (see guide here),
where fasta headers are sample names, and sequences are sample
specific index or index combination
|
|
allowed mismatches during the index search
|
|
number of overlap bases with the index
Recommended overlap is the maximum length of the index for
confident sequence assignments to samples
|
|
the index search window size. The default 35 means that
the forward index is searched among the first 35 bp and the reverse
index among the last 35 bp. This search restriction prevents random
index matches in the middle of the sequence
|
|
minimum length of the output sequence
|
|
do not allow insertions or deletions is primer search.
Mismatches are the only type of errors accounted in the error rate parameter
|
Note
Heterogenity spacers or any redundant base pairs attached to index sequences do not affect demultiplexing. Indexes are trimmed from the best matching position.
Indexes file example (fasta formatted)
Note
Only IUPAC codes are allowed in the sequences. Avoid using ‘.’ in the sample names (e.g. instead of sample.1, use sample_1)
Demultiplexing using single indexes:
>sample1AGCTGCACCTAA>sample2AGCTGTCAAGCT>sample3AGCTTCGACAGT>sample4AGGCTCCATGTA>sample5AGGCTTACGTGT>sample6AGGTACGCAATT
Demultiplexing using paired (dual) indexes:
Note
IMPORTANT! reverse indexes will be automatically oriented to 5’-3’ (for the search); so you can simply copy-paste the indexes from your lab protocol.
Note
Anchored indexes (https://cutadapt.readthedocs.io/en/stable/guide.html#anchored-5adapters) with ^ symbol are not supported in PipeCraft demultiplex GUI panel.
DO NOT USE, e.g.
How to compose indexes.fasta
In Excel (or any alternative program); first column represents sample names, second (and third) column represent indexes (or index combinations) per sample:
Example of single-end indexes
sample1 AGCTGCACCTAA
sample2 AGCTGTCAAGCT
sample3 AGCTTCGACAGT
sample4 AGGCTCCATGTA
sample5 AGGCTTACGTGT
sample6 AGGTACGCAATT
Example of paired indexes
sample1 AGCTGCACCTAA AGCTGCACCTAA
sample2 AGCTGTCAAGCT AGCTGTCAAGCT
sample3 AGCTTCGACAGT AGCTTCGACAGT
sample4 AGGCTCCATGTA AGGCTCCATGTA
sample5 AGGCTTACGTGT AGGCTTACGTGT
sample6 AGGTACGCAATT AGGTACGCAATT
Copy those two (or three) columns to text editor that support regular expressions, such as NotePad++ or Sublime Text.
single-end indexes:
Open ‘find & replace’ Find ^ (which denotes the beginning of each line). Replace with > (and DELETE THE LAST > in the beginning of empty row).
Find \t (which denotes tab). Replace with \n (which denotes the new line).
FASTA FORMATTED (single-end indexes) indexes.fasta file is ready; SAVE the file.
Paired indexes:
Open ‘find & replace’: Find ^ (denotes the beginning of each line); replace with > (and DELETE THE LAST > in the beginning of empty row).
Find .*\K\t (which captures the second tab); replace with … (to mark the linked paired-indexes).
Find \t (denotes the tab); replace with \n (denotes the new line).
FASTA FORMATTED (paired indexes) indexes.fasta file is ready; SAVE the file.
REORIENT
Sequences are often (if not always) in both, 5’-3’ and 3’-5’, orientations in the raw sequencing data sets. If the data still contains PCR primers that were used to generate amplicons, then by specifying these PCR primers, this panel will perform sequence reorientation of all sequences.
For reorienting, first the forward primer will be searched (using fqgrep) and if detected then the read is considered as forward complementary (5’-3’). Then the reverse primer will be searched (using fqgrep) from the same input data and if detected, then the read is considered to be in reverse complementary orientation (3’-5’). Latter reads will be transformed to 5’-3’ orientation and merged with other 5’-3’ reads. Note that for paired-end data, R1 files will be reoriented to 5’-3’ but R2 reads will be reoriented to 3’-5’ in order to merge paired-end reads.
At least one of the PCR primers must be found in the sequence. For example, read will be recorded if forward primer was found even though reverse primer was not found (and vice versa). Sequence is discarded if none of the PCR primers are found.
Sequences that contain multiple forward or reverse primers (multi-primer artefacts) are discarded as it is highly likely that these are chimeric sequences. Reorienting sequences will not remove primer strings from the sequences.
Note
For single-end data, sequences will be reoriented also during the ‘cut primers’ process (see below); therefore this step may be skipped when working with single-end data (such as data from PacBio machines OR already assembled paired-end data).
If in the clustering step of an “OTU pipeline”, both strands of the sequences can be compared prior forming OTUs; thus this step may be skipped in the OTU pipeline.
Supported file formats for paired-end input data are only fastq,
but also fasta for single-end data.
Outputs are fastq/fasta files in reoriented_out
directory.
Primers are not truncated from the sequences; this can be done using CUT PRIMER panel
Setting |
Tooltip |
---|---|
|
allowed mismatches in the primer search
|
|
specify forward primer (5’-3’); IUPAC codes allowed;
add up to 13 primers
|
|
specify reverse primer (3’-5’); IUPAC codes allowed;
add up to 13 primers
|
CUT PRIMERS
If the input data contains PCR primers (or e.g. adapters), these can be removed in the CUT PRIMERS
panel.
CUT PRIMERS processes mostly relies on cutadapt (Martin 2011).
For generating OTUs or ASVs, it is recommended to truncate the primers from the reads (unless ITS Extractor is used later to remove flanking primer binding regions from ITS1/ITS2/full ITS; in that case keep the primers better detection of the 18S, 5.8S and/or 28S regions). Sequences where PCR primer strings were not detected are discarded by default (but stored in ‘untrimmed’ directory). Reverse complementary search of the primers in the sequences is also performed. Thus, primers are clipped from both 5’-3’ and 3’-5’ oriented reads. However, note that paired-end reads will not be reoriented to 5’-3’ during this process, but single-end reads will be reoriented to 5’-3’ (thus no extra reorient step needed for single-end data).
Note
For paired-end data, the seqs_to_keep option should be left as default (‘keep_all’). This will output sequences where at least one primer has been clipped. ‘keep_only_linked’ option outputs only sequences where both the forward and reverse primers are found (i.e. 5’-forward…reverse-3’). ‘keep_only_linked’ may be used for single-end data to keep only full-length amplicons.
primersCut_out
directory. Primers are truncated from the sequences.Setting |
Tooltip |
---|---|
|
specify forward primer (5’-3’); IUPAC codes allowed;
add up to 13 primers
|
|
specify reverse primer (3’-5’); IUPAC codes allowed;
add up to 13 primers
|
|
allowed mismatches in the primer search
|
|
number of overlap bases with the primer sequence.
Partial matches are allowed, but short matches may occur by chance,
leading to erroneously clipped bases.
Specifying higher overlap than the length of primer sequnce
will still clip the primer (e.g. primer length is 22 bp,
but overlap is specified as 25 - this does not affect the
identification and clipping of the primer as long as the match is
in the specified mismatch error range)
|
|
keep sequences where at least one primer was found (fwd or rev);
recommended when cutting primers from paired-end data (unassembled),
when individual R1 or R2 read lengths are shorther than the expected
amplicon length. ‘keep_only_linked’ = keep sequences if primers are found
in both ends (fwd…rev); discards the read if both primers were not found
in this read
|
|
applies only for paired-end data.
‘both’, means that a read is discarded only if both, corresponding R1 and R2,
reads do not contain primer strings (i.e. a read is kept if R1 contains
primer string, but no primer string found in R2 read). Option ‘any’ discards
the read if primers are not found in both, R1 and R2 reads
|
|
minimum length of the output sequence
|
|
do not allow insertions or deletions is primer search. Mismatches are the
only type of errprs accounted in the error rate parameter
|
QUALITY FILTERING
Quality filter and trim sequences.
qualFiltered_out
directory.vsearch
vsearch setting |
Tooltip |
---|---|
|
maximum number of expected errors per sequence (see here).
Sequences with higher error rates will be discarded
|
|
discard sequences with more than the specified number of Ns
|
|
minimum length of the filtered output sequence
|
|
discard sequences with more than the specified number of bases.
Note that if ‘trunc length’ setting is specified, then ‘max length’
SHOULD NOT be lower than ‘trunc length’ (otherwise all reads are discared)
[empty field = no action taken]
Note that if ‘trunc length’ setting is specified, then ‘min length’
SHOULD BE lower than ‘trunc length’ (otherwise all reads are discared)
|
|
specify the maximum quality score accepted when reading FASTQ files.
The default is 41, which is usual for recent Sanger/Illumina 1.8+ files.
For PacBio data use 93
|
|
truncate sequences to the specified length. Shorter sequences are discarded;
thus if specified, check that ‘min length’ setting is lower than ‘trunc length’
(‘min length’ therefore has basically no effect) [empty field = no action taken]
|
|
the minimum quality score accepted for FASTQ files. The default is 0, which is
usual for recent Sanger/Illumina 1.8+ files.
Older formats may use scores between -5 and 2
|
|
discard sequences with more than the specified number of expected errors per base
|
|
discard sequences with an abundance lower than the specified value
|
trimmomatic
trimmomatic setting |
Tooltip |
---|---|
|
the number of bases to average base qualities
Starts scanning at the 5’-end of a sequence and trimms the read once the
average required quality (required_qual) within the window size falls
below the threshold
|
|
the average quality required for selected window size
|
|
minimum length of the filtered output sequence
|
|
quality score threshold to remove low quality bases from the beginning of the read.
As long as a base has a value below this threshold the base is removed and
the next base will be investigated
|
|
quality score threshold to remove low quality bases from the end of the read.
As long as a base has a value below this threshold the base is removed and
the next base will be investigated
|
|
phred quality scored encoding.
Use phred64 if working with data from older Illumina (Solexa) machines
|
fastp
fastp setting |
Tooltip |
---|---|
|
the window size for calculating mean quality
|
|
the mean quality requirement per sliding window (window_size)
|
|
the quality value that a base is qualified. Default 15 means
phred quality >=Q15 is qualified
|
|
how many percents of bases are allowed to be unqualified (0-100)
|
|
discard sequences with more than the specified number of Ns
|
|
minimum length of the filtered output sequence. Shorter sequences are discarded
|
|
reads longer than ‘max length’ will be discarded, default 0 means no limitation
|
|
truncate sequences to specified length. Shorter sequences are discarded;
thus check that ‘min length’ setting is lower than ‘trunc length’
|
|
if one read’s average quality score <’aver_qual’, then this read/pair is discarded.
Default 0 means no requirement
|
|
enables low complexity filter and specify the threshold for low complexity filter.
The complexity is defined as the percentage of base that is different from its
next base (base[i] != base[i+1]).
E.g. vaule 30 means then 30% complexity is required.
Not specified = filter not applied
|
|
number of cores to use
|
DADA2 (‘filterAndTrim’ function)
DADA2 setting |
Tooltip |
---|---|
|
applies only for paired-end data.
Identifyer string that is common for all R1 reads
(e.g. when all R1 files have ‘.R1’ string, then enter ‘\.R1’.
Note that backslash is only needed to escape dot regex; e.g.
when all R1 files have ‘_R1’ string, then enter ‘_R1’.).
|
|
applies only for paired-end data.
Identifyer string that is common for all R2 reads
(e.g. when all R2 files have ‘.R2’ string, then enter ‘\.R2’.
Note that backslash is only needed to escape dot regex; e.g.
when all R2 files have ‘_R1’ string, then enter ‘_R2’.).
|
|
discard sequences with more than the specified number of expected errors
|
|
discard sequences with more than the specified number of N’s (ambiguous bases)
|
|
remove reads with length less than minLen. minLen is enforced
after all other trimming and truncation
|
|
truncate reads at the first instance of a quality score less than or equal to truncQ
|
|
truncate reads after truncLen bases
(applies to R1 reads when working with paired-end data).
Reads shorter than this are discarded.
Explore quality profiles (with QualityCheck module) and
see whether poor quality ends needs to be truncated
|
|
applies only for paired-end data.
Truncate R2 reads after truncLen bases.
Reads shorter than this are discarded.
Explore quality profiles (with QualityCheck module) and
see whether poor quality ends needs to truncated
|
|
remove reads with length greater than maxLen.
maxLen is enforced on the raw reads.
In dada2, the default = Inf, but here set as 9999
|
|
after truncation, reads contain a quality score below minQ will be discarded
|
|
applies only for paired-end data.
after truncation, reads contain a quality score below minQ will be discarded
|
ASSEMBLE PAIRED-END reads
Assemble paired-end sequences (such as those from Illumina or MGI-Tech platforms).
include_only_R1
represents additional in-built module. If TRUE,
unassembled R1 reads will be included to the set of assembled reads per sample.
This may be relevant when working with e.g. ITS2 sequences, because the ITS2 region in some
taxa is too long for paired-end assembly using current short-read sequencing technology.
Therefore longer ITS2 amplicon sequences are discarded completely after the assembly process.
Thus, including also unassembled R1 reads (include_only_R1
= TRUE), partial ITS2 sequences for
these taxa will be represented in the final output. But when using ITSx
, keep only_full
= FALSE and include partial
= 50.
Fastq formatted paired-end data is supported.
Outputs are fastq files in assembled_out
directory.
vsearch
Setting |
Tooltip |
---|---|
|
applies only for paired-end data. Identifyer string that is common
for all R1 reads (e.g. when all R1 files have ‘.R1’ string, then
enter ‘\.R1’. Note that backslash is only needed to escape dot
regex; e.g. when all R1 files have ‘_R1’ string, then enter ‘_R1’)’
|
|
minimum overlap between the merged reads
|
|
minimum length of the merged sequence
|
|
allow to merge staggered read pairs. Staggered pairs are pairs
where the 3’ end of the reverse read has an overhang to the left
of the 5’ end of the forward read. This situation can occur when a
very short fragment is sequenced
|
|
include unassembled R1 reads to the set of assembled reads per sample
|
|
the maximum number of non-matching nucleotides allowed in the overlap region
|
|
discard sequences with more than the specified number of Ns
|
|
maximum length of the merged sequence
|
|
output reads that were not merged into separate FASTQ files
|
|
maximum quality score accepted when reading FASTQ files.
The default is 41, which is usual for recent Sanger/Illumina 1.8+ files
|
DADA2
Important
Here, dada2 will perform also denoising (function ‘dada’) before assembling paired-end data. Because of that, input sequences (in fastq format) must consist of only A/T/C/Gs.
Setting |
Tooltip |
---|---|
|
identifyer string that is common for all R1 reads
(e.g. when all R1 files have ‘.R1’ string, then enter ‘\.R1’.
Note that backslash is only needed to escape dot regex; e.g.
when all R1 files have ‘_R1’ string, then enter ‘_R1’.)
|
|
identifyer string that is common for all R2 reads
(e.g. when all R2 files have ‘.R2’ string, then enter ‘\.R2’.
Note that backslash is only needed to escape dot regex; e.g.
when all R2 files have ‘_R1’ string, then enter ‘_R2’.)
|
|
the minimum length of the overlap required for merging the forward and
reverse reads
|
|
the maximum mismatches allowed in the overlap region
|
|
if TRUE, overhangs in the alignment between the forwards and reverse read are
trimmed off. Overhangs are when the reverse read extends past the start of
the forward read, and vice-versa, as can happen when reads are longer than the
amplicon and read into the other-direction primer region
|
|
if TRUE, the forward and reverse-complemented reverse read are concatenated
rather than merged, with a NNNNNNNNNN (10 Ns) spacer inserted between them
|
|
denoising setting. If TRUE, the algorithm will pool together all samples
prior to sample inference. Pooling improves the detection of rare variants,
but is computationally more expensive.
If pool = ‘pseudo’, the algorithm will perform pseudo-pooling between
individually processed samples.
|
|
denoising setting. If TRUE, the algorithm will alternate between sample
inference and error rate estimation until convergence
|
|
‘Auto’ means to attempt to auto-detect the fastq quality encoding.
This may fail for PacBio files with uniformly high quality scores,
in which case use ‘FastqQuality’
|
CHIMERA FILTERING
Perform de-novo and reference database based chimera filtering.
Chimera filtering is performed by sample-wise approach (i.e. each sample (input file) is treated separately).
chimera_Filtered_out
directory.uchime_denovo
Setting |
Tooltip |
---|---|
|
identity percentage when performing ‘pre-clustering’ with –cluster_size
for denovo chimera filtering with –uchime_denovo
|
|
minimum amount of a unique sequences in a fasta file. If value = 1, then
no sequences are discarded after dereplication; if value = 2, then sequences,
which are represented only once in a given file are discarded; and so on
|
|
if TRUE, then perform denovo chimera filtering with –uchime_denovo
|
|
perform reference database based chimera filtering with –uchime_ref.
Select fasta formatted reference database (e.g. UNITE for ITS reads).
If denovo = TRUE, then reference based chimera filtering will be performed
after denovo.
|
|
the abundance skew is used to distinguish in a threeway alignment which
sequence is the chimera and which are the parents. The assumption is that
chimeras appear later in the PCR amplification process and are therefore
less abundant than their parents. The default value is 2.0, which means that
the parents should be at least 2 times more abundant than their chimera.
Any positive value equal or greater than 1.0 can be used
|
|
minimum score (h). Increasing this value tends to reduce the number of false
positives and to decrease sensitivity. Values ranging from 0.0 to 1.0 included
are accepted
|
uchime3_denovo
Setting |
Tooltip |
---|---|
|
identity percentage when performing ‘pre-clustering’ with –cluster_size
for denovo chimera filtering with –uchime_denovo
|
|
minimum amount of a unique sequences in a fasta file. If value = 1, then
no sequences are discarded after dereplication; if value = 2, then sequences,
which are represented only once in a given file are discarded; and so on
|
|
if TRUE, then perform denovo chimera filtering with –uchime_denovo
|
|
perform reference database based chimera filtering with –uchime_ref.
Select fasta formatted reference database (e.g. UNITE for ITS reads).
If denovo = TRUE, then reference based chimera filtering will be performed
after denovo.
|
|
the abundance skew is used to distinguish in a threeway alignment which
sequence is the chimera and which are the parents. The assumption is that
chimeras appear later in the PCR amplification process and are therefore
less abundant than their parents. The default value is 2.0, which means that
the parents should be at least 2 times more abundant than their chimera.
Any positive value equal or greater than 1.0 can be used
|
|
minimum score (h). Increasing this value tends to reduce the number of false
positives and to decrease sensitivity. Values ranging from 0.0 to 1.0 included
are accepted
|
ITS Extractor
When working with ITS amplicons, then extract ITS regions with ITS Extractor (Bengtsson-Palme et al. 2013)
Note
Note that for better detection of the 18S, 5.8S and/or 28S regions, keep the primers (i.e. do not use ‘CUT PRIMERS’)
ITSx_out
directory.Note
To START, specify working directory under SELECT WORKDIR
and the sequence files extension
, but the read types (single-end or paired-end) does not matter here (just click ‘Next’).
Setting |
Tooltip |
---|---|
|
set of profiles to use for the search. Can be used to restrict the search to
only a few organism groups types to save time, if one or more of the origins
are not relevant to the dataset under study
|
|
ITS regions to output (note that ‘all’ will output also full ITS region [ITS1-5.8S-ITS2])
|
|
if larger than 0, ITSx will save additional FASTA-files for full and partial ITS sequences
longer than the specified cutoff value. If his setting is left to 0 (zero),
it means OFF
|
|
domain e-value cutoff a sequence must obtain in the HMMER-based step to be
included in the output
|
|
domain score cutoff that a sequence must obtain in the HMMER-based step to
be included in the output
|
|
the minimum number of domains (different HMM gene profiles) that must match
a sequence for it to be included in the output (detected as an ITS sequence).
Setting the value lower than two will increase the number of false positives,
while increasing it above two will decrease ITSx detection abilities
on fragmentary data
|
|
if TRUE, ITSx checks both DNA strands for matches to HMM-profiles
|
|
If TRUE, the output is limited to full-length ITS1 and ITS2 regions only
|
|
removes ends of ITS sequences if they are outside of the ITS region.
If FALSE, the whole input sequence is saved
|
CLUSTERING
Cluster sequences, generate OTUs or zOTUs (with UNOISE3)
clustering_out
directory.Note
output OTU table is tab delimited text file.
vsearch
Tooltip |
|
---|---|
|
centroid” = output centroid sequences; “consensus” = output
consensus sequences
|
|
define OTUs based on the sequence similarity threshold; 0.97 = 97%
similarity threshold
|
|
when comparing sequences with the cluster seed, check both strands
(forward and reverse complementary) or the plus strand only
|
|
if TRUE, then singleton OTUs will be discarded (OTUs with only one sequence)
|
|
pairwise sequence identity definition –iddef
|
|
size = sort the sequences by decreasing abundance;
“length” = sort the sequences by decreasing length (–cluster_fast);
“no” = do not sort sequences (–cluster_smallmem –usersort)
|
|
“similarity” = assign representative sequence to the closest (most similar)
centroid (distance-based greedy clustering);
“abundance” = assign representative sequence to the most abundant centroid
(abundance-based greedy clustering; –sizeorder),
max_hits should be > 1 |
|
maximum number of hits to accept before stopping the search
(should be > 1 for abundance-based selection of centroids [centroid type])
|
|
mask regions in sequences using the “dust” method, or do not mask (“none”)
|
|
prior the OTU table creation, mask regions in sequences using the
“dust” method, or do not mask (“none”)
|
UNOISE3, with vsearch
Tooltip |
|
---|---|
|
sequence similarity threshold for zOTU table creation;
1 = 100% similarity threshold for zOTUs
|
|
optionally cluster zOTUs to OTUs based on the sequence similarity threshold;
if id = 1, no OTU clustering will be performed
|
|
pairwise sequence identity definition for OTU clustering
|
|
maximum number of hits to accept before stopping the search
|
|
maximum number of non-matching target sequences to consider before stopping the search
|
|
mask regions in sequences using the “dust” method, or do not mask (“none”)
|
|
when comparing sequences with the cluster seed,
check both strands (forward and reverse complementary) or the plus strand only
|
|
minimum abundance of sequences for denoising
|
|
alpha parameter to the vsearch –cluster_unoise command.
default = 2.0.
|
|
at which level to perform denoising; global = by pooling samples,
individual = independently for each sample
(if samples are denoised individually, reducing minsize to 4 may
be more reasonable for higher sensitivity)
|
|
perform chimera removal with uchime3_denovo algoritm
|
|
the abundance skew of chimeric sequences in comparsion with
parental sequences (by default, parents should be at least
16 times more abundant than their chimera)
|
|
number of cores to use for clustering
|
POSTCLUSTERING
Perform OTU post-clustering. Merge co-occurring ‘daughter’ OTUs.
LULU
LULU description from the LULU repository: the purpose of LULU is to reduce the number of erroneous OTUs in OTU tables to achieve more realistic biodiversity metrics. By evaluating the co-occurence patterns of OTUs among samples LULU identifies OTUs that consistently satisfy some user selected criteria for being errors of more abundant OTUs and merges these. It has been shown that curation with LULU consistently result in more realistic diversity metrics.
- Additional information:
table
) and OTU sequences (rep_seqs
) in fasta format (see input examples below).Note
To START, specify working directory under SELECT WORKDIR
, but the file formats do not matter here (just click ‘Next’).
lulu_out
directory:Tooltip |
|
---|---|
|
select OTU/ASV table. If no file is selected, then PipeCraft will
look OTU_table.txt or ASV_table.txt in the working directory.
|
|
select fasta formatted sequence file containing your OTU/ASV reads.
|
|
sets whether a potential error must have lower abundance than the parent
in all samples ‘min’ (default), or if an error just needs to have lower
abundance on average ‘avg’
|
|
set the minimim abundance ratio between a potential error and a
potential parent to be identified as an error
|
|
specify minimum threshold of sequence similarity for considering
any OTU as an error of another
|
|
minimum co-occurrence rate. Default = 0.95 (meaning that 1 in 20 samples
are allowed to have no parent presence)
|
|
use either ‘blastn’ or ‘vsearch’ to generate match list for LULU.
Default is ‘vsearch’ (much faster)
|
|
applies only when ‘vsearch’ is used as ‘match_list_soft’.
Pairwise sequence identity definition (–iddef)
|
|
percent identity cutoff for match list. Excluding pairwise comparisons
with lower sequence identity percentage than specified threshold
|
|
percent query coverage per hit. Excluding pairwise comparisons with
lower sequence coverage than specified threshold
|
|
query strand to search against database. Both = search also reverse complement
|
|
number of cores to use for generating match list for LULU
|
DADA2 collapse ASVs
DADA2 collapseNoMismatch function to collapse identical ASVs; and ASVs filtering based on minimum accepted sequence length (custom R functions).
To START, specify working directory under SELECT WORKDIR
, but the file formats do not matter here (just click ‘Next’).
filtered_table
directory:Setting |
Tooltip |
---|---|
|
select the RDS file (ASV table), output from DADA2 workflow;
usually in ASVs_out.dada2/ASVs_table.denoised-merged.rds
|
|
collapses ASVs that are identical up to shifts or
length variation, i.e. that have no mismatches or internal indels
|
|
discard ASVs from the ASV table that are shorter than specified
value (in base pairs). Value 0 means OFF, no filtering by length
|
|
collapseNoMismatch setting. Default = 20. The minimum overlap of
base pairs between ASV sequences required to collapse them together
|
|
collapseNoMismatch setting. Default = TRUE. Use the vectorized
aligner. Should be turned off if sequences exceed 2kb in length
|
ASSIGN TAXONOMY
Implemented tools for taxonomy annotation:
BLAST (Camacho et al. 2009)
Important
BLAST database needs to be an unzipped fasta file in a separate folder (fasta will be automatically converted to BLAST database files). If converted BLAST database files (.ndb, .nhr, .nin, .not, .nsq, .ntf, .nto) already exist, then just SELECT one of those files as BLAST database in ‘ASSIGN TAXONOMY’ panel.
Note
To START, specify working directory under SELECT WORKDIR
and the sequence files extension
(to look for input OTUs/ASVs fasta file), but the read types (single-end or paired-end) and data format (demultiplexed or multiplexed) does not matter here (just click ‘Next’).
Note
BLAST values filed separator is ‘+’. When pasting the taxonomy results to e.g. Excel, then first denote ‘+’ as as filed separator to align the columns.
Setting |
Tooltip |
---|---|
|
select a database file in fasta format.
Fasta format will be automatically converted to BLAST database
|
|
BLAST search settings according to blastn or megablast
|
|
query strand to search against database. Both = search also reverse complement
|
|
a parameter that describes the number of hits one can expect to see
by chance when searching a database of a particular size.
The lower the e-value the more ‘significant’ the match is
|
|
the size of the initial word that must be matched between the database
and the query sequence
|
|
reward for a match
|
|
penalty for a mismatch
|
|
cost to open a gap
|
|
cost to extend a gap
|
DADA2 classifier
Note
To START, specify working directory under SELECT WORKDIR
and the sequence files extension
(to look for input OTUs/ASVs fasta file), but the read types (single-end or paired-end) and data format (demultiplexed or multiplexed) does not matter here (just click ‘Next’).
Setting |
Tooltip |
---|---|
|
select a reference database fasta file for taxonomy annotation
|
|
the minimum bootstrap confidence for assigning a taxonomic level
|
|
the reverse-complement of each sequences will be used for classification
if it is a better match to the reference sequences than the forward sequence
|
Sequence databases
A (noncomprehensive) list of public databases available for taxonomy annotation
Database |
Version |
Description (click to download) |
---|---|---|
8.3
|
||
138.1
|
||
SILVA 99% |
138.1
|
|
246
|
||
5.1
|
||
DADA2-formatted reference databases |
||
DIAT.BARCODE database |
POSTPROCESSING
Post-processing tools. See this page
Expert-mode (PipeCraft2 console)
Bioinformatic tools used by PipeCraft2 are stored on Dockerhub as Docker images. These images can be used to launch any tool with the Docker CLI to utilize the compiled tools. Especially useful in Windows OS, where majority of implemented modules are not compatible.
See list of docker images with implemented software here
Show a list of all images in your system (using e.g. Expert-mode):
docker images
Download an image if required (from Dockerhub):
docker pull pipecraft/vsearch:2.18
Delete an image
docker rmi pipecraft/vsearch:2.18
Run docker container in your working directory to access the files. Outputs will be generated into the specified working directory. Specify the working directory under the -v flag:
docker run -i --tty -v users/Tom/myFiles/:/Files pipecraft/vsearch:2.18
Once inside the container, move to /Files directory, which represents your working directory in the container; and run analyses
cd Files
vsearch --help
vsearch *--whateversettings*
Exit from the container:
exit