1 Intro

1.1 Inputs

This notebook will download trimmed WGBS FastQs (trimmed on 20220829 (Sam’s Notebook entry)) and align them to the C.viginica genome using Bismark (Krueger and Andrews 2011) and Bowtie2 (Langmead et al. 2018; Langmead and Salzberg 2012). Alignment results will be summarized by MultiQC (Ewels et al. 2016).

1.2 Outputs

The expected outputs will be:

  • *_R1_001.fastp-trim-20220827_bismark_bt2_PE_report.txt: A text file summarizing the alignment input and results. Despite the R1 naming, these reports are based on paired reads; the R1 naming is a quirk of Bismark.

  • *_R1_001.fastp-trim-20220827_bismark_bt2_pe.bam:A BAM alignment. Despite the R1 naming, these BAMs are paired reads; the R1 naming is a quirk of Bismark.

  • bismark_summary.txt: An overall summary report of the alignment process. Essentially, this is all of the individual *report.txt files combined into a single file.

  • multiqc_report.html: A summary report of the alignment results generated by MultiQC, in HTML format.

Due to the large file sizes of BAMS, these cannot be hosted in the ceasmallr GitHub repo. As such these files are available for download here:

2 Create a Bash variables file

This allows usage of Bash variables across R Markdown chunks.

{
echo "#### Assign Variables ####"
echo ""

echo "# Data directories"
echo 'export repo_dir=/home/shared/8TB_HDD_01/sam/gitrepos/ceasmallr'
echo 'export output_dir_top=${repo_dir}/output/02.00-bismark-bowtie2-alignment'
echo 'export trimmed_reads_url="https://gannet.fish.washington.edu/Atumefaciens/20220826-cvir-larvae_zygote-BSseq-fastp_trimming/"'
echo 'export trimmed_fastqs_dir="${output_dir_top}/trimmed-fastqs"'
echo ""

echo "# Input files"
echo 'export bisulfite_genome_url="http://owl.fish.washington.edu/halfshell/genomic-databank/Cvirginica_v300_bisulfite.tar.gz"'
echo 'export bisulfite_genome_gz="Cvirginica_v300_bisulfite.tar.gz"'
echo 'export bisulfite_genome_dir="${repo_dir}/data/Cvirginica_v300"'
echo ""

echo "# Paths to programs"
echo 'export programs_dir="/home/shared"'
echo 'export bismark_dir="${programs_dir}/Bismark-0.24.0"'
echo 'export bowtie2_dir="${programs_dir}/bowtie2-2.4.4-linux-x86_64"'
echo 'export multiqc="/home/sam/programs/mambaforge/bin/multiqc"'
echo 'export samtools_dir="${programs_dir}/samtools-1.12"'
echo ""

echo "# Program settings"
echo 'export bowtie2_min_score="L,0,-0.6"'
echo ""

echo "# Set FastQ filename patterns"
echo "export fastq_pattern='*.fq.gz'"
echo "export R1_fastq_pattern='*_R1_*.fq.gz'"
echo "export R2_fastq_pattern='*_R2_*.fq.gz'"
echo 'export reads_basename="_001.fastp-trim.20220827.fq.gz"'
echo 'export R1_reads_basename="_R1_001.fastp-trim.20220827.fq.gz"'
echo 'export R2_reads_basename="_R2_001.fastp-trim.20220827.fq.gz"'
echo "export trimmed_fastq_pattern='*fastp-trim*.fq.gz'"
echo ""

echo "# Set number of CPUs to use"
echo 'export threads=4'
echo "# Bismark already spawns multiple instances and additional threads are multiplicative."
echo 'export bismark_threads=2' 
echo ""


echo "## Inititalize arrays"
echo 'export fastq_array_R1=()'
echo 'export fastq_array_R2=()'
echo 'export trimmed_fastqs_array=()'
echo 'export R1_names_array=()'
echo 'export R2_names_array=()'
echo ""

echo "# Print formatting"
echo 'export line="--------------------------------------------------------"'
echo ""
} > .bashvars

cat .bashvars
#### Assign Variables ####

# Data directories
export repo_dir=/home/shared/8TB_HDD_01/sam/gitrepos/ceasmallr
export output_dir_top=${repo_dir}/output/02.00-bismark-bowtie2-alignment
export trimmed_reads_url="https://gannet.fish.washington.edu/Atumefaciens/20220826-cvir-larvae_zygote-BSseq-fastp_trimming/"
export trimmed_fastqs_dir="${output_dir_top}/trimmed-fastqs"

# Input files
export bisulfite_genome_url="http://owl.fish.washington.edu/halfshell/genomic-databank/Cvirginica_v300_bisulfite.tar.gz"
export bisulfite_genome_gz="Cvirginica_v300_bisulfite.tar.gz"
export bisulfite_genome_dir="${repo_dir}/data/Cvirginica_v300"

# Paths to programs
export programs_dir="/home/shared"
export bismark_dir="${programs_dir}/Bismark-0.24.0"
export bowtie2_dir="${programs_dir}/bowtie2-2.4.4-linux-x86_64"
export multiqc="/home/sam/programs/mambaforge/bin/multiqc"
export samtools_dir="${programs_dir}/samtools-1.12"

# Program settings
export bowtie2_min_score="L,0,-0.6"

# Set FastQ filename patterns
export fastq_pattern='*.fq.gz'
export R1_fastq_pattern='*_R1_*.fq.gz'
export R2_fastq_pattern='*_R2_*.fq.gz'
export reads_basename="_001.fastp-trim.20220827.fq.gz"
export R1_reads_basename="_R1_001.fastp-trim.20220827.fq.gz"
export R2_reads_basename="_R2_001.fastp-trim.20220827.fq.gz"
export trimmed_fastq_pattern='*fastp-trim*.fq.gz'

# Set number of CPUs to use
export threads=4
# Bismark already spawns multiple instances and additional threads are multiplicative.
export bismark_threads=2

## Inititalize arrays
export fastq_array_R1=()
export fastq_array_R2=()
export trimmed_fastqs_array=()
export R1_names_array=()
export R2_names_array=()

# Print formatting
export line="--------------------------------------------------------"

3 Download bisulfite genome

# Load bash variables into memory
source .bashvars

cd "${repo_dir}"/data

wget --quiet \
--continue \
"${bisulfite_genome_url}"

ls -ltrh
total 2.6G
-rw-r--r-- 1 sam sam 662M Jun  7  2018 Cvirginica_v300.fa
drwxr-xr-x 3 sam sam 4.0K Mar  7  2019 Cvirginica_v300
-rw-r--r-- 1 sam sam 2.0G Mar  7  2019 Cvirginica_v300_bisulfite.tar.gz
-rw-r--r-- 1 sam sam  307 Oct 23 12:29 README.md
-rw-r--r-- 1 sam sam 2.6K Oct 23 12:29 L18_larvae_meta.csv
-rw-r--r-- 1 sam sam  398 Oct 23 13:38 Cvirginica_v300.fa.fai

3.1 Unpack bisulfite genome

# Load bash variables into memory
source .bashvars

cd "${repo_dir}"/data

if [ ! -d "${bisulfite_genome_dir}" ]; then
  tar -xzf "${bisulfite_genome_gz}"
fi

tree "${bisulfite_genome_dir}"
/home/shared/8TB_HDD_01/sam/gitrepos/ceasmallr/data/Cvirginica_v300
├── Bisulfite_Genome
│   ├── CT_conversion
│   │   ├── BS_CT.1.bt2
│   │   ├── BS_CT.2.bt2
│   │   ├── BS_CT.3.bt2
│   │   ├── BS_CT.4.bt2
│   │   ├── BS_CT.rev.1.bt2
│   │   ├── BS_CT.rev.2.bt2
│   │   └── genome_mfa.CT_conversion.fa
│   └── GA_conversion
│       ├── BS_GA.1.bt2
│       ├── BS_GA.2.bt2
│       ├── BS_GA.3.bt2
│       ├── BS_GA.4.bt2
│       ├── BS_GA.rev.1.bt2
│       ├── BS_GA.rev.2.bt2
│       └── genome_mfa.GA_conversion.fa
├── Cvirginica_v300.fa
├── Cvirginica_v300.fa.fai
└── readme.txt

3 directories, 17 files

4 Download trimmed RNA-seq reads

Reads are downloaded from https://gannet.fish.washington.edu/Atumefaciens/20220826-cvir-larvae_zygote-BSseq-fastp_trimming/

The --cut-dirs 2 command cuts the preceding directory structure (i.e. Atumefaciens/20220826-cvir-larvae_zygote-BSseq-fastp_trimming/) so that we just end up with the reads.

# Load bash variables into memory
source .bashvars

# Make output directory if it doesn't exist
mkdir --parents ${trimmed_fastqs_dir}

# Run wget to retrieve FastQs and MD5 files
wget \
--directory-prefix ${trimmed_fastqs_dir} \
--no-check-certificate \
--continue \
--cut-dirs 2 \
--no-host-directories \
--no-parent \
--quiet \
--accept="${trimmed_fastq_pattern}, trimmed-fastq-checksums.md5" \
${trimmed_reads_url}

ls -lh "${trimmed_fastqs_dir}"
total 469G
-rw-r--r-- 1 sam sam 1.7G Aug 27  2022 CF01-CM01-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 1.8G Aug 27  2022 CF01-CM01-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.8G Aug 27  2022 CF01-CM02-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.1G Aug 27  2022 CF01-CM02-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam  11G Aug 27  2022 CF02-CM02-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam  11G Aug 27  2022 CF02-CM02-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.6G Aug 27  2022 CF03-CM03-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.7G Aug 27  2022 CF03-CM03-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.4G Aug 27  2022 CF03-CM04-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.7G Aug 27  2022 CF03-CM04-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.3G Aug 27  2022 CF03-CM05-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.5G Aug 27  2022 CF03-CM05-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.8G Aug 27  2022 CF04-CM04-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.0G Aug 27  2022 CF04-CM04-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 8.7G Aug 27  2022 CF05-CM02-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 9.1G Aug 27  2022 CF05-CM02-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.5G Aug 27  2022 CF05-CM05-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.7G Aug 27  2022 CF05-CM05-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.4G Aug 27  2022 CF06-CM01-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.6G Aug 27  2022 CF06-CM01-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.0G Aug 27  2022 CF06-CM02-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.3G Aug 27  2022 CF06-CM02-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam  11G Aug 27  2022 CF07-CM02-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam  12G Aug 27  2022 CF07-CM02-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.5G Aug 27  2022 CF08-CM03-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.7G Aug 27  2022 CF08-CM03-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 3.1G Aug 27  2022 CF08-CM04-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 3.2G Aug 27  2022 CF08-CM04-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 4.8G Aug 27  2022 CF08-CM05-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 4.9G Aug 27  2022 CF08-CM05-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.4G Aug 27  2022 EF01-EM01-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.7G Aug 27  2022 EF01-EM01-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.0G Aug 27  2022 EF02-EM02-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.2G Aug 27  2022 EF02-EM02-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.6G Aug 27  2022 EF03-EM03-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.8G Aug 27  2022 EF03-EM03-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.1G Aug 27  2022 EF03-EM04-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.3G Aug 27  2022 EF03-EM04-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.4G Aug 27  2022 EF03-EM05-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 8.0G Aug 27  2022 EF03-EM05-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.5G Aug 27  2022 EF04-EM04-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.8G Aug 27  2022 EF04-EM04-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.8G Aug 27  2022 EF04-EM05-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 8.2G Aug 27  2022 EF04-EM05-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 5.9G Aug 27  2022 EF05-EM01-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.1G Aug 27  2022 EF05-EM01-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.4G Aug 27  2022 EF05-EM05-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.6G Aug 27  2022 EF05-EM05-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam  12G Aug 27  2022 EF05-EM06-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam  13G Aug 27  2022 EF05-EM06-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.9G Aug 27  2022 EF06-EM01-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 8.3G Aug 27  2022 EF06-EM01-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.1G Aug 27  2022 EF06-EM02-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.4G Aug 27  2022 EF06-EM02-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 8.2G Aug 27  2022 EF06-EM06-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 8.4G Aug 27  2022 EF06-EM06-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam  13G Aug 27  2022 EF07-EM01-Zygote_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam  14G Aug 27  2022 EF07-EM01-Zygote_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.0G Aug 27  2022 EF07-EM03-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 7.3G Aug 27  2022 EF07-EM03-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 8.2G Aug 27  2022 EF08-EM03-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 8.5G Aug 27  2022 EF08-EM03-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 5.8G Aug 28  2022 EF08-EM04-Larvae_R1_001.fastp-trim.20220827.fq.gz
-rw-r--r-- 1 sam sam 6.0G Aug 28  2022 EF08-EM04-Larvae_R2_001.fastp-trim.20220827.fq.gz
-rw------- 1 sam sam  36K Nov 13 11:06 index.html.tmp
-rw-r--r-- 1 sam sam 5.3K Aug 28  2022 trimmed-fastq-checksums.md5

4.1 Verify trimmed read checksums

# Load bash variables into memory
source .bashvars

cd "${trimmed_fastqs_dir}"

# Checksums file contains other files, so this just looks for the sRNAseq files.
for file in *.md5
do
  md5sum --check "${file}"
done
CF01-CM01-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
CF01-CM01-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
CF01-CM02-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
CF01-CM02-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
CF02-CM02-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
CF02-CM02-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
CF03-CM03-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
CF03-CM03-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
CF03-CM04-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
CF03-CM04-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
CF03-CM05-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
CF03-CM05-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
CF04-CM04-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
CF04-CM04-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
CF05-CM02-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
CF05-CM02-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
CF05-CM05-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
CF05-CM05-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
CF06-CM01-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
CF06-CM01-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
CF06-CM02-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
CF06-CM02-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
CF07-CM02-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
CF07-CM02-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
CF08-CM03-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
CF08-CM03-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
CF08-CM04-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
CF08-CM04-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
CF08-CM05-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
CF08-CM05-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
EF01-EM01-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
EF01-EM01-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
EF02-EM02-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
EF02-EM02-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
EF03-EM03-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
EF03-EM03-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
EF03-EM04-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
EF03-EM04-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
EF03-EM05-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
EF03-EM05-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
EF04-EM04-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
EF04-EM04-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
EF04-EM05-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
EF04-EM05-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
EF05-EM01-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
EF05-EM01-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
EF05-EM05-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
EF05-EM05-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
EF05-EM06-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
EF05-EM06-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
EF06-EM01-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
EF06-EM01-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
EF06-EM02-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
EF06-EM02-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
EF06-EM06-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
EF06-EM06-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
EF07-EM01-Zygote_R1_001.fastp-trim.20220827.fq.gz: OK
EF07-EM01-Zygote_R2_001.fastp-trim.20220827.fq.gz: OK
EF07-EM03-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
EF07-EM03-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
EF08-EM03-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
EF08-EM03-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK
EF08-EM04-Larvae_R1_001.fastp-trim.20220827.fq.gz: OK
EF08-EM04-Larvae_R2_001.fastp-trim.20220827.fq.gz: OK

5 Alignment

# Load bash variables into memory
source .bashvars

cd "${trimmed_fastqs_dir}"

find ${trimmed_fastqs_dir}/${R1_fastq_pattern} \
| xargs basename -s "${R1_reads_basename}" \
| xargs -I {} ${bismark_dir}/bismark \
--path_to_bowtie2 ${bowtie2_dir} \
--genome ${bisulfite_genome_dir} \
--score_min "${bowtie2_min_score}" \
--parallel "${bismark_threads}" \
--non_directional \
--samtools_path "${samtools_dir}" \
--gzip \
-p "${threads}" \
-1 ${trimmed_fastqs_dir}/{}"${R1_reads_basename}" \
-2 ${trimmed_fastqs_dir}/{}"${R2_reads_basename}" \
--output_dir "${output_dir_top}" \
2> "${output_dir_top}"/bismark_summary.txt

5.1 List outputs

# Load bash variables into memory
source .bashvars

cd "${output_dir_top}"

ls -lh
total 184G
-rw-r--r-- 1 sam sam 1.2M Nov  6 15:20 bismark_summary.txt
-rw-r--r-- 1 sam sam 644M Oct 29 18:59 CF01-CM01-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Oct 29 18:59 CF01-CM01-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 7.9G Oct 30 00:16 CF01-CM02-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Oct 30 00:16 CF01-CM02-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 4.1G Oct 30 06:15 CF02-CM02-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Oct 30 06:15 CF02-CM02-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 4.9G Oct 30 10:37 CF03-CM03-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Oct 30 10:37 CF03-CM03-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam  10G Oct 30 16:43 CF03-CM04-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Oct 30 16:43 CF03-CM04-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 7.1G Oct 30 21:29 CF03-CM05-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Oct 30 21:29 CF03-CM05-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 5.4G Oct 31 02:01 CF04-CM04-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Oct 31 02:01 CF04-CM04-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam  11G Oct 31 08:52 CF05-CM02-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Oct 31 08:52 CF05-CM02-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 6.1G Oct 31 13:54 CF05-CM05-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Oct 31 13:54 CF05-CM05-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 5.3G Oct 31 18:45 CF06-CM01-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Oct 31 18:45 CF06-CM01-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 4.2G Oct 31 22:41 CF06-CM02-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Oct 31 22:41 CF06-CM02-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 3.7G Nov  1 04:50 CF07-CM02-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  1 04:50 CF07-CM02-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 4.1G Nov  1 09:30 CF08-CM03-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  1 09:30 CF08-CM03-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam  98M Nov  1 10:59 CF08-CM04-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  1 10:59 CF08-CM04-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 360M Nov  1 13:56 CF08-CM05-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  1 13:56 CF08-CM05-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-rw-r-- 1 sam sam 6.3K Nov  6 19:46 checksums.md5
-rw-r--r-- 1 sam sam 4.7G Nov  1 18:09 EF01-EM01-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  1 18:09 EF01-EM01-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 5.0G Nov  1 22:50 EF02-EM02-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  1 22:50 EF02-EM02-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 3.3G Nov  2 02:53 EF03-EM03-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  2 02:53 EF03-EM03-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 7.0G Nov  2 08:02 EF03-EM04-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  2 08:02 EF03-EM04-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 7.0G Nov  2 13:24 EF03-EM05-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  2 13:24 EF03-EM05-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 1.8G Nov  2 22:18 EF04-EM04-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  2 22:18 EF04-EM04-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 9.5G Nov  3 12:42 EF04-EM05-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  3 12:42 EF04-EM05-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 5.6G Nov  3 18:16 EF05-EM01-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  3 18:16 EF05-EM01-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 1.1G Nov  3 22:02 EF05-EM05-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  3 22:02 EF05-EM05-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam  15G Nov  4 10:27 EF05-EM06-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  4 10:27 EF05-EM06-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 9.2G Nov  4 16:52 EF06-EM01-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  4 16:52 EF06-EM01-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 7.1G Nov  4 22:24 EF06-EM02-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  4 22:24 EF06-EM02-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 6.3G Nov  5 06:56 EF06-EM06-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  5 06:56 EF06-EM06-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 3.5G Nov  5 17:19 EF07-EM01-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  5 17:19 EF07-EM01-Zygote_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 7.7G Nov  6 03:05 EF07-EM03-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  6 03:05 EF07-EM03-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam  11G Nov  6 10:19 EF08-EM03-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  6 10:20 EF08-EM03-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
-rw-r--r-- 1 sam sam 6.7G Nov  6 15:20 EF08-EM04-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_pe.bam
-rw-r--r-- 1 sam sam 2.1K Nov  6 15:20 EF08-EM04-Larvae_R1_001.fastp-trim.20220827_bismark_bt2_PE_report.txt
drwxr-xr-x 2 sam sam 4.0K Nov  9 12:15 multiqc_data
-rw-r--r-- 1 sam sam 1.1M Nov  9 12:15 multiqc_report.html
drwxr-xr-x 2 sam sam  12K Nov 13 10:26 trimmed-fastqs

6 MultiQC

# Load bash variables into memory
source .bashvars

cd "${output_dir_top}"

${multiqc} .

7 Checksums

# Load bash variables into memory
source .bashvars

cd "${output_dir_top}"

for file in *; do
  if [ "${file}" != "checksums.md5" ]; then
    md5sum "${file}" >> checksums.md5
  fi
done

Citations

Ewels, Philip, Måns Magnusson, Sverker Lundin, and Max Käller. 2016. “MultiQC: Summarize Analysis Results for Multiple Tools and Samples in a Single Report.” Bioinformatics 32 (19): 3047–48. https://doi.org/10.1093/bioinformatics/btw354.
Krueger, Felix, and Simon R. Andrews. 2011. “Bismark: A Flexible Aligner and Methylation Caller for Bisulfite-Seq Applications.” Bioinformatics 27 (11): 1571–72. https://doi.org/10.1093/bioinformatics/btr167.
Langmead, Ben, and Steven L Salzberg. 2012. “Fast Gapped-Read Alignment with Bowtie 2.” Nature Methods 9 (4): 357–59. https://doi.org/10.1038/nmeth.1923.
Langmead, Ben, Christopher Wilks, Valentin Antonescu, and Rone Charles. 2018. “Scaling Read Aligners to Hundreds of Threads on General-Purpose Processors.” Edited by John Hancock. Bioinformatics 35 (3): 421–32. https://doi.org/10.1093/bioinformatics/bty648.
LS0tCnRpdGxlOiAiMDIuMDAtYmlzbWFyay1ib3d0aWUyLWFsaWdubWVudCIKYXV0aG9yOiAiU2FtIFdoaXRlIgpkYXRlOiAiMjAyNC0xMC0yMyIKb3V0cHV0OiAKICBib29rZG93bjo6aHRtbF9kb2N1bWVudDI6CiAgICB0aGVtZTogY29zbW8KICAgIHRvYzogdHJ1ZQogICAgdG9jX2Zsb2F0OiB0cnVlCiAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUKICAgIGNvZGVfZm9sZGluZzogc2hvdwogICAgY29kZV9kb3dubG9hZDogdHJ1ZQogIGdpdGh1Yl9kb2N1bWVudDoKICAgIHRvYzogdHJ1ZQogICAgbnVtYmVyX3NlY3Rpb25zOiB0cnVlCiAgaHRtbF9kb2N1bWVudDoKICAgIHRoZW1lOiBjb3NtbwogICAgdG9jOiB0cnVlCiAgICB0b2NfZmxvYXQ6IHRydWUKICAgIG51bWJlcl9zZWN0aW9uczogdHJ1ZQogICAgY29kZV9mb2xkaW5nOiBzaG93CiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCmJpYmxpb2dyYXBoeTogcmVmZXJlbmNlcy5iaWIKLS0tCiMgSW50cm8KCiMjIElucHV0cwpUaGlzIG5vdGVib29rIHdpbGwgZG93bmxvYWQgdHJpbW1lZCBXR0JTIEZhc3RRcyAodHJpbW1lZCBvbiBbMjAyMjA4MjldKGh0dHBzOi8vcm9iZXJ0c2xhYi5naXRodWIuaW8vc2Ftcy1ub3RlYm9vay9wb3N0cy8yMDIyLzIwMjItMDgtMjktRmFzdFEtVHJpbW1pbmctYW5kLVFDLS0tQy52aXJnaW5pY2EtTGFydmFsLUJTLXNlcS1EYXRhLWZyb20tTG90dGVyaG9zLUxhYi1hbmQtUGFydC1vZi1DRUFCSUdSLVByb2plY3QtVXNpbmctZmFzdHAtb24tTW94L2luZGV4Lmh0bWwpIChTYW0ncyBOb3RlYm9vayBlbnRyeSkpIGFuZCBhbGlnbiB0aGVtIHRvIHRoZSAqQy52aWdpbmljYSogZ2Vub21lIHVzaW5nIFtCaXNtYXJrXShodHRwczovL2ZlbGl4a3J1ZWdlci5naXRodWIuaW8vQmlzbWFyay8pIFtAa3J1ZWdlcjIwMTFdIGFuZCBbQm93dGllMl0oaHR0cHM6Ly9naXRodWIuY29tL0JlbkxhbmdtZWFkL2Jvd3RpZTIpIFtAbGFuZ21lYWQyMDE4OyBAbGFuZ21lYWQyMDEyXS4gQWxpZ25tZW50IHJlc3VsdHMgd2lsbCBiZSBzdW1tYXJpemVkIGJ5IFtNdWx0aVFDXShodHRwczovL2dpdGh1Yi5jb20vTXVsdGlRQy9NdWx0aVFDKSBbQGV3ZWxzMjAxNl0uCgojIyBPdXRwdXRzCgpUaGUgZXhwZWN0ZWQgb3V0cHV0cyB3aWxsIGJlOgoKLSAgIGAqX1IxXzAwMS5mYXN0cC10cmltLTIwMjIwODI3X2Jpc21hcmtfYnQyX1BFX3JlcG9ydC50eHRgOiBBIHRleHQgZmlsZSBzdW1tYXJpemluZyB0aGUgYWxpZ25tZW50IGlucHV0IGFuZCByZXN1bHRzLiBEZXNwaXRlIHRoZSBgUjFgIG5hbWluZywgdGhlc2UgcmVwb3J0cyBhcmUgYmFzZWQgb24gKnBhaXJlZCByZWFkcyo7IHRoZSBgUjFgIG5hbWluZyBpcyBhIHF1aXJrIG9mIFtCaXNtYXJrXShodHRwczovL2ZlbGl4a3J1ZWdlci5naXRodWIuaW8vQmlzbWFyay8pLgoKLSAgIGAqX1IxXzAwMS5mYXN0cC10cmltLTIwMjIwODI3X2Jpc21hcmtfYnQyX3BlLmJhbWA6QSBCQU0gYWxpZ25tZW50LiBEZXNwaXRlIHRoZSBgUjFgIG5hbWluZywgdGhlc2UgQkFNcyBhcmUgKnBhaXJlZCByZWFkcyo7IHRoZSBgUjFgIG5hbWluZyBpcyBhIHF1aXJrIG9mIFtCaXNtYXJrXShodHRwczovL2ZlbGl4a3J1ZWdlci5naXRodWIuaW8vQmlzbWFyay8pLgoKLSAgIGBiaXNtYXJrX3N1bW1hcnkudHh0YDogQW4gb3ZlcmFsbCBzdW1tYXJ5IHJlcG9ydCBvZiB0aGUgYWxpZ25tZW50IHByb2Nlc3MuIEVzc2VudGlhbGx5LCB0aGlzIGlzIGFsbCBvZiB0aGUgaW5kaXZpZHVhbCBgKnJlcG9ydC50eHRgIGZpbGVzIGNvbWJpbmVkIGludG8gYSBzaW5nbGUgZmlsZS4KCi0gICBgbXVsdGlxY19yZXBvcnQuaHRtbGA6IEEgc3VtbWFyeSByZXBvcnQgb2YgdGhlIGFsaWdubWVudCByZXN1bHRzIGdlbmVyYXRlZCBieSBbTXVsdGlRQ10oaHR0cHM6Ly9naXRodWIuY29tL011bHRpUUMvTXVsdGlRQyksIGluIEhUTUwgZm9ybWF0LgoKRHVlIHRvIHRoZSBsYXJnZSBmaWxlIHNpemVzIG9mIEJBTVMsIHRoZXNlIGNhbm5vdCBiZSBob3N0ZWQgaW4gdGhlIFtjZWFzbWFsbHIgR2l0SHViIHJlcG9dKGh0dHBzOi8vZ2l0aHViLmNvbS9zcjMyMC9jZWFzbWFsbHIpLiBBcyBzdWNoIHRoZXNlIGZpbGVzIGFyZSBhdmFpbGFibGUgZm9yIGRvd25sb2FkIGhlcmU6CgotIFtodHRwczovL2dhbm5ldC5maXNoLndhc2hpbmd0b24uZWR1L0F0dW1lZmFjaWVucy9naXRyZXBvcy9jZWFzbWFsbHIvb3V0cHV0LzAyLjAwLWJpc21hcmstYm93dGllMi1hbGlnbm1lbnQvXShodHRwczovL2dhbm5ldC5maXNoLndhc2hpbmd0b24uZWR1L0F0dW1lZmFjaWVucy9naXRyZXBvcy9jZWFzbWFsbHIvb3V0cHV0LzAyLjAwLWJpc21hcmstYm93dGllMi1hbGlnbm1lbnQvKQoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmxpYnJhcnkoa25pdHIpCmtuaXRyOjpvcHRzX2NodW5rJHNldCgKICBlY2hvID0gVFJVRSwgICAgICAgICAjIERpc3BsYXkgY29kZSBjaHVua3MKICBldmFsID0gRkFMU0UsICAgICAgICAjIEV2YWx1YXRlIGNvZGUgY2h1bmtzCiAgd2FybmluZyA9IEZBTFNFLCAgICAgIyBIaWRlIHdhcm5pbmdzCiAgbWVzc2FnZSA9IEZBTFNFLCAgICAgIyBIaWRlIG1lc3NhZ2VzCiAgY29tbWVudCA9ICIiICAgICAgICAgIyBQcmV2ZW50cyBhcHBlbmRpbmcgJyMjJyB0byBiZWdpbm5pbmcgb2YgbGluZXMgaW4gY29kZSBvdXRwdXQKKQpgYGAKCiMgQ3JlYXRlIGEgQmFzaCB2YXJpYWJsZXMgZmlsZQoKVGhpcyBhbGxvd3MgdXNhZ2Ugb2YgQmFzaCB2YXJpYWJsZXMgYWNyb3NzIFIgTWFya2Rvd24gY2h1bmtzLgoKYGBge3Igc2F2ZS1iYXNoLXZhcmlhYmxlcy10by1ydmFycy1maWxlLCBlbmdpbmU9J2Jhc2gnLCBldmFsPVRSVUV9CnsKZWNobyAiIyMjIyBBc3NpZ24gVmFyaWFibGVzICMjIyMiCmVjaG8gIiIKCmVjaG8gIiMgRGF0YSBkaXJlY3RvcmllcyIKZWNobyAnZXhwb3J0IHJlcG9fZGlyPS9ob21lL3NoYXJlZC84VEJfSEREXzAxL3NhbS9naXRyZXBvcy9jZWFzbWFsbHInCmVjaG8gJ2V4cG9ydCBvdXRwdXRfZGlyX3RvcD0ke3JlcG9fZGlyfS9vdXRwdXQvMDIuMDAtYmlzbWFyay1ib3d0aWUyLWFsaWdubWVudCcKZWNobyAnZXhwb3J0IHRyaW1tZWRfcmVhZHNfdXJsPSJodHRwczovL2dhbm5ldC5maXNoLndhc2hpbmd0b24uZWR1L0F0dW1lZmFjaWVucy8yMDIyMDgyNi1jdmlyLWxhcnZhZV96eWdvdGUtQlNzZXEtZmFzdHBfdHJpbW1pbmcvIicKZWNobyAnZXhwb3J0IHRyaW1tZWRfZmFzdHFzX2Rpcj0iJHtvdXRwdXRfZGlyX3RvcH0vdHJpbW1lZC1mYXN0cXMiJwplY2hvICIiCgplY2hvICIjIElucHV0IGZpbGVzIgplY2hvICdleHBvcnQgYmlzdWxmaXRlX2dlbm9tZV91cmw9Imh0dHA6Ly9vd2wuZmlzaC53YXNoaW5ndG9uLmVkdS9oYWxmc2hlbGwvZ2Vub21pYy1kYXRhYmFuay9DdmlyZ2luaWNhX3YzMDBfYmlzdWxmaXRlLnRhci5neiInCmVjaG8gJ2V4cG9ydCBiaXN1bGZpdGVfZ2Vub21lX2d6PSJDdmlyZ2luaWNhX3YzMDBfYmlzdWxmaXRlLnRhci5neiInCmVjaG8gJ2V4cG9ydCBiaXN1bGZpdGVfZ2Vub21lX2Rpcj0iJHtyZXBvX2Rpcn0vZGF0YS9DdmlyZ2luaWNhX3YzMDAiJwplY2hvICIiCgplY2hvICIjIFBhdGhzIHRvIHByb2dyYW1zIgplY2hvICdleHBvcnQgcHJvZ3JhbXNfZGlyPSIvaG9tZS9zaGFyZWQiJwplY2hvICdleHBvcnQgYmlzbWFya19kaXI9IiR7cHJvZ3JhbXNfZGlyfS9CaXNtYXJrLTAuMjQuMCInCmVjaG8gJ2V4cG9ydCBib3d0aWUyX2Rpcj0iJHtwcm9ncmFtc19kaXJ9L2Jvd3RpZTItMi40LjQtbGludXgteDg2XzY0IicKZWNobyAnZXhwb3J0IG11bHRpcWM9Ii9ob21lL3NhbS9wcm9ncmFtcy9tYW1iYWZvcmdlL2Jpbi9tdWx0aXFjIicKZWNobyAnZXhwb3J0IHNhbXRvb2xzX2Rpcj0iJHtwcm9ncmFtc19kaXJ9L3NhbXRvb2xzLTEuMTIiJwplY2hvICIiCgplY2hvICIjIFByb2dyYW0gc2V0dGluZ3MiCmVjaG8gJ2V4cG9ydCBib3d0aWUyX21pbl9zY29yZT0iTCwwLC0wLjYiJwplY2hvICIiCgplY2hvICIjIFNldCBGYXN0USBmaWxlbmFtZSBwYXR0ZXJucyIKZWNobyAiZXhwb3J0IGZhc3RxX3BhdHRlcm49JyouZnEuZ3onIgplY2hvICJleHBvcnQgUjFfZmFzdHFfcGF0dGVybj0nKl9SMV8qLmZxLmd6JyIKZWNobyAiZXhwb3J0IFIyX2Zhc3RxX3BhdHRlcm49JypfUjJfKi5mcS5neiciCmVjaG8gJ2V4cG9ydCByZWFkc19iYXNlbmFtZT0iXzAwMS5mYXN0cC10cmltLjIwMjIwODI3LmZxLmd6IicKZWNobyAnZXhwb3J0IFIxX3JlYWRzX2Jhc2VuYW1lPSJfUjFfMDAxLmZhc3RwLXRyaW0uMjAyMjA4MjcuZnEuZ3oiJwplY2hvICdleHBvcnQgUjJfcmVhZHNfYmFzZW5hbWU9Il9SMl8wMDEuZmFzdHAtdHJpbS4yMDIyMDgyNy5mcS5neiInCmVjaG8gImV4cG9ydCB0cmltbWVkX2Zhc3RxX3BhdHRlcm49JypmYXN0cC10cmltKi5mcS5neiciCmVjaG8gIiIKCmVjaG8gIiMgU2V0IG51bWJlciBvZiBDUFVzIHRvIHVzZSIKZWNobyAnZXhwb3J0IHRocmVhZHM9NCcKZWNobyAiIyBCaXNtYXJrIGFscmVhZHkgc3Bhd25zIG11bHRpcGxlIGluc3RhbmNlcyBhbmQgYWRkaXRpb25hbCB0aHJlYWRzIGFyZSBtdWx0aXBsaWNhdGl2ZS4iCmVjaG8gJ2V4cG9ydCBiaXNtYXJrX3RocmVhZHM9MicgCmVjaG8gIiIKCgplY2hvICIjIyBJbml0aXRhbGl6ZSBhcnJheXMiCmVjaG8gJ2V4cG9ydCBmYXN0cV9hcnJheV9SMT0oKScKZWNobyAnZXhwb3J0IGZhc3RxX2FycmF5X1IyPSgpJwplY2hvICdleHBvcnQgdHJpbW1lZF9mYXN0cXNfYXJyYXk9KCknCmVjaG8gJ2V4cG9ydCBSMV9uYW1lc19hcnJheT0oKScKZWNobyAnZXhwb3J0IFIyX25hbWVzX2FycmF5PSgpJwplY2hvICIiCgplY2hvICIjIFByaW50IGZvcm1hdHRpbmciCmVjaG8gJ2V4cG9ydCBsaW5lPSItLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSInCmVjaG8gIiIKfSA+IC5iYXNodmFycwoKY2F0IC5iYXNodmFycwpgYGAKCiMgRG93bmxvYWQgYmlzdWxmaXRlIGdlbm9tZQoKYGBge2Jhc2ggZG93bmxvYWQtYmlzdWxmaXRlLWdlbm9tZSwgZW5naW5lPSdiYXNoJywgZXZhbD1UUlVFfQojIExvYWQgYmFzaCB2YXJpYWJsZXMgaW50byBtZW1vcnkKc291cmNlIC5iYXNodmFycwoKY2QgIiR7cmVwb19kaXJ9Ii9kYXRhCgp3Z2V0IC0tcXVpZXQgXAotLWNvbnRpbnVlIFwKIiR7YmlzdWxmaXRlX2dlbm9tZV91cmx9IgoKbHMgLWx0cmgKYGBgCgojIyBVbnBhY2sgYmlzdWxmaXRlIGdlbm9tZQoKYGBge2Jhc2ggdW5wYWNrLWJpc3VsZml0ZS1nZW5vbWUsIGVuZ2luZT0nYmFzaCcsIGV2YWw9VFJVRX0KIyBMb2FkIGJhc2ggdmFyaWFibGVzIGludG8gbWVtb3J5CnNvdXJjZSAuYmFzaHZhcnMKCmNkICIke3JlcG9fZGlyfSIvZGF0YQoKaWYgWyAhIC1kICIke2Jpc3VsZml0ZV9nZW5vbWVfZGlyfSIgXTsgdGhlbgogIHRhciAteHpmICIke2Jpc3VsZml0ZV9nZW5vbWVfZ3p9IgpmaQoKdHJlZSAiJHtiaXN1bGZpdGVfZ2Vub21lX2Rpcn0iCmBgYAoKIyBEb3dubG9hZCB0cmltbWVkIFJOQS1zZXEgcmVhZHMKClJlYWRzIGFyZSBkb3dubG9hZGVkIGZyb20gPGh0dHBzOi8vZ2FubmV0LmZpc2gud2FzaGluZ3Rvbi5lZHUvQXR1bWVmYWNpZW5zLzIwMjIwODI2LWN2aXItbGFydmFlX3p5Z290ZS1CU3NlcS1mYXN0cF90cmltbWluZy8+CgpUaGUgYC0tY3V0LWRpcnMgMmAgY29tbWFuZCBjdXRzIHRoZSBwcmVjZWRpbmcgZGlyZWN0b3J5IHN0cnVjdHVyZSAoaS5lLiBgQXR1bWVmYWNpZW5zLzIwMjIwODI2LWN2aXItbGFydmFlX3p5Z290ZS1CU3NlcS1mYXN0cF90cmltbWluZy9gKSBzbyB0aGF0IHdlIGp1c3QgZW5kIHVwIHdpdGggdGhlIHJlYWRzLgoKYGBge2Jhc2ggZG93bmxvYWQtdHJpbW1lZC1yZWFkcywgZW5naW5lPSdiYXNoJywgZXZhbD1UUlVFfQojIExvYWQgYmFzaCB2YXJpYWJsZXMgaW50byBtZW1vcnkKc291cmNlIC5iYXNodmFycwoKIyBNYWtlIG91dHB1dCBkaXJlY3RvcnkgaWYgaXQgZG9lc24ndCBleGlzdApta2RpciAtLXBhcmVudHMgJHt0cmltbWVkX2Zhc3Rxc19kaXJ9CgojIFJ1biB3Z2V0IHRvIHJldHJpZXZlIEZhc3RRcyBhbmQgTUQ1IGZpbGVzCndnZXQgXAotLWRpcmVjdG9yeS1wcmVmaXggJHt0cmltbWVkX2Zhc3Rxc19kaXJ9IFwKLS1uby1jaGVjay1jZXJ0aWZpY2F0ZSBcCi0tY29udGludWUgXAotLWN1dC1kaXJzIDIgXAotLW5vLWhvc3QtZGlyZWN0b3JpZXMgXAotLW5vLXBhcmVudCBcCi0tcXVpZXQgXAotLWFjY2VwdD0iJHt0cmltbWVkX2Zhc3RxX3BhdHRlcm59LCB0cmltbWVkLWZhc3RxLWNoZWNrc3Vtcy5tZDUiIFwKJHt0cmltbWVkX3JlYWRzX3VybH0KCmxzIC1saCAiJHt0cmltbWVkX2Zhc3Rxc19kaXJ9IgpgYGAKCiMjIFZlcmlmeSB0cmltbWVkIHJlYWQgY2hlY2tzdW1zCgpgYGB7YmFzaCB2ZXJpZnktdHJpbW1lZC1yZWFkLWNoZWNrc3VtcywgZW5naW5lPSdiYXNoJywgZXZhbD1UUlVFfQojIExvYWQgYmFzaCB2YXJpYWJsZXMgaW50byBtZW1vcnkKc291cmNlIC5iYXNodmFycwoKY2QgIiR7dHJpbW1lZF9mYXN0cXNfZGlyfSIKCiMgQ2hlY2tzdW1zIGZpbGUgY29udGFpbnMgb3RoZXIgZmlsZXMsIHNvIHRoaXMganVzdCBsb29rcyBmb3IgdGhlIHNSTkFzZXEgZmlsZXMuCmZvciBmaWxlIGluICoubWQ1CmRvCiAgbWQ1c3VtIC0tY2hlY2sgIiR7ZmlsZX0iCmRvbmUKYGBgCgojIEFsaWdubWVudAoKYGBge2Jhc2ggYWxpZ25tZW50LCBlbmdpbmU9J2Jhc2gnLCBldmFsPUZBTFNFfQojIExvYWQgYmFzaCB2YXJpYWJsZXMgaW50byBtZW1vcnkKc291cmNlIC5iYXNodmFycwoKY2QgIiR7dHJpbW1lZF9mYXN0cXNfZGlyfSIKCmZpbmQgJHt0cmltbWVkX2Zhc3Rxc19kaXJ9LyR7UjFfZmFzdHFfcGF0dGVybn0gXAp8IHhhcmdzIGJhc2VuYW1lIC1zICIke1IxX3JlYWRzX2Jhc2VuYW1lfSIgXAp8IHhhcmdzIC1JIHt9ICR7YmlzbWFya19kaXJ9L2Jpc21hcmsgXAotLXBhdGhfdG9fYm93dGllMiAke2Jvd3RpZTJfZGlyfSBcCi0tZ2Vub21lICR7YmlzdWxmaXRlX2dlbm9tZV9kaXJ9IFwKLS1zY29yZV9taW4gIiR7Ym93dGllMl9taW5fc2NvcmV9IiBcCi0tcGFyYWxsZWwgIiR7YmlzbWFya190aHJlYWRzfSIgXAotLW5vbl9kaXJlY3Rpb25hbCBcCi0tc2FtdG9vbHNfcGF0aCAiJHtzYW10b29sc19kaXJ9IiBcCi0tZ3ppcCBcCi1wICIke3RocmVhZHN9IiBcCi0xICR7dHJpbW1lZF9mYXN0cXNfZGlyfS97fSIke1IxX3JlYWRzX2Jhc2VuYW1lfSIgXAotMiAke3RyaW1tZWRfZmFzdHFzX2Rpcn0ve30iJHtSMl9yZWFkc19iYXNlbmFtZX0iIFwKLS1vdXRwdXRfZGlyICIke291dHB1dF9kaXJfdG9wfSIgXAoyPiAiJHtvdXRwdXRfZGlyX3RvcH0iL2Jpc21hcmtfc3VtbWFyeS50eHQKCmBgYAoKIyMgTGlzdCBvdXRwdXRzCgpgYGB7ciBsaXN0LW91dHB1dHMsIGVuZ2luZT0nYmFzaCcsIGV2YWw9VFJVRX0KIyBMb2FkIGJhc2ggdmFyaWFibGVzIGludG8gbWVtb3J5CnNvdXJjZSAuYmFzaHZhcnMKCmNkICIke291dHB1dF9kaXJfdG9wfSIKCmxzIC1saApgYGAKCiMgTXVsdGlRQwoKYGBge3IgbXVsdGlxYywgZW5naW5lPSdiYXNoJywgZXZhbD1GQUxTRX0KIyBMb2FkIGJhc2ggdmFyaWFibGVzIGludG8gbWVtb3J5CnNvdXJjZSAuYmFzaHZhcnMKCmNkICIke291dHB1dF9kaXJfdG9wfSIKCiR7bXVsdGlxY30gLgpgYGAKCiMgQ2hlY2tzdW1zCgpgYGB7ciBnZW5lcmF0ZS1jaGVja3N1bXMsIGVuZ2luZT0nYmFzaCcsIGV2YWw9RkFMU0V9CiMgTG9hZCBiYXNoIHZhcmlhYmxlcyBpbnRvIG1lbW9yeQpzb3VyY2UgLmJhc2h2YXJzCgpjZCAiJHtvdXRwdXRfZGlyX3RvcH0iCgpmb3IgZmlsZSBpbiAqOyBkbwogIGlmIFsgIiR7ZmlsZX0iICE9ICJjaGVja3N1bXMubWQ1IiBdOyB0aGVuCiAgICBtZDVzdW0gIiR7ZmlsZX0iID4+IGNoZWNrc3Vtcy5tZDUKICBmaQpkb25lCmBgYAoKIyBDaXRhdGlvbnMK