---
author: Sam White
toc-title: Contents
toc-depth: 5
toc-location: left
layout: post
title: FastQC-MultiQC - Yaamini's C.virginica RNAseq and WGBS from ZymoResearch on Mox
date: '2021-06-29 07:25'
tags: 
  - mox
  - fastqc
  - multiqc
  - yaamini
  - Crassostrea virginica
  - Eastern oyster
categories: 
  - 2021
  - Miscellaneous
---
Finally got around to running [`FastQC`](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/) on [Yaamini's RNAseq and WGBS sequencing data recieved on 20210528](https://robertslab.github.io/sams-notebook/posts/2021/2021-05-28-Data-Received---Yaaminis-C.virginica-WGBS-and-RNAseq-Data-from-ZymoResearch/).

SBATCH script (GitHub):

- [0210629_cvir_fastqc_yaamini_rnaseq-wgbs.sh](https://github.com/RobertsLab/sams-notebook/blob/master/sbatch_scripts/20210629_cvir_fastqc_yaamini_rnaseq-wgbs.sh)

```shell
#!/bin/bash
## Job Name
#SBATCH --job-name=20210629_cvir_fastqc_yaamini_rnaseq-wgbs
## Allocation Definition
#SBATCH --account=coenv
#SBATCH --partition=coenv
## Resources
## Nodes
#SBATCH --nodes=1
## Walltime (days-hours:minutes:seconds format)
#SBATCH --time=5-00:00:00
## Memory per node
#SBATCH --mem=200G
##turn on e-mail notification
#SBATCH --mail-type=ALL
#SBATCH --mail-user=samwhite@uw.edu
## Specify the working directory for this job
#SBATCH --chdir=/gscratch/scrubbed/samwhite/outputs/20210629_cvir_fastqc_yaamini_rnaseq-wgbs


### FastQC assessment of raw sequencing from Ronit's ploidy WGBS.


###################################################################################
# These variables need to be set by user

# FastQC output directory
output_dir=$(pwd)

# Set number of CPUs to use
threads=40

# Input/output files
checksums=fastq_checksums.md5
fastq_list=fastq_list.txt
raw_reads_dir=/gscratch/srlab/sam/data/C_gigas/wgbs

# Paths to programs
fastqc=/gscratch/srlab/programs/fastqc_v0.11.9/fastqc
multiqc=/gscratch/srlab/programs/anaconda3/bin/multiqc


# Programs associative array
declare -A programs_array
programs_array=(
[fastqc]="${fastqc}" \
[multiqc]="${multiqc}"
)

###################################################################################

# Exit script if any command fails
set -e

# Load Python Mox module for Python module availability
module load intel-python3_2017

# Sync raw FastQ files to working directory
rsync --archive --verbose \
/gscratch/scrubbed/samwhite/data/C_virginica/RNAseq/*.fastq.gz .

rsync --archive --verbose \
/gscratch/scrubbed/samwhite/data/C_virginica/DNAseq/*.fq.gz .

# Populate array with FastQ files
fastq_array=(*.f*q.gz)

# Pass array contents to new variable
fastqc_list=$(echo "${fastq_array[*]}")

# Run FastQC
# NOTE: Do NOT quote ${fastqc_list}
${programs_array[fastqc]} \
--threads ${threads} \
--outdir ${output_dir} \
${fastqc_list}


# Create list of fastq files used in analysis
echo "${fastqc_list}" | tr " " "\n" >> ${fastq_list}

# Generate checksums for reference
while read -r line
do

	# Generate MD5 checksums for each input FastQ file
	echo "Generating MD5 checksum for ${line}."
	md5sum "${line}" >> "${checksums}"
	echo "Completed: MD5 checksum for ${line}."
	echo ""

	# Remove fastq files from working directory
	echo "Removing ${line} from directory"
	rm "${line}"
	echo "Removed ${line} from directory"
	echo ""
done < ${fastq_list}

# Run MultiQC
${programs_array[multiqc]} .


# Capture program options
for program in "${!programs_array[@]}"
do
	{
  echo "Program options for ${program}: "
	echo ""
  # Handle samtools help menus
  if [[ "${program}" == "samtools_index" ]] \
  || [[ "${program}" == "samtools_sort" ]] \
  || [[ "${program}" == "samtools_view" ]]
  then
    ${programs_array[$program]}
  fi
	${programs_array[$program]} -h
	echo ""
	echo ""
	echo "----------------------------------------------"
	echo ""
	echo ""
} &>> program_options.log || true

  # If MultiQC is in programs_array, copy the config file to this directory.
  if [[ "${program}" == "multiqc" ]]; then
  	cp --preserve ~/.multiqc_config.yaml .
  fi
done


# Document programs in PATH (primarily for program version ID)
{
date
echo ""
echo "System PATH for $SLURM_JOB_ID"
echo ""
printf "%0.s-" {1..10}
echo "${PATH}" | tr : \\n
} >> system_path.log
```

---

# RESULTS

Runtime was ~3.75hrs:

![FastQC runtime on Mox](https://github.com/RobertsLab/sams-notebook/blob/master/images/screencaps/20210629_cvir_fastqc_yaamini_rnaseq-wgbs_runtime.png?raw=true)

Output folder:

- [20210629_cvir_fastqc_yaamini_rnaseq-wgbs/](https://gannet.fish.washington.edu/Atumefaciens/20210629_cvir_fastqc_yaamini_rnaseq-wgbs/)

  - #### MultiQC Report (HTML; open in browser)

    - [20210629_cvir_fastqc_yaamini_rnaseq-wgbs/multiqc_report.html](https://gannet.fish.washington.edu/Atumefaciens/20210629_cvir_fastqc_yaamini_rnaseq-wgbs/multiqc_report.html)

When viewing the [`MultiQC`](https://multiqc.info/) report, remember that there are two different sequencing projects here: RNAseq and WGBS. This explains why there appear to be such drastic differences between two "groups" of files in the report; because there are!

Will get this info added to [`Nightingales`](https://b.link/nightingales) (Google Sheet).