/var/spool/slurm/d/job2268263/slurm_script: line 20: fg: no job control Using an excessive number of cores has a diminishing return! It is recommended not to exceed 8 cores per trimming process (you asked for 8 cores). Please consider re-specifying Path to Cutadapt set as: '/gscratch/srlab/strigg/bin/anaconda3/bin/cutadapt' (user defined) Cutadapt seems to be working fine (tested command '/gscratch/srlab/strigg/bin/anaconda3/bin/cutadapt --version') Cutadapt version: 2.4 Could not detect version of Python used by Cutadapt from the first line of Cutadapt (but found this: >>>#!/bin/sh<<<) Letting the (modified) Cutadapt deal with the Python version instead Parallel gzip (pigz) detected. Proceeding with multicore (de)compression using 8 cores No quality encoding type selected. Assuming that the data provided uses Sanger encoded Phred scores (default) Output will be written into the directory: /gscratch/scrubbed/strigg/analyses/20200316/ Writing report to '/gscratch/scrubbed/strigg/analyses/20200316/EPI-167_S10_L002_R1_001.fastq.gz_trimming_report.txt' SUMMARISING RUN PARAMETERS ========================== Input filename: /gscratch/srlab/strigg/data/Pgenr/FASTQS/raw/EPI-167_S10_L002_R1_001.fastq.gz Trimming mode: paired-end Trim Galore version: 0.6.4_dev Cutadapt version: 2.4 Python version: could not detect Number of cores used for trimming: 8 Quality Phred score cutoff: 20 Quality encoding type selected: ASCII+33 Adapter sequence: 'AGATCGGAAGAGCACACGTCTGAAC' (user defined) Maximum trimming error rate: 0.1 (default) Optional adapter 2 sequence (only used for read 2 of paired-end files): 'AGATCGGAAGAGCGTCGTGTAGGGA' Minimum required adapter overlap (stringency): 1 bp Minimum required sequence length for both reads before a sequence pair gets removed: 20 bp File was specified to be an MspI-digested RRBS sample. Read 1 sequences with adapter contamination will be trimmed a further 2 bp from their 3' end, and Read 2 sequences will be trimmed by 2 bp from their 5' end to remove potential methylation-biased bases from the end-repair reaction All Read 1 sequences will be trimmed by 25 bp from their 5' end to avoid poor qualities or biases All Read 2 sequences will be trimmed by 25 bp from their 5' end to avoid poor qualities or biases (e.g. M-bias for BS-Seq applications) All Read 1 sequences will be trimmed by 10 bp from their 3' end to avoid poor qualities or biases All Read 2 sequences will be trimmed by 10 bp from their 3' end to avoid poor qualities or biases Running FastQC on the data once trimming has completed Running FastQC with the following extra arguments: '--outdir /gscratch/scrubbed/strigg/analyses/20200316/TG_EPI-Test2 --threads 28' Output file(s) will be GZIP compressed Cutadapt seems to be fairly up-to-date (version 2.4). Setting -j 8 >>> Now performing adaptive quality trimming with a Phred-score cutoff of: 20 <<< This is cutadapt 2.4 with Python 3.7.6 Command line parameters: -j 8 -e 0.1 -q 20 -a X /gscratch/srlab/strigg/data/Pgenr/FASTQS/raw/EPI-167_S10_L002_R1_001.fastq.gz Processing reads on 8 cores in single-end mode ... 10000000 sequences processed 20000000 sequences processed Finished in 58.40 s (2 us/read; 25.54 M reads/minute). === Summary === Total reads processed: 24,859,230 Reads with adapters: 0 (0.0%) Reads written (passing filters): 24,859,230 (100.0%) Total basepairs processed: 2,510,782,230 bp Quality-trimmed: 11,488,232 bp (0.5%) Total written (filtered): 2,499,293,998 bp (99.5%) === Adapter 1 === Sequence: X; Type: regular 3'; Length: 1; Trimmed: 0 times. >>> Quality trimming completed <<< 24859230 sequences processed in total Writing final adapter and quality trimmed output to EPI-167_S10_L002_R1_001_trimmed.fq.gz >>> Now performing adapter trimming for the adapter sequence: 'AGATCGGAAGAGCACACGTCTGAAC' from file EPI-167_S10_L002_R1_001.fastq.gz_qual_trimmed.fastq <<< 10000000 sequences processed 20000000 sequences processed This is cutadapt 2.4 with Python 3.7.6 Command line parameters: -j 8 -e 0.1 -O 1 -a AGATCGGAAGAGCACACGTCTGAAC /gscratch/scrubbed/strigg/analyses/20200316/EPI-167_S10_L002_R1_001.fastq.gz_qual_trimmed.fastq Processing reads on 8 cores in single-end mode ... Finished in 81.72 s (3 us/read; 18.25 M reads/minute). === Summary === Total reads processed: 24,859,230 Reads with adapters: 13,310,825 (53.5%) Reads written (passing filters): 24,859,230 (100.0%) Total basepairs processed: 2,499,293,998 bp Total written (filtered): 2,298,441,331 bp (92.0%) === Adapter 1 === Sequence: AGATCGGAAGAGCACACGTCTGAAC; Type: regular 3'; Length: 25; Trimmed: 13310825 times. No. of allowed errors: 0-9 bp: 0; 10-19 bp: 1; 20-25 bp: 2 Bases preceding removed adapters: A: 25.5% C: 9.5% G: 23.7% T: 41.3% none/other: 0.0% Overview of removed sequences length count expect max.err error counts 1 5079588 6214807.5 0 5079588 2 1202251 1553701.9 0 1202251 3 456300 388425.5 0 456300 4 315396 97106.4 0 315396 5 162023 24276.6 0 162023 6 154952 6069.1 0 154952 7 141317 1517.3 0 141317 8 147815 379.3 0 147815 9 156397 94.8 0 154875 1522 10 144818 23.7 1 138784 6034 11 151610 5.9 1 144610 7000 12 144234 1.5 1 137849 6385 13 138279 0.4 1 132093 6186 14 149582 0.1 1 141472 8110 15 139617 0.0 1 132371 7246 16 149533 0.0 1 140718 8815 17 143290 0.0 1 134911 8379 18 132507 0.0 1 123777 8159 571 19 143599 0.0 1 132190 10635 774 20 131755 0.0 2 122046 8589 1120 21 146494 0.0 2 132987 11445 2062 22 135126 0.0 2 124303 9630 1193 23 126194 0.0 2 115663 9314 1217 24 131857 0.0 2 119870 10387 1600 25 122353 0.0 2 112116 9190 1047 26 132356 0.0 2 119736 10869 1751 27 119873 0.0 2 109836 8721 1316 28 113147 0.0 2 103784 8301 1062 29 123716 0.0 2 112992 9263 1461 30 113300 0.0 2 103872 8363 1065 31 120517 0.0 2 109240 9537 1740 32 111072 0.0 2 102015 7969 1088 33 114305 0.0 2 104447 8569 1289 34 108779 0.0 2 99492 8120 1167 35 104061 0.0 2 95648 7427 986 36 99353 0.0 2 91169 7175 1009 37 105972 0.0 2 96779 8018 1175 38 92166 0.0 2 84732 6433 1001 39 93699 0.0 2 85639 6965 1095 40 94449 0.0 2 85795 7662 992 41 129224 0.0 2 119820 8107 1297 42 78034 0.0 2 72578 4766 690 43 35611 0.0 2 31892 3292 427 44 72241 0.0 2 66560 4941 740 45 66871 0.0 2 61637 4601 633 46 63480 0.0 2 58567 4379 534 47 65222 0.0 2 59962 4547 713 48 58338 0.0 2 53531 4197 610 49 59861 0.0 2 54840 4380 641 50 53802 0.0 2 49655 3678 469 51 50397 0.0 2 46525 3380 492 52 46801 0.0 2 43097 3252 452 53 42870 0.0 2 39683 2782 405 54 41405 0.0 2 38238 2746 421 55 40756 0.0 2 37782 2611 363 56 36903 0.0 2 34142 2423 338 57 33898 0.0 2 31227 2360 311 58 31239 0.0 2 28980 2027 232 59 30217 0.0 2 28005 1966 246 60 26732 0.0 2 24878 1668 186 61 26427 0.0 2 24460 1787 180 62 25361 0.0 2 23508 1647 206 63 21705 0.0 2 20154 1396 155 64 19937 0.0 2 18653 1152 132 65 17966 0.0 2 16711 1137 118 66 16513 0.0 2 15317 1088 108 67 15408 0.0 2 14252 1052 104 68 14019 0.0 2 12960 970 89 69 13548 0.0 2 12518 939 91 70 12389 0.0 2 11493 818 78 71 12247 0.0 2 11250 892 105 72 14140 0.0 2 12722 1193 225 73 24750 0.0 2 21535 2947 268 74 80751 0.0 2 75790 4690 271 75 62956 0.0 2 59263 3517 176 76 34298 0.0 2 32139 2050 109 77 19980 0.0 2 18745 1172 63 78 11508 0.0 2 10781 686 41 79 6171 0.0 2 5742 410 19 80 3991 0.0 2 3687 279 25 81 2402 0.0 2 2222 166 14 82 1642 0.0 2 1490 140 12 83 1273 0.0 2 1174 88 11 84 1130 0.0 2 1031 91 8 85 989 0.0 2 903 78 8 86 959 0.0 2 877 76 6 87 835 0.0 2 750 78 7 88 716 0.0 2 643 63 10 89 798 0.0 2 727 62 9 90 871 0.0 2 802 64 5 91 1229 0.0 2 1108 109 12 92 1963 0.0 2 1790 160 13 93 4579 0.0 2 4247 306 26 94 13773 0.0 2 12652 1011 110 95 24682 0.0 2 22814 1730 138 96 11975 0.0 2 11015 885 75 97 8863 0.0 2 8123 684 56 98 3854 0.0 2 3544 284 26 99 4127 0.0 2 3800 310 17 100 4411 0.0 2 4002 383 26 101 8135 0.0 2 7184 899 52 Successfully deleted temporary file EPI-167_S10_L002_R1_001.fastq.gz_qual_trimmed.fastq RUN STATISTICS FOR INPUT FILE: /gscratch/srlab/strigg/data/Pgenr/FASTQS/raw/EPI-167_S10_L002_R1_001.fastq.gz ============================================= 24859230 sequences processed in total Sequences were truncated to a varying degree because of deteriorating qualities (Phred score quality cutoff: 20): 2260450 (9.1%) The length threshold of paired-end sequences gets evaluated later on (in the validation step) RRBS reads trimmed by additional 2 bp when adapter contamination was detected: 13261661 (53.3%) Writing report to '/gscratch/scrubbed/strigg/analyses/20200316/EPI-167_S10_L002_R2_001.fastq.gz_trimming_report.txt' SUMMARISING RUN PARAMETERS ========================== Input filename: /gscratch/srlab/strigg/data/Pgenr/FASTQS/raw/EPI-167_S10_L002_R2_001.fastq.gz Trimming mode: paired-end Trim Galore version: 0.6.4_dev Cutadapt version: 2.4 Python version: could not detect Number of cores used for trimming: 8 Quality Phred score cutoff: 20 Quality encoding type selected: ASCII+33 Adapter sequence: 'AGATCGGAAGAGCACACGTCTGAAC' (user defined) Maximum trimming error rate: 0.1 (default) Optional adapter 2 sequence (only used for read 2 of paired-end files): 'AGATCGGAAGAGCGTCGTGTAGGGA' Minimum required adapter overlap (stringency): 1 bp Minimum required sequence length for both reads before a sequence pair gets removed: 20 bp File was specified to be an MspI-digested RRBS sample. Read 1 sequences with adapter contamination will be trimmed a further 2 bp from their 3' end, and Read 2 sequences will be trimmed by 2 bp from their 5' end to remove potential methylation-biased bases from the end-repair reaction All Read 1 sequences will be trimmed by 25 bp from their 5' end to avoid poor qualities or biases All Read 2 sequences will be trimmed by 25 bp from their 5' end to avoid poor qualities or biases (e.g. M-bias for BS-Seq applications) All Read 1 sequences will be trimmed by 10 bp from their 3' end to avoid poor qualities or biases All Read 2 sequences will be trimmed by 10 bp from their 3' end to avoid poor qualities or biases Running FastQC on the data once trimming has completed Running FastQC with the following extra arguments: '--outdir /gscratch/scrubbed/strigg/analyses/20200316/TG_EPI-Test2 --threads 28' Output file(s) will be GZIP compressed Cutadapt seems to be fairly up-to-date (version 2.4). Setting -j -j 8 >>> Now performing adaptive quality trimming with a Phred-score cutoff of: 20 <<< This is cutadapt 2.4 with Python 3.7.6 Command line parameters: -j 8 -e 0.1 -q 20 -a X /gscratch/srlab/strigg/data/Pgenr/FASTQS/raw/EPI-167_S10_L002_R2_001.fastq.gz Processing reads on 8 cores in single-end mode ... 10000000 sequences processed 20000000 sequences processed Finished in 58.75 s (2 us/read; 25.39 M reads/minute). === Summary === Total reads processed: 24,859,230 Reads with adapters: 0 (0.0%) Reads written (passing filters): 24,859,230 (100.0%) Total basepairs processed: 2,510,782,230 bp Quality-trimmed: 22,659,614 bp (0.9%) Total written (filtered): 2,488,122,616 bp (99.1%) === Adapter 1 === Sequence: X; Type: regular 3'; Length: 1; Trimmed: 0 times. >>> Quality trimming completed <<< 24859230 sequences processed in total Writing final adapter and quality trimmed output to EPI-167_S10_L002_R2_001_trimmed.fq.gz >>> Now performing adapter trimming for the adapter sequence: 'AGATCGGAAGAGCGTCGTGTAGGGA' from file EPI-167_S10_L002_R2_001.fastq.gz_qual_trimmed.fastq <<< 10000000 sequences processed 20000000 sequences processed This is cutadapt 2.4 with Python 3.7.6 Command line parameters: -j 8 -e 0.1 -O 1 -a AGATCGGAAGAGCGTCGTGTAGGGA /gscratch/scrubbed/strigg/analyses/20200316/EPI-167_S10_L002_R2_001.fastq.gz_qual_trimmed.fastq Processing reads on 8 cores in single-end mode ... Finished in 85.79 s (3 us/read; 17.39 M reads/minute). === Summary === Total reads processed: 24,859,230 Reads with adapters: 15,363,536 (61.8%) Reads written (passing filters): 24,859,230 (100.0%) Total basepairs processed: 2,488,122,616 bp Total written (filtered): 2,293,251,313 bp (92.2%) === Adapter 1 === Sequence: AGATCGGAAGAGCGTCGTGTAGGGA; Type: regular 3'; Length: 25; Trimmed: 15363536 times. No. of allowed errors: 0-9 bp: 0; 10-19 bp: 1; 20-25 bp: 2 Bases preceding removed adapters: A: 40.3% C: 20.3% G: 7.3% T: 32.1% none/other: 0.0% Overview of removed sequences length count expect max.err error counts 1 8554514 6214807.5 0 8554514 2 247131 1553701.9 0 247131 3 191193 388425.5 0 191193 4 164298 97106.4 0 164298 5 160566 24276.6 0 160566 6 159035 6069.1 0 159035 7 150319 1517.3 0 150319 8 152605 379.3 0 152605 9 152922 94.8 0 152285 637 10 151128 23.7 1 146689 4439 11 146920 5.9 1 141612 5308 12 148837 1.5 1 143363 5474 13 142341 0.4 1 137280 5061 14 151875 0.1 1 146116 5759 15 140621 0.0 1 135080 5541 16 140329 0.0 1 134360 5969 17 147590 0.0 1 141058 6532 18 131268 0.0 1 125180 5970 118 19 137242 0.0 1 130202 6888 152 20 134860 0.0 2 126296 7299 1265 21 136470 0.0 2 126594 8307 1569 22 137929 0.0 2 128046 8405 1478 23 132148 0.0 2 122951 7765 1432 24 137535 0.0 2 127293 8713 1529 25 121830 0.0 2 112500 7874 1456 26 122372 0.0 2 111493 9021 1858 27 122833 0.0 2 110779 9719 2335 28 125558 0.0 2 115435 8627 1496 29 121040 0.0 2 109875 9372 1793 30 127761 0.0 2 117798 8505 1458 31 110644 0.0 2 101253 7874 1517 32 113145 0.0 2 104687 7317 1141 33 117931 0.0 2 107886 8506 1539 34 119747 0.0 2 108853 9087 1807 35 109358 0.0 2 101564 6789 1005 36 102469 0.0 2 93985 7185 1299 37 101986 0.0 2 93931 6852 1203 38 88289 0.0 2 81349 5989 951 39 91384 0.0 2 83980 6298 1106 40 87968 0.0 2 81063 5902 1003 41 85810 0.0 2 79585 5436 789 42 82412 0.0 2 76871 4870 671 43 73026 0.0 2 67328 4920 778 44 72521 0.0 2 67133 4755 633 45 84963 0.0 2 79659 4649 655 46 65401 0.0 2 60883 3896 622 47 46340 0.0 2 42573 3316 451 48 62890 0.0 2 58972 3433 485 49 44178 0.0 2 41087 2712 379 50 45926 0.0 2 42356 3132 438 51 59803 0.0 2 56373 3012 418 52 36947 0.0 2 34103 2470 374 53 36693 0.0 2 33991 2310 392 54 32194 0.0 2 29695 2182 317 55 36861 0.0 2 34493 2101 267 56 34269 0.0 2 31729 2174 366 57 30720 0.0 2 28494 1961 265 58 28380 0.0 2 26339 1776 265 59 26631 0.0 2 24733 1671 227 60 25170 0.0 2 23184 1700 286 61 24593 0.0 2 22735 1596 262 62 24060 0.0 2 22223 1561 276 63 22601 0.0 2 20779 1597 225 64 21649 0.0 2 19897 1515 237 65 22204 0.0 2 20390 1572 242 66 24131 0.0 2 22040 1770 321 67 35937 0.0 2 31177 4455 305 68 129003 0.0 2 123606 5023 374 69 47313 0.0 2 44432 2606 275 70 24870 0.0 2 23330 1358 182 71 12976 0.0 2 11952 904 120 72 8650 0.0 2 7938 594 118 73 6006 0.0 2 5428 500 78 74 4673 0.0 2 4170 423 80 75 3587 0.0 2 3246 292 49 76 2967 0.0 2 2644 260 63 77 2595 0.0 2 2287 248 60 78 2277 0.0 2 1988 229 60 79 1974 0.0 2 1740 196 38 80 1588 0.0 2 1403 150 35 81 1399 0.0 2 1220 147 32 82 1178 0.0 2 1020 129 29 83 1067 0.0 2 929 111 27 84 908 0.0 2 767 111 30 85 844 0.0 2 702 108 34 86 765 0.0 2 629 110 26 87 784 0.0 2 659 101 24 88 797 0.0 2 673 96 28 89 913 0.0 2 759 123 31 90 1123 0.0 2 931 153 39 91 1402 0.0 2 1155 188 59 92 2147 0.0 2 1810 255 82 93 4579 0.0 2 3823 597 159 94 13431 0.0 2 11681 1388 362 95 23741 0.0 2 20778 2421 542 96 11598 0.0 2 10136 1204 258 97 8556 0.0 2 7512 866 178 98 3633 0.0 2 3223 345 65 99 3883 0.0 2 3382 409 92 100 4128 0.0 2 3619 404 105 101 7880 0.0 2 6828 858 194 Successfully deleted temporary file EPI-167_S10_L002_R2_001.fastq.gz_qual_trimmed.fastq RUN STATISTICS FOR INPUT FILE: /gscratch/srlab/strigg/data/Pgenr/FASTQS/raw/EPI-167_S10_L002_R2_001.fastq.gz ============================================= 24859230 sequences processed in total Sequences were truncated to a varying degree because of deteriorating qualities (Phred score quality cutoff: 20): 2502338 (10.1%) The length threshold of paired-end sequences gets evaluated later on (in the validation step) RRBS reads trimmed by additional 2 bp when adapter contamination was detected: 0 (0.0%) Validate paired-end files EPI-167_S10_L002_R1_001_trimmed.fq.gz and EPI-167_S10_L002_R2_001_trimmed.fq.gz file_1: EPI-167_S10_L002_R1_001_trimmed.fq.gz, file_2: EPI-167_S10_L002_R2_001_trimmed.fq.gz >>>>> Now validing the length of the 2 paired-end infiles: EPI-167_S10_L002_R1_001_trimmed.fq.gz and EPI-167_S10_L002_R2_001_trimmed.fq.gz <<<<< Writing validated paired-end Read 1 reads to EPI-167_S10_L002_R1_001_val_1.fq.gz Writing validated paired-end Read 2 reads to EPI-167_S10_L002_R2_001_val_2.fq.gz Total number of sequences analysed: 24859230 Number of sequence pairs removed because at least one read was shorter than the length cutoff (20 bp): 1422718 (5.72%) >>> Now running FastQC on the validated data EPI-167_S10_L002_R1_001_val_1.fq.gz<<< Started analysis of EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 5% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 10% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 15% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 20% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 25% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 30% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 35% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 40% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 45% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 50% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 55% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 60% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 65% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 70% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 75% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 80% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 85% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 90% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 95% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Analysis complete for EPI-167_S10_L002_R1_001_val_1.fq.gz >>> Now running FastQC on the validated data EPI-167_S10_L002_R2_001_val_2.fq.gz<<< Started analysis of EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 5% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 10% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 15% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 20% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 25% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 30% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 35% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 40% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 45% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 50% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 55% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 60% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 65% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 70% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 75% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 80% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 85% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 90% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 95% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Analysis complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Deleting both intermediate output files EPI-167_S10_L002_R1_001_trimmed.fq.gz and EPI-167_S10_L002_R2_001_trimmed.fq.gz ====================================================================================================