/var/spool/slurm/d/job2268256/slurm_script: line 20: fg: no job control Using an excessive number of cores has a diminishing return! It is recommended not to exceed 8 cores per trimming process (you asked for 8 cores). Please consider re-specifying Path to Cutadapt set as: '/gscratch/srlab/strigg/bin/anaconda3/bin/cutadapt' (user defined) Cutadapt seems to be working fine (tested command '/gscratch/srlab/strigg/bin/anaconda3/bin/cutadapt --version') Cutadapt version: 2.4 Could not detect version of Python used by Cutadapt from the first line of Cutadapt (but found this: >>>#!/bin/sh<<<) Letting the (modified) Cutadapt deal with the Python version instead Parallel gzip (pigz) detected. Proceeding with multicore (de)compression using 8 cores No quality encoding type selected. Assuming that the data provided uses Sanger encoded Phred scores (default) Output will be written into the directory: /gscratch/scrubbed/strigg/analyses/20200316/ Setting the option '--clip_r2 2' (to remove methylation bias from the start of Read 2) Writing report to '/gscratch/scrubbed/strigg/analyses/20200316/EPI-167_S10_L002_R1_001.fastq.gz_trimming_report.txt' SUMMARISING RUN PARAMETERS ========================== Input filename: /gscratch/srlab/strigg/data/Pgenr/FASTQS/raw/EPI-167_S10_L002_R1_001.fastq.gz Trimming mode: paired-end Trim Galore version: 0.6.4_dev Cutadapt version: 2.4 Python version: could not detect Number of cores used for trimming: 8 Quality Phred score cutoff: 20 Quality encoding type selected: ASCII+33 Adapter sequence: 'AGATCGGAAGAGCACACGTCTGAAC' (user defined) Maximum trimming error rate: 0.1 (default) Optional adapter 2 sequence (only used for read 2 of paired-end files): 'AGATCGGAAGAGCGTCGTGTAGGGA' Minimum required adapter overlap (stringency): 1 bp Minimum required sequence length for both reads before a sequence pair gets removed: 20 bp File was specified to be an MspI-digested RRBS sample. Read 1 sequences with adapter contamination will be trimmed a further 2 bp from their 3' end, and Read 2 sequences will be trimmed by 2 bp from their 5' end to remove potential methylation-biased bases from the end-repair reaction All Read 2 sequences will be trimmed by 2 bp from their 5' end to avoid poor qualities or biases (e.g. M-bias for BS-Seq applications) Running FastQC on the data once trimming has completed Running FastQC with the following extra arguments: '--outdir /gscratch/scrubbed/strigg/analyses/20200311/WGBS_MBD/FASTQC --threads 28' Output file(s) will be GZIP compressed Cutadapt seems to be fairly up-to-date (version 2.4). Setting -j 8 >>> Now performing adaptive quality trimming with a Phred-score cutoff of: 20 <<< This is cutadapt 2.4 with Python 3.7.6 Command line parameters: -j 8 -e 0.1 -q 20 -a X /gscratch/srlab/strigg/data/Pgenr/FASTQS/raw/EPI-167_S10_L002_R1_001.fastq.gz Processing reads on 8 cores in single-end mode ... 10000000 sequences processed 20000000 sequences processed Finished in 62.73 s (3 us/read; 23.78 M reads/minute). === Summary === Total reads processed: 24,859,230 Reads with adapters: 0 (0.0%) Reads written (passing filters): 24,859,230 (100.0%) Total basepairs processed: 2,510,782,230 bp Quality-trimmed: 11,488,232 bp (0.5%) Total written (filtered): 2,499,293,998 bp (99.5%) === Adapter 1 === Sequence: X; Type: regular 3'; Length: 1; Trimmed: 0 times. >>> Quality trimming completed <<< 24859230 sequences processed in total Writing final adapter and quality trimmed output to EPI-167_S10_L002_R1_001_trimmed.fq.gz >>> Now performing adapter trimming for the adapter sequence: 'AGATCGGAAGAGCACACGTCTGAAC' from file EPI-167_S10_L002_R1_001.fastq.gz_qual_trimmed.fastq <<< 10000000 sequences processed 20000000 sequences processed This is cutadapt 2.4 with Python 3.7.6 Command line parameters: -j 8 -e 0.1 -O 1 -a AGATCGGAAGAGCACACGTCTGAAC /gscratch/scrubbed/strigg/analyses/20200316/EPI-167_S10_L002_R1_001.fastq.gz_qual_trimmed.fastq Processing reads on 8 cores in single-end mode ... Finished in 81.27 s (3 us/read; 18.35 M reads/minute). === Summary === Total reads processed: 24,859,230 Reads with adapters: 13,310,825 (53.5%) Reads written (passing filters): 24,859,230 (100.0%) Total basepairs processed: 2,499,293,998 bp Total written (filtered): 2,298,441,331 bp (92.0%) === Adapter 1 === Sequence: AGATCGGAAGAGCACACGTCTGAAC; Type: regular 3'; Length: 25; Trimmed: 13310825 times. No. of allowed errors: 0-9 bp: 0; 10-19 bp: 1; 20-25 bp: 2 Bases preceding removed adapters: A: 25.5% C: 9.5% G: 23.7% T: 41.3% none/other: 0.0% Overview of removed sequences length count expect max.err error counts 1 5079588 6214807.5 0 5079588 2 1202251 1553701.9 0 1202251 3 456300 388425.5 0 456300 4 315396 97106.4 0 315396 5 162023 24276.6 0 162023 6 154952 6069.1 0 154952 7 141317 1517.3 0 141317 8 147815 379.3 0 147815 9 156397 94.8 0 154875 1522 10 144818 23.7 1 138784 6034 11 151610 5.9 1 144610 7000 12 144234 1.5 1 137849 6385 13 138279 0.4 1 132093 6186 14 149582 0.1 1 141472 8110 15 139617 0.0 1 132371 7246 16 149533 0.0 1 140718 8815 17 143290 0.0 1 134911 8379 18 132507 0.0 1 123777 8159 571 19 143599 0.0 1 132190 10635 774 20 131755 0.0 2 122046 8589 1120 21 146494 0.0 2 132987 11445 2062 22 135126 0.0 2 124303 9630 1193 23 126194 0.0 2 115663 9314 1217 24 131857 0.0 2 119870 10387 1600 25 122353 0.0 2 112116 9190 1047 26 132356 0.0 2 119736 10869 1751 27 119873 0.0 2 109836 8721 1316 28 113147 0.0 2 103784 8301 1062 29 123716 0.0 2 112992 9263 1461 30 113300 0.0 2 103872 8363 1065 31 120517 0.0 2 109240 9537 1740 32 111072 0.0 2 102015 7969 1088 33 114305 0.0 2 104447 8569 1289 34 108779 0.0 2 99492 8120 1167 35 104061 0.0 2 95648 7427 986 36 99353 0.0 2 91169 7175 1009 37 105972 0.0 2 96779 8018 1175 38 92166 0.0 2 84732 6433 1001 39 93699 0.0 2 85639 6965 1095 40 94449 0.0 2 85795 7662 992 41 129224 0.0 2 119820 8107 1297 42 78034 0.0 2 72578 4766 690 43 35611 0.0 2 31892 3292 427 44 72241 0.0 2 66560 4941 740 45 66871 0.0 2 61637 4601 633 46 63480 0.0 2 58567 4379 534 47 65222 0.0 2 59962 4547 713 48 58338 0.0 2 53531 4197 610 49 59861 0.0 2 54840 4380 641 50 53802 0.0 2 49655 3678 469 51 50397 0.0 2 46525 3380 492 52 46801 0.0 2 43097 3252 452 53 42870 0.0 2 39683 2782 405 54 41405 0.0 2 38238 2746 421 55 40756 0.0 2 37782 2611 363 56 36903 0.0 2 34142 2423 338 57 33898 0.0 2 31227 2360 311 58 31239 0.0 2 28980 2027 232 59 30217 0.0 2 28005 1966 246 60 26732 0.0 2 24878 1668 186 61 26427 0.0 2 24460 1787 180 62 25361 0.0 2 23508 1647 206 63 21705 0.0 2 20154 1396 155 64 19937 0.0 2 18653 1152 132 65 17966 0.0 2 16711 1137 118 66 16513 0.0 2 15317 1088 108 67 15408 0.0 2 14252 1052 104 68 14019 0.0 2 12960 970 89 69 13548 0.0 2 12518 939 91 70 12389 0.0 2 11493 818 78 71 12247 0.0 2 11250 892 105 72 14140 0.0 2 12722 1193 225 73 24750 0.0 2 21535 2947 268 74 80751 0.0 2 75790 4690 271 75 62956 0.0 2 59263 3517 176 76 34298 0.0 2 32139 2050 109 77 19980 0.0 2 18745 1172 63 78 11508 0.0 2 10781 686 41 79 6171 0.0 2 5742 410 19 80 3991 0.0 2 3687 279 25 81 2402 0.0 2 2222 166 14 82 1642 0.0 2 1490 140 12 83 1273 0.0 2 1174 88 11 84 1130 0.0 2 1031 91 8 85 989 0.0 2 903 78 8 86 959 0.0 2 877 76 6 87 835 0.0 2 750 78 7 88 716 0.0 2 643 63 10 89 798 0.0 2 727 62 9 90 871 0.0 2 802 64 5 91 1229 0.0 2 1108 109 12 92 1963 0.0 2 1790 160 13 93 4579 0.0 2 4247 306 26 94 13773 0.0 2 12652 1011 110 95 24682 0.0 2 22814 1730 138 96 11975 0.0 2 11015 885 75 97 8863 0.0 2 8123 684 56 98 3854 0.0 2 3544 284 26 99 4127 0.0 2 3800 310 17 100 4411 0.0 2 4002 383 26 101 8135 0.0 2 7184 899 52 Successfully deleted temporary file EPI-167_S10_L002_R1_001.fastq.gz_qual_trimmed.fastq RUN STATISTICS FOR INPUT FILE: /gscratch/srlab/strigg/data/Pgenr/FASTQS/raw/EPI-167_S10_L002_R1_001.fastq.gz ============================================= 24859230 sequences processed in total Sequences were truncated to a varying degree because of deteriorating qualities (Phred score quality cutoff: 20): 2260450 (9.1%) The length threshold of paired-end sequences gets evaluated later on (in the validation step) RRBS reads trimmed by additional 2 bp when adapter contamination was detected: 13261661 (53.3%) Writing report to '/gscratch/scrubbed/strigg/analyses/20200316/EPI-167_S10_L002_R2_001.fastq.gz_trimming_report.txt' SUMMARISING RUN PARAMETERS ========================== Input filename: /gscratch/srlab/strigg/data/Pgenr/FASTQS/raw/EPI-167_S10_L002_R2_001.fastq.gz Trimming mode: paired-end Trim Galore version: 0.6.4_dev Cutadapt version: 2.4 Python version: could not detect Number of cores used for trimming: 8 Quality Phred score cutoff: 20 Quality encoding type selected: ASCII+33 Adapter sequence: 'AGATCGGAAGAGCACACGTCTGAAC' (user defined) Maximum trimming error rate: 0.1 (default) Optional adapter 2 sequence (only used for read 2 of paired-end files): 'AGATCGGAAGAGCGTCGTGTAGGGA' Minimum required adapter overlap (stringency): 1 bp Minimum required sequence length for both reads before a sequence pair gets removed: 20 bp File was specified to be an MspI-digested RRBS sample. Read 1 sequences with adapter contamination will be trimmed a further 2 bp from their 3' end, and Read 2 sequences will be trimmed by 2 bp from their 5' end to remove potential methylation-biased bases from the end-repair reaction All Read 2 sequences will be trimmed by 2 bp from their 5' end to avoid poor qualities or biases (e.g. M-bias for BS-Seq applications) Running FastQC on the data once trimming has completed Running FastQC with the following extra arguments: '--outdir /gscratch/scrubbed/strigg/analyses/20200311/WGBS_MBD/FASTQC --threads 28' Output file(s) will be GZIP compressed Cutadapt seems to be fairly up-to-date (version 2.4). Setting -j -j 8 >>> Now performing adaptive quality trimming with a Phred-score cutoff of: 20 <<< This is cutadapt 2.4 with Python 3.7.6 Command line parameters: -j 8 -e 0.1 -q 20 -a X /gscratch/srlab/strigg/data/Pgenr/FASTQS/raw/EPI-167_S10_L002_R2_001.fastq.gz Processing reads on 8 cores in single-end mode ... 10000000 sequences processed 20000000 sequences processed Finished in 59.04 s (2 us/read; 25.26 M reads/minute). === Summary === Total reads processed: 24,859,230 Reads with adapters: 0 (0.0%) Reads written (passing filters): 24,859,230 (100.0%) Total basepairs processed: 2,510,782,230 bp Quality-trimmed: 22,659,614 bp (0.9%) Total written (filtered): 2,488,122,616 bp (99.1%) === Adapter 1 === Sequence: X; Type: regular 3'; Length: 1; Trimmed: 0 times. >>> Quality trimming completed <<< 24859230 sequences processed in total Writing final adapter and quality trimmed output to EPI-167_S10_L002_R2_001_trimmed.fq.gz >>> Now performing adapter trimming for the adapter sequence: 'AGATCGGAAGAGCGTCGTGTAGGGA' from file EPI-167_S10_L002_R2_001.fastq.gz_qual_trimmed.fastq <<< 10000000 sequences processed 20000000 sequences processed This is cutadapt 2.4 with Python 3.7.6 Command line parameters: -j 8 -e 0.1 -O 1 -a AGATCGGAAGAGCGTCGTGTAGGGA /gscratch/scrubbed/strigg/analyses/20200316/EPI-167_S10_L002_R2_001.fastq.gz_qual_trimmed.fastq Processing reads on 8 cores in single-end mode ... Finished in 85.41 s (3 us/read; 17.46 M reads/minute). === Summary === Total reads processed: 24,859,230 Reads with adapters: 15,363,536 (61.8%) Reads written (passing filters): 24,859,230 (100.0%) Total basepairs processed: 2,488,122,616 bp Total written (filtered): 2,293,251,313 bp (92.2%) === Adapter 1 === Sequence: AGATCGGAAGAGCGTCGTGTAGGGA; Type: regular 3'; Length: 25; Trimmed: 15363536 times. No. of allowed errors: 0-9 bp: 0; 10-19 bp: 1; 20-25 bp: 2 Bases preceding removed adapters: A: 40.3% C: 20.3% G: 7.3% T: 32.1% none/other: 0.0% Overview of removed sequences length count expect max.err error counts 1 8554514 6214807.5 0 8554514 2 247131 1553701.9 0 247131 3 191193 388425.5 0 191193 4 164298 97106.4 0 164298 5 160566 24276.6 0 160566 6 159035 6069.1 0 159035 7 150319 1517.3 0 150319 8 152605 379.3 0 152605 9 152922 94.8 0 152285 637 10 151128 23.7 1 146689 4439 11 146920 5.9 1 141612 5308 12 148837 1.5 1 143363 5474 13 142341 0.4 1 137280 5061 14 151875 0.1 1 146116 5759 15 140621 0.0 1 135080 5541 16 140329 0.0 1 134360 5969 17 147590 0.0 1 141058 6532 18 131268 0.0 1 125180 5970 118 19 137242 0.0 1 130202 6888 152 20 134860 0.0 2 126296 7299 1265 21 136470 0.0 2 126594 8307 1569 22 137929 0.0 2 128046 8405 1478 23 132148 0.0 2 122951 7765 1432 24 137535 0.0 2 127293 8713 1529 25 121830 0.0 2 112500 7874 1456 26 122372 0.0 2 111493 9021 1858 27 122833 0.0 2 110779 9719 2335 28 125558 0.0 2 115435 8627 1496 29 121040 0.0 2 109875 9372 1793 30 127761 0.0 2 117798 8505 1458 31 110644 0.0 2 101253 7874 1517 32 113145 0.0 2 104687 7317 1141 33 117931 0.0 2 107886 8506 1539 34 119747 0.0 2 108853 9087 1807 35 109358 0.0 2 101564 6789 1005 36 102469 0.0 2 93985 7185 1299 37 101986 0.0 2 93931 6852 1203 38 88289 0.0 2 81349 5989 951 39 91384 0.0 2 83980 6298 1106 40 87968 0.0 2 81063 5902 1003 41 85810 0.0 2 79585 5436 789 42 82412 0.0 2 76871 4870 671 43 73026 0.0 2 67328 4920 778 44 72521 0.0 2 67133 4755 633 45 84963 0.0 2 79659 4649 655 46 65401 0.0 2 60883 3896 622 47 46340 0.0 2 42573 3316 451 48 62890 0.0 2 58972 3433 485 49 44178 0.0 2 41087 2712 379 50 45926 0.0 2 42356 3132 438 51 59803 0.0 2 56373 3012 418 52 36947 0.0 2 34103 2470 374 53 36693 0.0 2 33991 2310 392 54 32194 0.0 2 29695 2182 317 55 36861 0.0 2 34493 2101 267 56 34269 0.0 2 31729 2174 366 57 30720 0.0 2 28494 1961 265 58 28380 0.0 2 26339 1776 265 59 26631 0.0 2 24733 1671 227 60 25170 0.0 2 23184 1700 286 61 24593 0.0 2 22735 1596 262 62 24060 0.0 2 22223 1561 276 63 22601 0.0 2 20779 1597 225 64 21649 0.0 2 19897 1515 237 65 22204 0.0 2 20390 1572 242 66 24131 0.0 2 22040 1770 321 67 35937 0.0 2 31177 4455 305 68 129003 0.0 2 123606 5023 374 69 47313 0.0 2 44432 2606 275 70 24870 0.0 2 23330 1358 182 71 12976 0.0 2 11952 904 120 72 8650 0.0 2 7938 594 118 73 6006 0.0 2 5428 500 78 74 4673 0.0 2 4170 423 80 75 3587 0.0 2 3246 292 49 76 2967 0.0 2 2644 260 63 77 2595 0.0 2 2287 248 60 78 2277 0.0 2 1988 229 60 79 1974 0.0 2 1740 196 38 80 1588 0.0 2 1403 150 35 81 1399 0.0 2 1220 147 32 82 1178 0.0 2 1020 129 29 83 1067 0.0 2 929 111 27 84 908 0.0 2 767 111 30 85 844 0.0 2 702 108 34 86 765 0.0 2 629 110 26 87 784 0.0 2 659 101 24 88 797 0.0 2 673 96 28 89 913 0.0 2 759 123 31 90 1123 0.0 2 931 153 39 91 1402 0.0 2 1155 188 59 92 2147 0.0 2 1810 255 82 93 4579 0.0 2 3823 597 159 94 13431 0.0 2 11681 1388 362 95 23741 0.0 2 20778 2421 542 96 11598 0.0 2 10136 1204 258 97 8556 0.0 2 7512 866 178 98 3633 0.0 2 3223 345 65 99 3883 0.0 2 3382 409 92 100 4128 0.0 2 3619 404 105 101 7880 0.0 2 6828 858 194 Successfully deleted temporary file EPI-167_S10_L002_R2_001.fastq.gz_qual_trimmed.fastq RUN STATISTICS FOR INPUT FILE: /gscratch/srlab/strigg/data/Pgenr/FASTQS/raw/EPI-167_S10_L002_R2_001.fastq.gz ============================================= 24859230 sequences processed in total Sequences were truncated to a varying degree because of deteriorating qualities (Phred score quality cutoff: 20): 2502338 (10.1%) The length threshold of paired-end sequences gets evaluated later on (in the validation step) RRBS reads trimmed by additional 2 bp when adapter contamination was detected: 0 (0.0%) Validate paired-end files EPI-167_S10_L002_R1_001_trimmed.fq.gz and EPI-167_S10_L002_R2_001_trimmed.fq.gz file_1: EPI-167_S10_L002_R1_001_trimmed.fq.gz, file_2: EPI-167_S10_L002_R2_001_trimmed.fq.gz >>>>> Now validing the length of the 2 paired-end infiles: EPI-167_S10_L002_R1_001_trimmed.fq.gz and EPI-167_S10_L002_R2_001_trimmed.fq.gz <<<<< Writing validated paired-end Read 1 reads to EPI-167_S10_L002_R1_001_val_1.fq.gz Writing validated paired-end Read 2 reads to EPI-167_S10_L002_R2_001_val_2.fq.gz Total number of sequences analysed: 24859230 Number of sequence pairs removed because at least one read was shorter than the length cutoff (20 bp): 375991 (1.51%) >>> Now running FastQC on the validated data EPI-167_S10_L002_R1_001_val_1.fq.gz<<< Started analysis of EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 5% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 10% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 15% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 20% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 25% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 30% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 35% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 40% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 45% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 50% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 55% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 60% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 65% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 70% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 75% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 80% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 85% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 90% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Approx 95% complete for EPI-167_S10_L002_R1_001_val_1.fq.gz Analysis complete for EPI-167_S10_L002_R1_001_val_1.fq.gz >>> Now running FastQC on the validated data EPI-167_S10_L002_R2_001_val_2.fq.gz<<< Started analysis of EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 5% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 10% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 15% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 20% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 25% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 30% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 35% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 40% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 45% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 50% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 55% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 60% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 65% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 70% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 75% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 80% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 85% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 90% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Approx 95% complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Analysis complete for EPI-167_S10_L002_R2_001_val_2.fq.gz Deleting both intermediate output files EPI-167_S10_L002_R1_001_trimmed.fq.gz and EPI-167_S10_L002_R2_001_trimmed.fq.gz ====================================================================================================