/var/spool/slurm/d/job2268224/slurm_script: line 20: fg: no job control Using an excessive number of cores has a diminishing return! It is recommended not to exceed 8 cores per trimming process (you asked for 8 cores). Please consider re-specifying Path to Cutadapt set as: '/gscratch/srlab/strigg/bin/anaconda3/bin/cutadapt' (user defined) Cutadapt seems to be working fine (tested command '/gscratch/srlab/strigg/bin/anaconda3/bin/cutadapt --version') Cutadapt version: 2.4 Could not detect version of Python used by Cutadapt from the first line of Cutadapt (but found this: >>>#!/bin/sh<<<) Letting the (modified) Cutadapt deal with the Python version instead Parallel gzip (pigz) detected. Proceeding with multicore (de)compression using 8 cores No quality encoding type selected. Assuming that the data provided uses Sanger encoded Phred scores (default) Output will be written into the directory: /gscratch/scrubbed/strigg/analyses/20200316/ AUTO-DETECTING ADAPTER TYPE =========================== Attempting to auto-detect adapter type from the first 1 million sequences of the first file (>> /gscratch/scrubbed/sr320/froger-raw/00_fastq/Meth13_R1_001.fastq.gz <<) Found perfect matches for the following adapter sequences: Adapter type Count Sequence Sequences analysed Percentage Illumina 457885 AGATCGGAAGAGC 1000000 45.79 Nextera 0 CTGTCTCTTATA 1000000 0.00 smallRNA 0 TGGAATTCTCGG 1000000 0.00 Using Illumina adapter for trimming (count: 457885). Second best hit was Nextera (count: 0) Writing report to '/gscratch/scrubbed/strigg/analyses/20200316/Meth13_R1_001.fastq.gz_trimming_report.txt' SUMMARISING RUN PARAMETERS ========================== Input filename: /gscratch/scrubbed/sr320/froger-raw/00_fastq/Meth13_R1_001.fastq.gz Trimming mode: paired-end Trim Galore version: 0.6.4_dev Cutadapt version: 2.4 Python version: could not detect Number of cores used for trimming: 8 Quality Phred score cutoff: 20 Quality encoding type selected: ASCII+33 Adapter sequence: 'AGATCGGAAGAGC' (Illumina TruSeq, Sanger iPCR; auto-detected) Maximum trimming error rate: 0.1 (default) Minimum required adapter overlap (stringency): 1 bp Minimum required sequence length for both reads before a sequence pair gets removed: 20 bp File was specified to be an MspI-digested RRBS sample. Read 1 sequences with adapter contamination will be trimmed a further 2 bp from their 3' end, and Read 2 sequences will be trimmed by 2 bp from their 5' end to remove potential methylation-biased bases from the end-repair reaction File was specified to be a non-directional MspI-digested RRBS sample. Sequences starting with either 'CAA' or 'CGA' will have the first 2 bp trimmed off to remove potential methylation-biased bases from the end-repair reaction Output file(s) will be GZIP compressed Cutadapt seems to be fairly up-to-date (version 2.4). Setting -j 8 >>> Now performing adaptive quality trimming with a Phred-score cutoff of: 20 <<< This is cutadapt 2.4 with Python 3.7.6 Command line parameters: -j 8 -e 0.1 -q 20 -a X /gscratch/scrubbed/sr320/froger-raw/00_fastq/Meth13_R1_001.fastq.gz Processing reads on 8 cores in single-end mode ... 10000000 sequences processed 20000000 sequences processed 30000000 sequences processed 40000000 sequences processed 50000000 sequences processed 60000000 sequences processed 70000000 sequences processed 80000000 sequences processed 90000000 sequences processed 100000000 sequences processed Finished in 323.80 s (3 us/read; 18.71 M reads/minute). === Summary === Total reads processed: 100,994,125 Reads with adapters: 0 (0.0%) Reads written (passing filters): 100,994,125 (100.0%) Total basepairs processed: 15,149,118,750 bp Quality-trimmed: 15,277,665 bp (0.1%) Total written (filtered): 15,133,841,085 bp (99.9%) === Adapter 1 === Sequence: X; Type: regular 3'; Length: 1; Trimmed: 0 times. >>> Quality trimming completed <<< 100994125 sequences processed in total Writing final adapter and quality trimmed output to Meth13_R1_001_trimmed.fq.gz >>> Now performing adapter trimming for the adapter sequence: 'AGATCGGAAGAGC' from file Meth13_R1_001.fastq.gz_qual_trimmed.fastq <<< 10000000 sequences processed 20000000 sequences processed 30000000 sequences processed 40000000 sequences processed 50000000 sequences processed 60000000 sequences processed 70000000 sequences processed 80000000 sequences processed 90000000 sequences processed 100000000 sequences processed This is cutadapt 2.4 with Python 3.7.6 Command line parameters: -j 8 -e 0.1 -O 1 -a AGATCGGAAGAGC /gscratch/scrubbed/strigg/analyses/20200316/Meth13_R1_001.fastq.gz_qual_trimmed.fastq Processing reads on 8 cores in single-end mode ... Finished in 372.47 s (4 us/read; 16.27 M reads/minute). === Summary === Total reads processed: 100,994,125 Reads with adapters: 75,452,441 (74.7%) Reads written (passing filters): 100,994,125 (100.0%) Total basepairs processed: 15,133,841,085 bp Total written (filtered): 11,286,581,533 bp (74.6%) === Adapter 1 === Sequence: AGATCGGAAGAGC; Type: regular 3'; Length: 13; Trimmed: 75452441 times. No. of allowed errors: 0-9 bp: 0; 10-13 bp: 1 Bases preceding removed adapters: A: 46.8% C: 3.5% G: 42.0% T: 7.8% none/other: 0.0% Overview of removed sequences length count expect max.err error counts 1 16889854 25248531.2 0 16889854 2 1387092 6312132.8 0 1387092 3 993122 1578033.2 0 993122 4 431060 394508.3 0 431060 5 275682 98627.1 0 275682 6 233401 24656.8 0 233401 7 293154 6164.2 0 293154 8 291627 1541.0 0 291627 9 282078 385.3 0 278508 3570 10 329147 96.3 1 300652 28495 11 581523 24.1 1 522909 58614 12 349882 6.0 1 315325 34557 13 313904 1.5 1 280320 33584 14 311573 1.5 1 277270 34303 15 260519 1.5 1 232317 28202 16 280060 1.5 1 248629 31431 17 309633 1.5 1 275006 34627 18 299523 1.5 1 268511 31012 19 308381 1.5 1 275762 32619 20 319544 1.5 1 285436 34108 21 307838 1.5 1 274585 33253 22 346274 1.5 1 309836 36438 23 436976 1.5 1 390674 46302 24 385479 1.5 1 342645 42834 25 336519 1.5 1 300157 36362 26 286169 1.5 1 255735 30434 27 302039 1.5 1 269115 32924 28 505249 1.5 1 451790 53459 29 356797 1.5 1 316417 40380 30 333714 1.5 1 298431 35283 31 348698 1.5 1 311129 37569 32 367127 1.5 1 327760 39367 33 317415 1.5 1 282607 34808 34 352429 1.5 1 314640 37789 35 326336 1.5 1 290643 35693 36 339179 1.5 1 301577 37602 37 481410 1.5 1 429087 52323 38 448729 1.5 1 395717 53012 39 414453 1.5 1 370021 44432 40 544488 1.5 1 486617 57871 41 545768 1.5 1 486251 59517 42 474348 1.5 1 425187 49161 43 335612 1.5 1 295805 39807 44 363737 1.5 1 321199 42538 45 612969 1.5 1 550568 62401 46 236931 1.5 1 210315 26616 47 340836 1.5 1 303534 37302 48 428571 1.5 1 381101 47470 49 789485 1.5 1 705703 83782 50 421164 1.5 1 373382 47782 51 489216 1.5 1 432238 56978 52 533011 1.5 1 475013 57998 53 470642 1.5 1 418275 52367 54 823609 1.5 1 732350 91259 55 528241 1.5 1 470022 58219 56 413780 1.5 1 367680 46100 57 485033 1.5 1 430576 54457 58 739339 1.5 1 657723 81616 59 419372 1.5 1 371464 47908 60 857360 1.5 1 765572 91788 61 470376 1.5 1 417741 52635 62 433854 1.5 1 384546 49308 63 401851 1.5 1 356696 45155 64 433832 1.5 1 385879 47953 65 358350 1.5 1 317317 41033 66 444050 1.5 1 393209 50841 67 582832 1.5 1 515092 67740 68 746325 1.5 1 660769 85556 69 670332 1.5 1 580864 89468 70 2073202 1.5 1 1871175 202027 71 121451 1.5 1 105945 15506 72 52792 1.5 1 43475 9317 73 538152 1.5 1 472151 66001 74 1756813 1.5 1 1560951 195862 75 441617 1.5 1 389006 52611 76 504090 1.5 1 445913 58177 77 565547 1.5 1 500475 65072 78 426572 1.5 1 377709 48863 79 456994 1.5 1 404267 52727 80 462729 1.5 1 408983 53746 81 405931 1.5 1 358406 47525 82 437959 1.5 1 387478 50481 83 408216 1.5 1 360003 48213 84 562518 1.5 1 497661 64857 85 514509 1.5 1 454185 60324 86 424696 1.5 1 374471 50225 87 445051 1.5 1 391943 53108 88 443537 1.5 1 390586 52951 89 546419 1.5 1 481197 65222 90 499277 1.5 1 439779 59498 91 566915 1.5 1 498750 68165 92 375931 1.5 1 329894 46037 93 500854 1.5 1 440092 60762 94 547235 1.5 1 480575 66660 95 391331 1.5 1 343323 48008 96 422734 1.5 1 370502 52232 97 536091 1.5 1 469971 66120 98 532528 1.5 1 466445 66083 99 488932 1.5 1 428065 60867 100 500021 1.5 1 436747 63274 101 389481 1.5 1 340084 49397 102 384198 1.5 1 336082 48116 103 467887 1.5 1 408747 59140 104 410457 1.5 1 357935 52522 105 398892 1.5 1 347934 50958 106 628657 1.5 1 548475 80182 107 497594 1.5 1 432357 65237 108 499014 1.5 1 433694 65320 109 553229 1.5 1 481173 72056 110 382989 1.5 1 332050 50939 111 364439 1.5 1 316458 47981 112 405699 1.5 1 351377 54322 113 356009 1.5 1 308400 47609 114 619887 1.5 1 538428 81459 115 1146598 1.5 1 996001 150597 116 330227 1.5 1 283825 46402 117 447731 1.5 1 387042 60689 118 395667 1.5 1 341652 54015 119 446331 1.5 1 385057 61274 120 254368 1.5 1 218804 35564 121 257448 1.5 1 221667 35781 122 238783 1.5 1 205563 33220 123 250549 1.5 1 215898 34651 124 240416 1.5 1 206474 33942 125 321804 1.5 1 276819 44985 126 183366 1.5 1 157670 25696 127 148680 1.5 1 127230 21450 128 93397 1.5 1 80218 13179 129 76162 1.5 1 65344 10818 130 57264 1.5 1 49107 8157 131 74918 1.5 1 64528 10390 132 32450 1.5 1 27800 4650 133 11091 1.5 1 9620 1471 134 4995 1.5 1 4285 710 135 1217 1.5 1 1029 188 136 490 1.5 1 426 64 137 53 1.5 1 49 4 138 8 1.5 1 7 1 139 9 1.5 1 9 140 9 1.5 1 6 3 141 6 1.5 1 4 2 142 1 1.5 1 1 145 1 1.5 1 1 146 1 1.5 1 0 1 147 2 1.5 1 0 2 148 86 1.5 1 67 19 149 7 1.5 1 6 1 150 327 1.5 1 286 41 Successfully deleted temporary file Meth13_R1_001.fastq.gz_qual_trimmed.fastq RUN STATISTICS FOR INPUT FILE: /gscratch/scrubbed/sr320/froger-raw/00_fastq/Meth13_R1_001.fastq.gz ============================================= 100994125 sequences processed in total Sequences were truncated to a varying degree because of deteriorating qualities (Phred score quality cutoff: 20): 5451098 (5.4%) The length threshold of paired-end sequences gets evaluated later on (in the validation step) RRBS reads trimmed by additional 2 bp when adapter contamination was detected: 36165065 (35.8%) RRBS reads trimmed by 2 bp at the start when read started with CAA (1139376) or CGA (49790870) in total: 50930246 (50.4%) Writing report to '/gscratch/scrubbed/strigg/analyses/20200316/Meth13_R2_001.fastq.gz_trimming_report.txt' SUMMARISING RUN PARAMETERS ========================== Input filename: /gscratch/scrubbed/sr320/froger-raw/00_fastq/Meth13_R2_001.fastq.gz Trimming mode: paired-end Trim Galore version: 0.6.4_dev Cutadapt version: 2.4 Python version: could not detect Number of cores used for trimming: 8 Quality Phred score cutoff: 20 Quality encoding type selected: ASCII+33 Adapter sequence: 'AGATCGGAAGAGC' (Illumina TruSeq, Sanger iPCR; auto-detected) Maximum trimming error rate: 0.1 (default) Minimum required adapter overlap (stringency): 1 bp Minimum required sequence length for both reads before a sequence pair gets removed: 20 bp File was specified to be an MspI-digested RRBS sample. Read 1 sequences with adapter contamination will be trimmed a further 2 bp from their 3' end, and Read 2 sequences will be trimmed by 2 bp from their 5' end to remove potential methylation-biased bases from the end-repair reaction File was specified to be a non-directional MspI-digested RRBS sample. Sequences starting with either 'CAA' or 'CGA' will have the first 2 bp trimmed off to remove potential methylation-biased bases from the end-repair reaction Output file(s) will be GZIP compressed Cutadapt seems to be fairly up-to-date (version 2.4). Setting -j -j 8 >>> Now performing adaptive quality trimming with a Phred-score cutoff of: 20 <<< This is cutadapt 2.4 with Python 3.7.6 Command line parameters: -j 8 -e 0.1 -q 20 -a X /gscratch/scrubbed/sr320/froger-raw/00_fastq/Meth13_R2_001.fastq.gz Processing reads on 8 cores in single-end mode ... 10000000 sequences processed 20000000 sequences processed 30000000 sequences processed 40000000 sequences processed 50000000 sequences processed 60000000 sequences processed 70000000 sequences processed 80000000 sequences processed 90000000 sequences processed 100000000 sequences processed Finished in 335.11 s (3 us/read; 18.08 M reads/minute). === Summary === Total reads processed: 100,994,125 Reads with adapters: 0 (0.0%) Reads written (passing filters): 100,994,125 (100.0%) Total basepairs processed: 15,149,118,750 bp Quality-trimmed: 116,375,081 bp (0.8%) Total written (filtered): 15,032,743,669 bp (99.2%) === Adapter 1 === Sequence: X; Type: regular 3'; Length: 1; Trimmed: 0 times. >>> Quality trimming completed <<< 100994125 sequences processed in total Writing final adapter and quality trimmed output to Meth13_R2_001_trimmed.fq.gz >>> Now performing adapter trimming for the adapter sequence: 'AGATCGGAAGAGC' from file Meth13_R2_001.fastq.gz_qual_trimmed.fastq <<< 10000000 sequences processed 20000000 sequences processed 30000000 sequences processed 40000000 sequences processed 50000000 sequences processed 60000000 sequences processed 70000000 sequences processed 80000000 sequences processed 90000000 sequences processed 100000000 sequences processed This is cutadapt 2.4 with Python 3.7.6 Command line parameters: -j 8 -e 0.1 -O 1 -a AGATCGGAAGAGC /gscratch/scrubbed/strigg/analyses/20200316/Meth13_R2_001.fastq.gz_qual_trimmed.fastq Processing reads on 8 cores in single-end mode ... Finished in 401.74 s (4 us/read; 15.08 M reads/minute). === Summary === Total reads processed: 100,994,125 Reads with adapters: 74,282,317 (73.6%) Reads written (passing filters): 100,994,125 (100.0%) Total basepairs processed: 15,032,743,669 bp Total written (filtered): 11,294,465,516 bp (75.1%) === Adapter 1 === Sequence: AGATCGGAAGAGC; Type: regular 3'; Length: 13; Trimmed: 74282317 times. No. of allowed errors: 0-9 bp: 0; 10-13 bp: 1 Bases preceding removed adapters: A: 46.4% C: 3.1% G: 42.4% T: 8.1% none/other: 0.0% Overview of removed sequences length count expect max.err error counts 1 16858905 25248531.2 0 16858905 2 1409178 6312132.8 0 1409178 3 1022247 1578033.2 0 1022247 4 449685 394508.3 0 449685 5 282584 98627.1 0 282584 6 231575 24656.8 0 231575 7 275852 6164.2 0 275852 8 305134 1541.0 0 305134 9 261671 385.3 0 259774 1897 10 318513 96.3 1 293638 24875 11 567085 24.1 1 515255 51830 12 364336 6.0 1 330294 34042 13 290786 1.5 1 258428 32358 14 303138 1.5 1 270634 32504 15 246093 1.5 1 220948 25145 16 285123 1.5 1 253722 31401 17 284664 1.5 1 255137 29527 18 286469 1.5 1 258114 28355 19 308210 1.5 1 277962 30248 20 324953 1.5 1 291571 33382 21 303171 1.5 1 270805 32366 22 350374 1.5 1 316862 33512 23 396907 1.5 1 358066 38841 24 381124 1.5 1 343563 37561 25 381056 1.5 1 343740 37316 26 245479 1.5 1 220057 25422 27 291422 1.5 1 259208 32214 28 487231 1.5 1 443161 44070 29 347094 1.5 1 312544 34550 30 331761 1.5 1 300206 31555 31 338587 1.5 1 306037 32550 32 358592 1.5 1 324285 34307 33 314593 1.5 1 283789 30804 34 342874 1.5 1 309361 33513 35 336170 1.5 1 304544 31626 36 344386 1.5 1 308783 35603 37 415065 1.5 1 375228 39837 38 459882 1.5 1 415569 44313 39 404887 1.5 1 365056 39831 40 571974 1.5 1 518878 53096 41 505738 1.5 1 457619 48119 42 365004 1.5 1 331197 33807 43 413898 1.5 1 374763 39135 44 357178 1.5 1 323405 33773 45 457513 1.5 1 412411 45102 46 435158 1.5 1 391388 43770 47 457055 1.5 1 415593 41462 48 395224 1.5 1 357064 38160 49 674244 1.5 1 611485 62759 50 434007 1.5 1 393088 40919 51 427500 1.5 1 385958 41542 52 527271 1.5 1 477474 49797 53 471108 1.5 1 426085 45023 54 923952 1.5 1 841152 82800 55 367371 1.5 1 332096 35275 56 430114 1.5 1 387358 42756 57 935761 1.5 1 849893 85868 58 412305 1.5 1 370915 41390 59 386265 1.5 1 348305 37960 60 1040895 1.5 1 947342 93553 61 455365 1.5 1 412374 42991 62 358198 1.5 1 316355 41843 63 1602264 1.5 1 1461599 140665 64 380614 1.5 1 342988 37626 65 211938 1.5 1 190908 21030 66 250630 1.5 1 223979 26651 67 816496 1.5 1 742960 73536 68 606413 1.5 1 549483 56930 69 513297 1.5 1 464468 48829 70 778232 1.5 1 706072 72160 71 638686 1.5 1 578942 59744 72 481606 1.5 1 435034 46572 73 985078 1.5 1 894565 90513 74 1294627 1.5 1 1176076 118551 75 402361 1.5 1 363629 38732 76 367554 1.5 1 332914 34640 77 298809 1.5 1 267530 31279 78 347369 1.5 1 313518 33851 79 412815 1.5 1 372637 40178 80 429933 1.5 1 387774 42159 81 384917 1.5 1 347266 37651 82 421423 1.5 1 380392 41031 83 424394 1.5 1 382335 42059 84 551836 1.5 1 497555 54281 85 529303 1.5 1 476883 52420 86 424156 1.5 1 381288 42868 87 432689 1.5 1 389037 43652 88 428451 1.5 1 385513 42938 89 518280 1.5 1 466734 51546 90 480143 1.5 1 432447 47696 91 534963 1.5 1 481319 53644 92 359938 1.5 1 323524 36414 93 478512 1.5 1 430743 47769 94 513875 1.5 1 462209 51666 95 371059 1.5 1 333246 37813 96 402062 1.5 1 361080 40982 97 514433 1.5 1 461861 52572 98 516253 1.5 1 463634 52619 99 484360 1.5 1 434891 49469 100 492619 1.5 1 440850 51769 101 379216 1.5 1 339118 40098 102 367605 1.5 1 329106 38499 103 441408 1.5 1 394943 46465 104 386261 1.5 1 345006 41255 105 380001 1.5 1 339706 40295 106 591703 1.5 1 528612 63091 107 472566 1.5 1 421694 50872 108 474327 1.5 1 422729 51598 109 522002 1.5 1 466969 55033 110 362933 1.5 1 323370 39563 111 347657 1.5 1 309474 38183 112 388527 1.5 1 345990 42537 113 339670 1.5 1 302340 37330 114 591254 1.5 1 526319 64935 115 1088418 1.5 1 968434 119984 116 319102 1.5 1 283133 35969 117 434552 1.5 1 386087 48465 118 390585 1.5 1 345582 45003 119 434222 1.5 1 384929 49293 120 249611 1.5 1 221221 28390 121 250773 1.5 1 222295 28478 122 229186 1.5 1 202720 26466 123 241833 1.5 1 213756 28077 124 232011 1.5 1 205157 26854 125 312472 1.5 1 276186 36286 126 178172 1.5 1 157076 21096 127 143853 1.5 1 126656 17197 128 91075 1.5 1 80088 10987 129 73099 1.5 1 64502 8597 130 55464 1.5 1 48732 6732 131 71783 1.5 1 63274 8509 132 30986 1.5 1 27253 3733 133 10709 1.5 1 9385 1324 134 4811 1.5 1 4208 603 135 1166 1.5 1 1010 156 136 476 1.5 1 425 51 137 53 1.5 1 50 3 138 11 1.5 1 8 3 139 8 1.5 1 8 140 6 1.5 1 6 141 6 1.5 1 5 1 142 1 1.5 1 1 143 1 1.5 1 1 145 1 1.5 1 1 146 1 1.5 1 1 148 83 1.5 1 78 5 149 11 1.5 1 10 1 150 304 1.5 1 276 28 Successfully deleted temporary file Meth13_R2_001.fastq.gz_qual_trimmed.fastq RUN STATISTICS FOR INPUT FILE: /gscratch/scrubbed/sr320/froger-raw/00_fastq/Meth13_R2_001.fastq.gz ============================================= 100994125 sequences processed in total Sequences were truncated to a varying degree because of deteriorating qualities (Phred score quality cutoff: 20): 11269639 (11.2%) The length threshold of paired-end sequences gets evaluated later on (in the validation step) RRBS reads trimmed by additional 2 bp when adapter contamination was detected: 37071977 (36.7%) RRBS reads trimmed by 2 bp at the start when read started with CAA (1082919) or CGA (46567288) in total: 47650207 (47.2%) Validate paired-end files Meth13_R1_001_trimmed.fq.gz and Meth13_R2_001_trimmed.fq.gz file_1: Meth13_R1_001_trimmed.fq.gz, file_2: Meth13_R2_001_trimmed.fq.gz >>>>> Now validing the length of the 2 paired-end infiles: Meth13_R1_001_trimmed.fq.gz and Meth13_R2_001_trimmed.fq.gz <<<<< Writing validated paired-end Read 1 reads to Meth13_R1_001_val_1.fq.gz Writing validated paired-end Read 2 reads to Meth13_R2_001_val_2.fq.gz Total number of sequences analysed: 100994125 Number of sequence pairs removed because at least one read was shorter than the length cutoff (20 bp): 387079 (0.38%) Deleting both intermediate output files Meth13_R1_001_trimmed.fq.gz and Meth13_R2_001_trimmed.fq.gz ====================================================================================================