MEGAHIT v1.1.3 --- [Wed Jan 2 14:26:04 2019] Start assembly. Number of CPU threads 28 --- --- [Wed Jan 2 14:26:04 2019] Available memory: 540350111744, used: 486315100569 --- [Wed Jan 2 14:26:04 2019] Converting reads to binaries --- /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core buildlib ./megahit_out/tmp/reads.lib ./megahit_out/tmp/reads.lib b' [read_lib_functions-inl.h : 209] Lib 0 (/gscratch/srlab/sam/data/metagenomics/P_generosa/Library_Geoduck_MG_1_S3_L002_R1_001_val_1_val_1_val_1.fq.gz,/gscratch/srlab/sam/data/metagenomics/P_generosa/Library_Geoduck_MG_1_S3_L002_R2_001_val_2_val_2_val_2.fq.gz): pe, 163236894 reads, 131 max length' b' [read_lib_functions-inl.h : 209] Lib 1 (/gscratch/srlab/sam/data/metagenomics/P_generosa/Library_Geoduck_MG_2_S4_L002_R1_001_val_1_val_1_val_1.fq.gz,/gscratch/srlab/sam/data/metagenomics/P_generosa/Library_Geoduck_MG_2_S4_L002_R2_001_val_2_val_2_val_2.fq.gz): pe, 160288380 reads, 131 max length' b' [read_lib_functions-inl.h : 209] Lib 2 (/gscratch/srlab/sam/data/metagenomics/P_generosa/Library_Geoduck_MG_3_S1_L002_R1_001_val_1_val_1_val_1.fq.gz,/gscratch/srlab/sam/data/metagenomics/P_generosa/Library_Geoduck_MG_3_S1_L002_R2_001_val_2_val_2_val_2.fq.gz): pe, 167467174 reads, 131 max length' b' [read_lib_functions-inl.h : 209] Lib 3 (/gscratch/srlab/sam/data/metagenomics/P_generosa/Library_Geoduck_MG_5_S6_L002_R1_001_val_1_val_1_val_1.fq.gz,/gscratch/srlab/sam/data/metagenomics/P_generosa/Library_Geoduck_MG_5_S6_L002_R2_001_val_2_val_2_val_2.fq.gz): pe, 141110330 reads, 131 max length' b' [read_lib_functions-inl.h : 209] Lib 4 (/gscratch/srlab/sam/data/metagenomics/P_generosa/Library_Geoduck_MG_6_S5_L002_R1_001_val_1_val_1_val_1.fq.gz,/gscratch/srlab/sam/data/metagenomics/P_generosa/Library_Geoduck_MG_6_S5_L002_R2_001_val_2_val_2_val_2.fq.gz): pe, 143722488 reads, 131 max length' b' [read_lib_functions-inl.h : 209] Lib 5 (/gscratch/srlab/sam/data/metagenomics/P_generosa/Library_Geoduck_MG_7_S2_L002_R1_001_val_1_val_1_val_1.fq.gz,/gscratch/srlab/sam/data/metagenomics/P_generosa/Library_Geoduck_MG_7_S2_L002_R2_001_val_2_val_2_val_2.fq.gz): pe, 172333246 reads, 131 max length' b' [utils.h : 126] Real: 958.0890\tuser: 848.5140\tsys: 85.3828\tmaxrss: 153540' --- [Wed Jan 2 14:42:02 2019] k list: 21,29,39,59,79,99,119,141 --- --- [Wed Jan 2 14:42:02 2019] Extracting solid (k+1)-mers for k = 21 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_sdbg_build count -k 21 -m 2 --host_mem 486315100569 --mem_flag 1 --gpu_mem 0 --output_prefix ./megahit_out/tmp/k21/21 --num_cpu_threads 28 --num_output_threads 9 --read_lib_file ./megahit_out/tmp/reads.lib b' [sdbg_builder.cpp : 112] Host memory to be used: 486315100569' b' [sdbg_builder.cpp : 113] Number CPU threads: 28' b' [cx1.h : 450] Preparing data...' b' [read_lib_functions-inl.h : 256] Before reading, sizeof seq_package: 34014649232' b' [read_lib_functions-inl.h : 260] After reading, sizeof seq_package: 34014649232' b' [cx1_kmer_count.cpp : 136] 948158512 reads, 131 max read length' b' [cx1.h : 457] Preparing data... Done. Time elapsed: 147.7016' b' [cx1.h : 464] Preparing partitions and initialing global data...' b' [cx1_kmer_count.cpp : 227] 2 words per substring, 2 words per edge' b' [cx1_kmer_count.cpp : 322] Set: 48404219284, 449503767225' b' [cx1_kmer_count.cpp : 356] 11441149645, 109984196 48404219284 449503767225' b' [cx1_kmer_count.cpp : 363] Memory for reads: 34840567424' b' [cx1_kmer_count.cpp : 364] max # lv.1 items = 11441149645' b' [cx1.h : 480] Preparing partitions and initialing global data... Done. Time elapsed: 50.3116' b' [cx1.h : 486] Start main loop...' b' [cx1.h : 515] Lv1 scanning from bucket 0 to 239' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 47.6719' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 79.3930' b' [cx1.h : 515] Lv1 scanning from bucket 239 to 1362' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 51.8779' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 161.2627' b' [cx1.h : 515] Lv1 scanning from bucket 1362 to 3661' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 54.6589' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 187.6076' b' [cx1.h : 515] Lv1 scanning from bucket 3661 to 7419' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 57.2469' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 197.3286' b' [cx1.h : 515] Lv1 scanning from bucket 7419 to 13068' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 59.0495' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 201.3897' b' [cx1.h : 515] Lv1 scanning from bucket 13068 to 21315' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 61.4707' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 202.9357' b' [cx1.h : 515] Lv1 scanning from bucket 21315 to 33711' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 64.5604' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 199.1393' b' [cx1.h : 515] Lv1 scanning from bucket 33711 to 58050' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 66.4966' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 188.3428' b' [cx1.h : 515] Lv1 scanning from bucket 58050 to 65536' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 45.5270' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 15.6625' b' [cx1.h : 607] Main loop done. Time elapsed: 1941.6232' b' [cx1.h : 613] Postprocessing...' b' [cx1_kmer_count.cpp : 860] Total number of candidate reads: 29176536(81770544)' b' [cx1_kmer_count.cpp : 871] Total number of solid edges: 4780772532' b' [cx1.h : 621] Postprocess done. Time elapsed: 20.2620' b' [utils.h : 126] Real: 2159.9970\tuser: 54653.6182\tsys: 130.7931\tmaxrss: 88779564' --- [Wed Jan 2 15:18:02 2019] Building graph for k = 21 --- /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_sdbg_build seq2sdbg --host_mem 486315100569 --mem_flag 1 --gpu_mem 0 --output_prefix ./megahit_out/tmp/k21/21 --num_cpu_threads 28 -k 21 --kmer_from 0 --num_edge_files 9 --input_prefix ./megahit_out/tmp/k21/21 --need_mercy b' [sdbg_builder.cpp : 339] Host memory to be used: 486315100569' b' [sdbg_builder.cpp : 340] Number CPU threads: 28' b' [cx1.h : 450] Preparing data...' b' [cx1_seq2sdbg.cpp : 394] Number edges: 4780772532' b' [cx1_seq2sdbg.cpp : 434] Bases to reserve: 131471244630, number contigs: 0, number multiplicity: 5975965665' b' [cx1_seq2sdbg.cpp : 440] Before reading, sizeof seq_package: 32867811168, multiplicity vector: 5975965665' b' [cx1_seq2sdbg.cpp : 455] Adding mercy edges...' b' [cx1_seq2sdbg.cpp : 373] Number of reads: 29176536, Number of mercy edges: 282394855' b' [cx1_seq2sdbg.cpp : 462] Done. Time elapsed: 305.6062' b' [cx1_seq2sdbg.cpp : 529] After reading, sizeof seq_package: 32867811168, multiplicity vector: 5975965665' b' [cx1.h : 457] Preparing data... Done. Time elapsed: 2005.8130' b' [cx1.h : 464] Preparing partitions and initialing global data...' b' [cx1_seq2sdbg.cpp : 740] Memory for sequence: 43864377862' b' [cx1_seq2sdbg.cpp : 741] max # lv.1 items = 4050533909' b' [cx1.h : 480] Preparing partitions and initialing global data... Done. Time elapsed: 10.4009' b' [cx1.h : 486] Start main loop...' b' [cx1.h : 515] Lv1 scanning from bucket 0 to 1214' b' [cx1.h : 528] Lv1 scanning done. Large diff: 254. Time elapsed: 9.8146' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 39.6478' b' [cx1.h : 515] Lv1 scanning from bucket 1214 to 4372' b' [cx1.h : 528] Lv1 scanning done. Large diff: 448. Time elapsed: 10.2289' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 46.2727' b' [cx1.h : 515] Lv1 scanning from bucket 4372 to 9393' b' [cx1.h : 528] Lv1 scanning done. Large diff: 520. Time elapsed: 10.5899' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 46.5948' b' [cx1.h : 515] Lv1 scanning from bucket 9393 to 16364' b' [cx1.h : 528] Lv1 scanning done. Large diff: 490. Time elapsed: 10.6451' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 45.1118' b' [cx1.h : 515] Lv1 scanning from bucket 16364 to 25501' b' [cx1.h : 528] Lv1 scanning done. Large diff: 486. Time elapsed: 11.0274' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 44.8397' b' [cx1.h : 515] Lv1 scanning from bucket 25501 to 37180' b' [cx1.h : 528] Lv1 scanning done. Large diff: 449. Time elapsed: 11.6706' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 45.0953' b' [cx1.h : 515] Lv1 scanning from bucket 37180 to 52600' b' [cx1.h : 528] Lv1 scanning done. Large diff: 357. Time elapsed: 11.9736' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 45.5373' b' [cx1.h : 515] Lv1 scanning from bucket 52600 to 65536' b' [cx1.h : 528] Lv1 scanning done. Large diff: 116. Time elapsed: 10.0152' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 25.5790' b' [cx1.h : 607] Main loop done. Time elapsed: 424.6449' b' [cx1.h : 613] Postprocessing...' b' [cx1_seq2sdbg.cpp :1139] Number of $ A C G T A- C- G- T-:' b' [cx1_seq2sdbg.cpp :1142] 91724221 2941708748 2022158216 2016161107 2954474805 100393916 40930302 40318264 101876223' b' [cx1_seq2sdbg.cpp :1151] Total number of edges: 10309745802' b' [cx1_seq2sdbg.cpp :1152] Total number of ONEs: 9934502876' b' [cx1_seq2sdbg.cpp :1153] Total number of $v edges: 91724221' b' [cx1.h : 621] Postprocess done. Time elapsed: 0.0752' b' [utils.h : 126] Real: 2441.0365\tuser: 19573.3768\tsys: 179.4339\tmaxrss: 53658056' --- [Wed Jan 2 15:58:46 2019] Assembling contigs from SdBG for k = 21 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core assemble -s ./megahit_out/tmp/k21/21 -o ./megahit_out/intermediate_contigs/k21 -t 28 --min_standalone 300.0 --prune_level 2 --merge_len 20 --merge_similar 0.95 --low_local_ratio 0.2 --min_depth 2 --bubble_level 2 --max_tip_len -1 --careful_bubble b' [assembler.cpp : 148] Loading succinct de Bruijn graph: ./megahit_out/tmp/k21/21 Done. Time elapsed: 1584.954125' b' [assembler.cpp : 152] Number of Edges: 10309745802; K value: 21' b' [assembler.cpp : 162] Number of CPU threads: 28' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 2; Accumulated tips removed: 12404388; time elapsed: 33.2350' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 4; Accumulated tips removed: 36787765; time elapsed: 39.8991' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 8; Accumulated tips removed: 59317293; time elapsed: 53.4981' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 16; Accumulated tips removed: 86658824; time elapsed: 85.9306' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 32; Accumulated tips removed: 109023288; time elapsed: 122.7427' b' [assembly_algorithms.cpp : 170] Removing tips with length less than 42; Accumulated tips removed: 115273467; time elapsed: 124.4106' b' [assembler.cpp : 179] Tips removal done! Time elapsed(sec): 460.083061' b' [assembler.cpp : 188] unitig graph size: 299257433, time for building: 1608.739475' b' [assembler.cpp : 211] Number of bubbles removed: 6988665, Time elapsed(sec): 79.507466' b' [assembler.cpp : 225] Number of complex bubbles removed: 1210328, Time elapsed(sec): 88.614734' b' [assembler.cpp : 243] Number unitigs disconnected: 121357511, time: 216.859559' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 1061250, time: 33.746011' b' [assembler.cpp : 200] Tips removed: 928740, time: 84.903750' b' [assembler.cpp : 211] Number of bubbles removed: 308453, Time elapsed(sec): 32.998910' b' [assembler.cpp : 225] Number of complex bubbles removed: 1261806, Time elapsed(sec): 59.015148' b' [assembler.cpp : 243] Number unitigs disconnected: 11772674, time: 47.157591' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 1007, time: 31.154521' b' [assembler.cpp : 200] Tips removed: 9915605, time: 87.894053' b' [assembler.cpp : 211] Number of bubbles removed: 396006, Time elapsed(sec): 31.526386' b' [assembler.cpp : 225] Number of complex bubbles removed: 73168, Time elapsed(sec): 52.296700' b' [assembler.cpp : 243] Number unitigs disconnected: 77781, time: 36.362695' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 61, time: 30.421257' b' [assembler.cpp : 200] Tips removed: 24287, time: 76.027906' b' [assembler.cpp : 211] Number of bubbles removed: 1768, Time elapsed(sec): 29.697324' b' [assembler.cpp : 225] Number of complex bubbles removed: 1845, Time elapsed(sec): 51.364155' b' [assembler.cpp : 243] Number unitigs disconnected: 1874, time: 36.389185' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 8, time: 30.582261' b' [assembler.cpp : 200] Tips removed: 645, time: 75.657374' b' [assembler.cpp : 211] Number of bubbles removed: 16, Time elapsed(sec): 29.544274' b' [assembler.cpp : 225] Number of complex bubbles removed: 71, Time elapsed(sec): 51.289254' b' [assembler.cpp : 243] Number unitigs disconnected: 53, time: 36.489808' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 2, time: 30.593217' b' [assembler.cpp : 132] Total length: 8506475631, N50: 42, Mean: 40, number of contigs: 211175868' b' [assembler.cpp : 133] Maximum length: 10144' b' [assembler.cpp : 290] Time to output: 378.013919' b' [assembler.cpp : 317] Number of local low depth unitigs removed: 7870281, complex bubbles removed: 273773, time: 2941.900196' b' [assembler.cpp : 132] Total length: 7111077820, N50: 39, Mean: 38, number of contigs: 183682455' b' [assembler.cpp : 133] Maximum length: 10144' b' [utils.h : 126] Real: 8524.1793\tuser: 183182.1952\tsys: 224.9385\tmaxrss: 48578588' --- [Wed Jan 2 18:20:51 2019] Local assembling k = 21 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core local -c ./megahit_out/intermediate_contigs/k21.contigs.fa -l ./megahit_out/tmp/reads.lib -t 28 -o ./megahit_out/intermediate_contigs/k21.local.fa --kmax 29 b' [local_assembler.cpp : 89] Number of contigs: 2017974, Mapper size: 71295080' b' [read_lib_functions-inl.h : 256] Before reading, sizeof seq_package: 34014649232' b' [read_lib_functions-inl.h : 260] After reading, sizeof seq_package: 34014649232' b' [local_assembler.cpp : 330] Lib 0, insert size: 239.24 sd: 84.61' b' [local_assembler.cpp : 330] Lib 1, insert size: 245.13 sd: 90.45' b' [local_assembler.cpp : 330] Lib 2, insert size: 241.03 sd: 91.97' b' [local_assembler.cpp : 330] Lib 3, insert size: 247.14 sd: 91.79' b' [local_assembler.cpp : 330] Lib 4, insert size: 240.04 sd: 88.25' b' [local_assembler.cpp : 330] Lib 5, insert size: 235.03 sd: 86.28' b' [local_assembler.cpp : 462] Lib 0: total 163236894 reads, aligned 36014513, added 19765776 reads for local assembly' b' [local_assembler.cpp : 462] Lib 1: total 160288380 reads, aligned 34072113, added 18030787 reads for local assembly' b' [local_assembler.cpp : 462] Lib 2: total 167467174 reads, aligned 27158444, added 15224036 reads for local assembly' b' [local_assembler.cpp : 462] Lib 3: total 141110330 reads, aligned 36202392, added 18001400 reads for local assembly' b' [local_assembler.cpp : 462] Lib 4: total 143722488 reads, aligned 30916504, added 16674962 reads for local assembly' b' [local_assembler.cpp : 462] Lib 5: total 172333246 reads, aligned 36526217, added 18756711 reads for local assembly' b' [utils.h : 126] Real: 1452.6070\tuser: 36750.9407\tsys: 76.0992\tmaxrss: 38943748' --- [Wed Jan 2 18:45:04 2019] Extracting iterative edges from k = 21 to 29 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core iterate -c ./megahit_out/intermediate_contigs/k21.contigs.fa -b ./megahit_out/intermediate_contigs/k21.bubble_seq.fa -t 28 -k 21 -s 8 -o ./megahit_out/tmp/k29/29 -r ./megahit_out/tmp/reads.lib.bin -f binary b' [iterate_edges.cpp : 570] Number of crusial kmers: 28030347' b' [iterate_edges.cpp : 570] Number of crusial kmers: 380069425' b' [iterate_edges.cpp : 443] Processed: 67108864, aligned: 49560456. Iterative edges: 340856357' b' [iterate_edges.cpp : 443] Processed: 134217728, aligned: 99231746. Iterative edges: 485988451' b' [iterate_edges.cpp : 443] Processed: 201326592, aligned: 149572854. Iterative edges: 624923184' b' [iterate_edges.cpp : 443] Processed: 268435456, aligned: 200305330. Iterative edges: 744238739' b' [iterate_edges.cpp : 443] Processed: 335544320, aligned: 250944149. Iterative edges: 864128456' b' [iterate_edges.cpp : 443] Processed: 402653184, aligned: 300484361. Iterative edges: 992013373' b' [iterate_edges.cpp : 443] Processed: 469762048, aligned: 350027932. Iterative edges: 1038591128' b' [iterate_edges.cpp : 443] Processed: 536870912, aligned: 400285465. Iterative edges: 1081282655' b' [iterate_edges.cpp : 443] Processed: 603979776, aligned: 450884479. Iterative edges: 1119585612' b' [iterate_edges.cpp : 443] Processed: 671088640, aligned: 502307221. Iterative edges: 1154957575' b' [iterate_edges.cpp : 443] Processed: 738197504, aligned: 554331384. Iterative edges: 1186231258' b' [iterate_edges.cpp : 443] Processed: 805306368, aligned: 606253755. Iterative edges: 1216929470' b' [iterate_edges.cpp : 443] Processed: 872415232, aligned: 657928462. Iterative edges: 1247477716' b' [iterate_edges.cpp : 443] Processed: 939524096, aligned: 709723639. Iterative edges: 1269395491' b' [iterate_edges.cpp : 447] Total: 948158512, aligned: 716430890. Iterative edges: 1271738829' b' [iterate_edges.cpp : 451] Writing iterative edges...' b' [utils.h : 126] Real: 1924.6977\tuser: 48003.2432\tsys: 64.3913\tmaxrss: 43738432' --- [Wed Jan 2 19:17:09 2019] Building graph for k = 29 --- /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_sdbg_build seq2sdbg --host_mem 486315100569 --mem_flag 1 --gpu_mem 0 --output_prefix ./megahit_out/tmp/k29/29 --num_cpu_threads 28 -k 29 --kmer_from 21 --num_edge_files 1 --input_prefix ./megahit_out/tmp/k29/29 --addi_contig ./megahit_out/intermediate_contigs/k21.addi.fa --local_contig ./megahit_out/intermediate_contigs/k21.local.fa --contig ./megahit_out/intermediate_contigs/k21.contigs.fa --bubble ./megahit_out/intermediate_contigs/k21.bubble_seq.fa b' [sdbg_builder.cpp : 339] Host memory to be used: 486315100569' b' [sdbg_builder.cpp : 340] Number CPU threads: 28' b' [cx1.h : 450] Preparing data...' b' [cx1_seq2sdbg.cpp : 394] Number edges: 1271738829' b' [cx1_seq2sdbg.cpp : 434] Bases to reserve: 54881019577, number contigs: 394793014, number multiplicity: 1666531843' b' [cx1_seq2sdbg.cpp : 440] Before reading, sizeof seq_package: 16878599016, multiplicity vector: 1666531843' b' [cx1_seq2sdbg.cpp : 529] After reading, sizeof seq_package: 16878599016, multiplicity vector: 1666531843' b' [cx1.h : 457] Preparing data... Done. Time elapsed: 134.8576' b' [cx1.h : 464] Preparing partitions and initialing global data...' b' [cx1_seq2sdbg.cpp : 740] Memory for sequence: 20010459416' b' [cx1_seq2sdbg.cpp : 741] max # lv.1 items = 2413460962' b' [cx1.h : 480] Preparing partitions and initialing global data... Done. Time elapsed: 30.2443' b' [cx1.h : 486] Start main loop...' b' [cx1.h : 515] Lv1 scanning from bucket 0 to 806' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 23.3462' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 32.5436' b' [cx1.h : 515] Lv1 scanning from bucket 806 to 3416' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 30.2498' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 44.3440' b' [cx1.h : 515] Lv1 scanning from bucket 3416 to 7912' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 35.2075' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 45.8159' b' [cx1.h : 515] Lv1 scanning from bucket 7912 to 14423' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 39.6143' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 44.4112' b' [cx1.h : 515] Lv1 scanning from bucket 14423 to 23204' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 44.6554' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 43.3346' b' [cx1.h : 515] Lv1 scanning from bucket 23204 to 34680' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 43.9457' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 42.3497' b' [cx1.h : 515] Lv1 scanning from bucket 34680 to 49997' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 47.4774' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 43.3446' b' [cx1.h : 515] Lv1 scanning from bucket 49997 to 65536' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 37.0424' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 29.8619' b' [cx1.h : 607] Main loop done. Time elapsed: 627.5454' b' [cx1.h : 613] Postprocessing...' b' [cx1_seq2sdbg.cpp :1139] Number of $ A C G T A- C- G- T-:' b' [cx1_seq2sdbg.cpp :1142] 87880902 2483785852 1687425348 1685332394 2487413678 15177851 11037421 8791766 16203073' b' [cx1_seq2sdbg.cpp :1151] Total number of edges: 8483048285' b' [cx1_seq2sdbg.cpp :1152] Total number of ONEs: 8343957272' b' [cx1_seq2sdbg.cpp :1153] Total number of $v edges: 87880902' b' [cx1.h : 621] Postprocess done. Time elapsed: 0.0611' b' [utils.h : 126] Real: 792.7498\tuser: 10401.0186\tsys: 82.7079\tmaxrss: 24942376' --- [Wed Jan 2 19:30:23 2019] Assembling contigs from SdBG for k = 29 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core assemble -s ./megahit_out/tmp/k29/29 -o ./megahit_out/intermediate_contigs/k29 -t 28 --min_standalone 300.0 --prune_level 2 --merge_len 20 --merge_similar 0.95 --low_local_ratio 0.2 --min_depth 2 --bubble_level 2 --max_tip_len -1 --careful_bubble b' [assembler.cpp : 148] Loading succinct de Bruijn graph: ./megahit_out/tmp/k29/29 Done. Time elapsed: 1334.843593' b' [assembler.cpp : 152] Number of Edges: 8483048285; K value: 29' b' [assembler.cpp : 162] Number of CPU threads: 28' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 2; Accumulated tips removed: 4323238; time elapsed: 30.2854' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 4; Accumulated tips removed: 13732717; time elapsed: 38.8832' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 8; Accumulated tips removed: 23988776; time elapsed: 56.7528' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 16; Accumulated tips removed: 42524422; time elapsed: 95.6526' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 32; Accumulated tips removed: 63531644; time elapsed: 153.8827' b' [assembly_algorithms.cpp : 170] Removing tips with length less than 58; Accumulated tips removed: 79938533; time elapsed: 199.4051' b' [assembler.cpp : 179] Tips removal done! Time elapsed(sec): 575.166965' b' [assembler.cpp : 188] unitig graph size: 61836248, time for building: 606.832117' b' [assembler.cpp : 211] Number of bubbles removed: 6115110, Time elapsed(sec): 69.638728' b' [assembler.cpp : 225] Number of complex bubbles removed: 1127667, Time elapsed(sec): 30.179962' b' [assembler.cpp : 243] Number unitigs disconnected: 2055098, time: 9.601468' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 76108, time: 6.743382' b' [assembler.cpp : 200] Tips removed: 106037, time: 16.577444' b' [assembler.cpp : 211] Number of bubbles removed: 84715, Time elapsed(sec): 8.087183' b' [assembler.cpp : 225] Number of complex bubbles removed: 66924, Time elapsed(sec): 13.346119' b' [assembler.cpp : 243] Number unitigs disconnected: 326269, time: 8.031777' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 843, time: 6.649642' b' [assembler.cpp : 200] Tips removed: 108584, time: 16.328013' b' [assembler.cpp : 211] Number of bubbles removed: 14143, Time elapsed(sec): 6.644457' b' [assembler.cpp : 225] Number of complex bubbles removed: 10119, Time elapsed(sec): 12.703991' b' [assembler.cpp : 243] Number unitigs disconnected: 43616, time: 7.686328' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 118, time: 6.599144' b' [assembler.cpp : 200] Tips removed: 4697, time: 16.185332' b' [assembler.cpp : 211] Number of bubbles removed: 323, Time elapsed(sec): 6.180147' b' [assembler.cpp : 225] Number of complex bubbles removed: 565, Time elapsed(sec): 12.515457' b' [assembler.cpp : 243] Number unitigs disconnected: 8078, time: 7.652331' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 10, time: 6.611097' b' [assembler.cpp : 200] Tips removed: 537, time: 16.190658' b' [assembler.cpp : 211] Number of bubbles removed: 18, Time elapsed(sec): 6.139247' b' [assembler.cpp : 225] Number of complex bubbles removed: 59, Time elapsed(sec): 12.481276' b' [assembler.cpp : 243] Number unitigs disconnected: 1758, time: 7.659713' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 4, time: 6.614411' b' [assembler.cpp : 132] Total length: 4658332985, N50: 129, Mean: 99, number of contigs: 46937146' b' [assembler.cpp : 133] Maximum length: 76927' b' [assembler.cpp : 290] Time to output: 151.711305' b' [assembler.cpp : 317] Number of local low depth unitigs removed: 1663122, complex bubbles removed: 117518, time: 613.443289' b' [assembler.cpp : 132] Total length: 2390926794, N50: 106, Mean: 81, number of contigs: 29381243' b' [assembler.cpp : 133] Maximum length: 76927' b' [utils.h : 126] Real: 3670.4844\tuser: 60778.9966\tsys: 115.0636\tmaxrss: 24097180' --- [Wed Jan 2 20:31:34 2019] Local assembling k = 29 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core local -c ./megahit_out/intermediate_contigs/k29.contigs.fa -l ./megahit_out/tmp/reads.lib -t 28 -o ./megahit_out/intermediate_contigs/k29.local.fa --kmax 39 b' [local_assembler.cpp : 89] Number of contigs: 3584865, Mapper size: 183674583' b' [read_lib_functions-inl.h : 256] Before reading, sizeof seq_package: 34014649232' b' [read_lib_functions-inl.h : 260] After reading, sizeof seq_package: 34014649232' b' [local_assembler.cpp : 330] Lib 0, insert size: 258.12 sd: 96.35' b' [local_assembler.cpp : 330] Lib 1, insert size: 260.46 sd: 99.95' b' [local_assembler.cpp : 330] Lib 2, insert size: 252.11 sd: 98.61' b' [local_assembler.cpp : 330] Lib 3, insert size: 261.31 sd: 100.45' b' [local_assembler.cpp : 330] Lib 4, insert size: 255.57 sd: 98.05' b' [local_assembler.cpp : 330] Lib 5, insert size: 250.12 sd: 96.12' b' [local_assembler.cpp : 462] Lib 0: total 163236894 reads, aligned 69997185, added 23669421 reads for local assembly' b' [local_assembler.cpp : 462] Lib 1: total 160288380 reads, aligned 66420087, added 23946193 reads for local assembly' b' [local_assembler.cpp : 462] Lib 2: total 167467174 reads, aligned 66914126, added 23029558 reads for local assembly' b' [local_assembler.cpp : 462] Lib 3: total 141110330 reads, aligned 65467399, added 22768721 reads for local assembly' b' [local_assembler.cpp : 462] Lib 4: total 143722488 reads, aligned 60809697, added 23264406 reads for local assembly' b' [local_assembler.cpp : 462] Lib 5: total 172333246 reads, aligned 72792020, added 26046090 reads for local assembly' b' [utils.h : 126] Real: 2182.2902\tuser: 57424.3849\tsys: 103.6187\tmaxrss: 44637164' --- [Wed Jan 2 21:07:57 2019] Extracting iterative edges from k = 29 to 39 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core iterate -c ./megahit_out/intermediate_contigs/k29.contigs.fa -b ./megahit_out/intermediate_contigs/k29.bubble_seq.fa -t 28 -k 29 -s 10 -o ./megahit_out/tmp/k39/39 -r ./megahit_out/tmp/reads.lib.bin -f binary b' [iterate_edges.cpp : 570] Number of crusial kmers: 19969999' b' [iterate_edges.cpp : 570] Number of crusial kmers: 82072279' b' [iterate_edges.cpp : 443] Processed: 67108864, aligned: 32073039. Iterative edges: 138599315' b' [iterate_edges.cpp : 443] Processed: 134217728, aligned: 64243259. Iterative edges: 188392830' b' [iterate_edges.cpp : 443] Processed: 201326592, aligned: 97468046. Iterative edges: 232781812' b' [iterate_edges.cpp : 443] Processed: 268435456, aligned: 131291865. Iterative edges: 268255206' b' [iterate_edges.cpp : 443] Processed: 335544320, aligned: 164573408. Iterative edges: 304180046' b' [iterate_edges.cpp : 443] Processed: 402653184, aligned: 194448265. Iterative edges: 345504437' b' [iterate_edges.cpp : 443] Processed: 469762048, aligned: 224293072. Iterative edges: 361427091' b' [iterate_edges.cpp : 443] Processed: 536870912, aligned: 257306373. Iterative edges: 373051790' b' [iterate_edges.cpp : 443] Processed: 603979776, aligned: 291711863. Iterative edges: 382689586' b' [iterate_edges.cpp : 443] Processed: 671088640, aligned: 327438826. Iterative edges: 392388054' b' [iterate_edges.cpp : 443] Processed: 738197504, aligned: 363988339. Iterative edges: 401277901' b' [iterate_edges.cpp : 443] Processed: 805306368, aligned: 400185279. Iterative edges: 410027143' b' [iterate_edges.cpp : 443] Processed: 872415232, aligned: 435711272. Iterative edges: 418724327' b' [iterate_edges.cpp : 443] Processed: 939524096, aligned: 471359000. Iterative edges: 425080944' b' [iterate_edges.cpp : 447] Total: 948158512, aligned: 476025065. Iterative edges: 425765721' b' [iterate_edges.cpp : 451] Writing iterative edges...' b' [utils.h : 126] Real: 1486.3663\tuser: 38234.9095\tsys: 36.9612\tmaxrss: 14481860' --- [Wed Jan 2 21:32:43 2019] Building graph for k = 39 --- /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_sdbg_build seq2sdbg --host_mem 486315100569 --mem_flag 1 --gpu_mem 0 --output_prefix ./megahit_out/tmp/k39/39 --num_cpu_threads 28 -k 39 --kmer_from 29 --num_edge_files 1 --input_prefix ./megahit_out/tmp/k39/39 --addi_contig ./megahit_out/intermediate_contigs/k29.addi.fa --local_contig ./megahit_out/intermediate_contigs/k29.local.fa --contig ./megahit_out/intermediate_contigs/k29.contigs.fa --bubble ./megahit_out/intermediate_contigs/k29.bubble_seq.fa b' [sdbg_builder.cpp : 339] Host memory to be used: 486315100569' b' [sdbg_builder.cpp : 340] Number CPU threads: 28' b' [cx1.h : 450] Preparing data...' b' [cx1_seq2sdbg.cpp : 394] Number edges: 425765721' b' [cx1_seq2sdbg.cpp : 434] Bases to reserve: 26082209089, number contigs: 82902273, number multiplicity: 508667994' b' [cx1_seq2sdbg.cpp : 440] Before reading, sizeof seq_package: 7183770468, multiplicity vector: 508667994' b' [cx1_seq2sdbg.cpp : 529] After reading, sizeof seq_package: 7183770468, multiplicity vector: 508667994' b' [cx1.h : 457] Preparing data... Done. Time elapsed: 67.6374' b' [cx1.h : 464] Preparing partitions and initialing global data...' b' [cx1_seq2sdbg.cpp : 740] Memory for sequence: 8346074526' b' [cx1_seq2sdbg.cpp : 741] max # lv.1 items = 2080281041' b' [cx1.h : 480] Preparing partitions and initialing global data... Done. Time elapsed: 30.5430' b' [cx1.h : 486] Start main loop...' b' [cx1.h : 515] Lv1 scanning from bucket 0 to 1099' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 27.7957' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 32.0332' b' [cx1.h : 515] Lv1 scanning from bucket 1099 to 4092' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 37.1672' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 42.2220' b' [cx1.h : 515] Lv1 scanning from bucket 4092 to 8888' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 40.1412' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 42.5798' b' [cx1.h : 515] Lv1 scanning from bucket 8888 to 15614' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 42.8345' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 42.2879' b' [cx1.h : 515] Lv1 scanning from bucket 15614 to 24550' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 44.0174' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 40.4792' b' [cx1.h : 515] Lv1 scanning from bucket 24550 to 36166' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 44.9411' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 40.8197' b' [cx1.h : 515] Lv1 scanning from bucket 36166 to 51765' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 43.8814' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 41.8413' b' [cx1.h : 515] Lv1 scanning from bucket 51765 to 65536' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 31.5982' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 24.5492' b' [cx1.h : 607] Main loop done. Time elapsed: 619.1902' b' [cx1.h : 613] Postprocessing...' b' [cx1_seq2sdbg.cpp :1139] Number of $ A C G T A- C- G- T-:' b' [cx1_seq2sdbg.cpp :1142] 39083682 2068040687 1433953030 1432178088 2070242191 7356301 4055030 4010696 7371298' b' [cx1_seq2sdbg.cpp :1151] Total number of edges: 7066291003' b' [cx1_seq2sdbg.cpp :1152] Total number of ONEs: 7004413996' b' [cx1_seq2sdbg.cpp :1153] Total number of $v edges: 39083682' b' [cx1.h : 621] Postprocess done. Time elapsed: 0.3139' b' [utils.h : 126] Real: 717.7119\tuser: 9531.9842\tsys: 51.4864\tmaxrss: 16359824' --- [Wed Jan 2 21:44:41 2019] Assembling contigs from SdBG for k = 39 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core assemble -s ./megahit_out/tmp/k39/39 -o ./megahit_out/intermediate_contigs/k39 -t 28 --min_standalone 300.0 --prune_level 2 --merge_len 20 --merge_similar 0.95 --low_local_ratio 0.2 --min_depth 2 --bubble_level 2 --max_tip_len -1 --careful_bubble b' [assembler.cpp : 148] Loading succinct de Bruijn graph: ./megahit_out/tmp/k39/39 Done. Time elapsed: 1131.848845' b' [assembler.cpp : 152] Number of Edges: 7066291003; K value: 39' b' [assembler.cpp : 162] Number of CPU threads: 28' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 2; Accumulated tips removed: 528448; time elapsed: 20.5410' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 4; Accumulated tips removed: 2366452; time elapsed: 24.5221' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 8; Accumulated tips removed: 4480907; time elapsed: 33.9028' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 16; Accumulated tips removed: 6996053; time elapsed: 56.0494' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 32; Accumulated tips removed: 10520061; time elapsed: 100.1161' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 64; Accumulated tips removed: 21582538; time elapsed: 167.2202' b' [assembly_algorithms.cpp : 170] Removing tips with length less than 78; Accumulated tips removed: 27210522; time elapsed: 153.9935' b' [assembler.cpp : 179] Tips removal done! Time elapsed(sec): 556.599869' b' [assembler.cpp : 188] unitig graph size: 34149490, time for building: 479.959559' b' [assembler.cpp : 211] Number of bubbles removed: 4258775, Time elapsed(sec): 71.162325' b' [assembler.cpp : 225] Number of complex bubbles removed: 802826, Time elapsed(sec): 26.303509' b' [assembler.cpp : 243] Number unitigs disconnected: 1102843, time: 5.112857' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 251457, time: 4.130739' b' [assembler.cpp : 200] Tips removed: 64881, time: 10.071802' b' [assembler.cpp : 211] Number of bubbles removed: 98706, Time elapsed(sec): 6.277846' b' [assembler.cpp : 225] Number of complex bubbles removed: 56394, Time elapsed(sec): 7.810657' b' [assembler.cpp : 243] Number unitigs disconnected: 106801, time: 4.144165' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 3938, time: 3.694284' b' [assembler.cpp : 200] Tips removed: 41956, time: 9.787038' b' [assembler.cpp : 211] Number of bubbles removed: 7577, Time elapsed(sec): 3.254878' b' [assembler.cpp : 225] Number of complex bubbles removed: 5761, Time elapsed(sec): 6.359604' b' [assembler.cpp : 243] Number unitigs disconnected: 12917, time: 4.036784' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 768, time: 3.667198' b' [assembler.cpp : 200] Tips removed: 1751, time: 9.628367' b' [assembler.cpp : 211] Number of bubbles removed: 113, Time elapsed(sec): 2.948078' b' [assembler.cpp : 225] Number of complex bubbles removed: 239, Time elapsed(sec): 6.225905' b' [assembler.cpp : 243] Number unitigs disconnected: 3129, time: 4.017807' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 172, time: 3.670635' b' [assembler.cpp : 200] Tips removed: 251, time: 9.588195' b' [assembler.cpp : 211] Number of bubbles removed: 10, Time elapsed(sec): 2.944580' b' [assembler.cpp : 225] Number of complex bubbles removed: 34, Time elapsed(sec): 6.218971' b' [assembler.cpp : 243] Number unitigs disconnected: 862, time: 4.012737' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 53, time: 3.675345' b' [assembler.cpp : 132] Total length: 3752069601, N50: 202, Mean: 158, number of contigs: 23606594' b' [assembler.cpp : 133] Maximum length: 228169' b' [assembler.cpp : 290] Time to output: 121.461396' b' [assembler.cpp : 317] Number of local low depth unitigs removed: 494535, complex bubbles removed: 44224, time: 346.663736' b' [assembler.cpp : 132] Total length: 1838906339, N50: 229, Mean: 134, number of contigs: 13651334' b' [assembler.cpp : 133] Maximum length: 228169' b' [utils.h : 126] Real: 2912.0381\tuser: 46467.4261\tsys: 86.9239\tmaxrss: 19594704' --- [Wed Jan 2 22:33:14 2019] Local assembling k = 39 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core local -c ./megahit_out/intermediate_contigs/k39.contigs.fa -l ./megahit_out/tmp/reads.lib -t 28 -o ./megahit_out/intermediate_contigs/k39.local.fa --kmax 59 b' [local_assembler.cpp : 89] Number of contigs: 3575528, Mapper size: 223973677' b' [read_lib_functions-inl.h : 256] Before reading, sizeof seq_package: 34014649232' b' [read_lib_functions-inl.h : 260] After reading, sizeof seq_package: 34014649232' b' [local_assembler.cpp : 330] Lib 0, insert size: 262.50 sd: 99.08' b' [local_assembler.cpp : 330] Lib 1, insert size: 266.54 sd: 103.38' b' [local_assembler.cpp : 330] Lib 2, insert size: 257.32 sd: 101.93' b' [local_assembler.cpp : 330] Lib 3, insert size: 266.66 sd: 103.93' b' [local_assembler.cpp : 330] Lib 4, insert size: 261.41 sd: 101.87' b' [local_assembler.cpp : 330] Lib 5, insert size: 255.21 sd: 99.50' b' [local_assembler.cpp : 462] Lib 0: total 163236894 reads, aligned 86207905, added 26810813 reads for local assembly' b' [local_assembler.cpp : 462] Lib 1: total 160288380 reads, aligned 83475345, added 27482408 reads for local assembly' b' [local_assembler.cpp : 462] Lib 2: total 167467174 reads, aligned 83847477, added 25931975 reads for local assembly' b' [local_assembler.cpp : 462] Lib 3: total 141110330 reads, aligned 80533009, added 24958043 reads for local assembly' b' [local_assembler.cpp : 462] Lib 4: total 143722488 reads, aligned 77356145, added 26451782 reads for local assembly' b' [local_assembler.cpp : 462] Lib 5: total 172333246 reads, aligned 92946429, added 29816686 reads for local assembly' b' [utils.h : 126] Real: 3078.3607\tuser: 82674.4644\tsys: 106.5002\tmaxrss: 45804552' --- [Wed Jan 2 23:24:33 2019] Extracting iterative edges from k = 39 to 59 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core iterate -c ./megahit_out/intermediate_contigs/k39.contigs.fa -b ./megahit_out/intermediate_contigs/k39.bubble_seq.fa -t 28 -k 39 -s 20 -o ./megahit_out/tmp/k59/59 -r ./megahit_out/tmp/reads.lib.bin -f binary b' [iterate_edges.cpp : 570] Number of crusial kmers: 13151558' b' [iterate_edges.cpp : 570] Number of crusial kmers: 40318371' b' [iterate_edges.cpp : 443] Processed: 67108864, aligned: 20182990. Iterative edges: 111785971' b' [iterate_edges.cpp : 443] Processed: 134217728, aligned: 40366545. Iterative edges: 157596927' b' [iterate_edges.cpp : 443] Processed: 201326592, aligned: 61162404. Iterative edges: 197491789' b' [iterate_edges.cpp : 443] Processed: 268435456, aligned: 82032634. Iterative edges: 229124076' b' [iterate_edges.cpp : 443] Processed: 335544320, aligned: 102403241. Iterative edges: 261392640' b' [iterate_edges.cpp : 443] Processed: 402653184, aligned: 119502220. Iterative edges: 304703805' b' [iterate_edges.cpp : 443] Processed: 469762048, aligned: 136504942. Iterative edges: 325734236' b' [iterate_edges.cpp : 443] Processed: 536870912, aligned: 156825875. Iterative edges: 338888866' b' [iterate_edges.cpp : 443] Processed: 603979776, aligned: 178466325. Iterative edges: 348866694' b' [iterate_edges.cpp : 443] Processed: 671088640, aligned: 201343069. Iterative edges: 359524179' b' [iterate_edges.cpp : 443] Processed: 738197504, aligned: 224778494. Iterative edges: 369658943' b' [iterate_edges.cpp : 443] Processed: 805306368, aligned: 247863703. Iterative edges: 379119858' b' [iterate_edges.cpp : 443] Processed: 872415232, aligned: 270144965. Iterative edges: 388477334' b' [iterate_edges.cpp : 443] Processed: 939524096, aligned: 292499272. Iterative edges: 395731974' b' [iterate_edges.cpp : 447] Total: 948158512, aligned: 295488742. Iterative edges: 396556142' b' [iterate_edges.cpp : 451] Writing iterative edges...' b' [utils.h : 126] Real: 1299.0971\tuser: 32510.3941\tsys: 38.6246\tmaxrss: 14148424' --- [Wed Jan 2 23:46:12 2019] Building graph for k = 59 --- /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_sdbg_build seq2sdbg --host_mem 486315100569 --mem_flag 1 --gpu_mem 0 --output_prefix ./megahit_out/tmp/k59/59 --num_cpu_threads 28 -k 59 --kmer_from 39 --num_edge_files 1 --input_prefix ./megahit_out/tmp/k59/59 --addi_contig ./megahit_out/intermediate_contigs/k39.addi.fa --local_contig ./megahit_out/intermediate_contigs/k39.local.fa --contig ./megahit_out/intermediate_contigs/k39.contigs.fa --bubble ./megahit_out/intermediate_contigs/k39.bubble_seq.fa b' [sdbg_builder.cpp : 339] Host memory to be used: 486315100569' b' [sdbg_builder.cpp : 340] Number CPU threads: 28' b' [cx1.h : 450] Preparing data...' b' [cx1_seq2sdbg.cpp : 394] Number edges: 396556142' b' [cx1_seq2sdbg.cpp : 434] Bases to reserve: 31911128192, number contigs: 44655650, number multiplicity: 441211792' b' [cx1_seq2sdbg.cpp : 440] Before reading, sizeof seq_package: 8335027260, multiplicity vector: 441211792' b' [cx1_seq2sdbg.cpp : 529] After reading, sizeof seq_package: 8335027260, multiplicity vector: 441211792' b' [cx1.h : 457] Preparing data... Done. Time elapsed: 62.9248' b' [cx1.h : 464] Preparing partitions and initialing global data...' b' [cx1_seq2sdbg.cpp : 740] Memory for sequence: 9440490882' b' [cx1_seq2sdbg.cpp : 741] max # lv.1 items = 1925569177' b' [cx1.h : 480] Preparing partitions and initialing global data... Done. Time elapsed: 33.2230' b' [cx1.h : 486] Start main loop...' b' [cx1.h : 515] Lv1 scanning from bucket 0 to 935' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 29.5466' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 27.0840' b' [cx1.h : 515] Lv1 scanning from bucket 935 to 3695' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 39.2081' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 39.0820' b' [cx1.h : 515] Lv1 scanning from bucket 3695 to 8242' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 42.9171' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 39.8185' b' [cx1.h : 515] Lv1 scanning from bucket 8242 to 14722' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 47.1400' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 39.3163' b' [cx1.h : 515] Lv1 scanning from bucket 14722 to 23453' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 46.7636' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 37.1380' b' [cx1.h : 515] Lv1 scanning from bucket 23453 to 34969' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 48.1120' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 36.2987' b' [cx1.h : 515] Lv1 scanning from bucket 34969 to 50611' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 50.3569' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 37.5493' b' [cx1.h : 515] Lv1 scanning from bucket 50611 to 65536' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 34.9266' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 24.1246' b' [cx1.h : 607] Main loop done. Time elapsed: 619.3836' b' [cx1.h : 613] Postprocessing...' b' [cx1_seq2sdbg.cpp :1139] Number of $ A C G T A- C- G- T-:' b' [cx1_seq2sdbg.cpp :1142] 26217770 1808458563 1231259090 1230171870 1810235057 3547231 2641825 2131329 3779208' b' [cx1_seq2sdbg.cpp :1151] Total number of edges: 6118441943' b' [cx1_seq2sdbg.cpp :1152] Total number of ONEs: 6080124580' b' [cx1_seq2sdbg.cpp :1153] Total number of $v edges: 26217770' b' [cx1.h : 621] Postprocess done. Time elapsed: 0.0590' b' [utils.h : 126] Real: 715.6163\tuser: 8641.5742\tsys: 56.4372\tmaxrss: 16993280' --- [Wed Jan 2 23:58:08 2019] Assembling contigs from SdBG for k = 59 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core assemble -s ./megahit_out/tmp/k59/59 -o ./megahit_out/intermediate_contigs/k59 -t 28 --min_standalone 300.0 --prune_level 2 --merge_len 20 --merge_similar 0.95 --low_local_ratio 0.2 --min_depth 2 --bubble_level 2 --max_tip_len -1 --careful_bubble b' [assembler.cpp : 148] Loading succinct de Bruijn graph: ./megahit_out/tmp/k59/59 Done. Time elapsed: 1050.093425' b' [assembler.cpp : 152] Number of Edges: 6118441943; K value: 59' b' [assembler.cpp : 162] Number of CPU threads: 28' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 2; Accumulated tips removed: 230474; time elapsed: 17.1636' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 4; Accumulated tips removed: 874086; time elapsed: 19.7507' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 8; Accumulated tips removed: 1916232; time elapsed: 26.2439' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 16; Accumulated tips removed: 3605621; time elapsed: 42.4623' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 32; Accumulated tips removed: 5576688; time elapsed: 71.9679' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 64; Accumulated tips removed: 12321983; time elapsed: 123.4360' b' [assembly_algorithms.cpp : 170] Removing tips with length less than 118; Accumulated tips removed: 23983955; time elapsed: 131.6301' b' [assembler.cpp : 179] Tips removal done! Time elapsed(sec): 432.878151' b' [assembler.cpp : 188] unitig graph size: 14934034, time for building: 337.350211' b' [assembler.cpp : 211] Number of bubbles removed: 2043186, Time elapsed(sec): 59.977468' b' [assembler.cpp : 225] Number of complex bubbles removed: 462462, Time elapsed(sec): 21.776879' b' [assembler.cpp : 243] Number unitigs disconnected: 1078326, time: 2.791913' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 369551, time: 2.571233' b' [assembler.cpp : 200] Tips removed: 20441, time: 4.004055' b' [assembler.cpp : 211] Number of bubbles removed: 85310, Time elapsed(sec): 4.543432' b' [assembler.cpp : 225] Number of complex bubbles removed: 57841, Time elapsed(sec): 5.328297' b' [assembler.cpp : 243] Number unitigs disconnected: 73320, time: 1.838589' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 6198, time: 1.613997' b' [assembler.cpp : 200] Tips removed: 26310, time: 3.805437' b' [assembler.cpp : 211] Number of bubbles removed: 3492, Time elapsed(sec): 1.356710' b' [assembler.cpp : 225] Number of complex bubbles removed: 3783, Time elapsed(sec): 2.833240' b' [assembler.cpp : 243] Number unitigs disconnected: 7712, time: 1.761240' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 995, time: 1.601428' b' [assembler.cpp : 200] Tips removed: 963, time: 3.896944' b' [assembler.cpp : 211] Number of bubbles removed: 72, Time elapsed(sec): 1.136377' b' [assembler.cpp : 225] Number of complex bubbles removed: 174, Time elapsed(sec): 2.623296' b' [assembler.cpp : 243] Number unitigs disconnected: 1644, time: 1.752418' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 227, time: 1.606658' b' [assembler.cpp : 200] Tips removed: 132, time: 3.717711' b' [assembler.cpp : 211] Number of bubbles removed: 10, Time elapsed(sec): 1.130229' b' [assembler.cpp : 225] Number of complex bubbles removed: 22, Time elapsed(sec): 2.629293' b' [assembler.cpp : 243] Number unitigs disconnected: 438, time: 1.749973' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 69, time: 1.597162' b' [assembler.cpp : 132] Total length: 2755346982, N50: 523, Mean: 317, number of contigs: 8681545' b' [assembler.cpp : 133] Maximum length: 495048' b' [assembler.cpp : 290] Time to output: 91.403132' b' [assembler.cpp : 317] Number of local low depth unitigs removed: 155574, complex bubbles removed: 14229, time: 140.059848' b' [assembler.cpp : 132] Total length: 1314513965, N50: 619, Mean: 267, number of contigs: 4916438' b' [assembler.cpp : 133] Maximum length: 495048' b' [utils.h : 126] Real: 2232.3157\tuser: 30672.7636\tsys: 73.8049\tmaxrss: 15227012' --- [Thu Jan 3 00:35:22 2019] Local assembling k = 59 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core local -c ./megahit_out/intermediate_contigs/k59.contigs.fa -l ./megahit_out/tmp/reads.lib -t 28 -o ./megahit_out/intermediate_contigs/k59.local.fa --kmax 79 b' [local_assembler.cpp : 89] Number of contigs: 3774608, Mapper size: 260197397' b' [read_lib_functions-inl.h : 256] Before reading, sizeof seq_package: 34014649232' b' [read_lib_functions-inl.h : 260] After reading, sizeof seq_package: 34014649232' b' [local_assembler.cpp : 330] Lib 0, insert size: 265.98 sd: 101.65' b' [local_assembler.cpp : 330] Lib 1, insert size: 270.03 sd: 106.09' b' [local_assembler.cpp : 330] Lib 2, insert size: 262.25 sd: 104.81' b' [local_assembler.cpp : 330] Lib 3, insert size: 270.68 sd: 106.99' b' [local_assembler.cpp : 330] Lib 4, insert size: 265.85 sd: 105.36' b' [local_assembler.cpp : 330] Lib 5, insert size: 259.70 sd: 102.81' b' [local_assembler.cpp : 462] Lib 0: total 163236894 reads, aligned 94926141, added 27092767 reads for local assembly' b' [local_assembler.cpp : 462] Lib 1: total 160288380 reads, aligned 91790536, added 27288063 reads for local assembly' b' [local_assembler.cpp : 462] Lib 2: total 167467174 reads, aligned 89796256, added 21956815 reads for local assembly' b' [local_assembler.cpp : 462] Lib 3: total 141110330 reads, aligned 87027126, added 24112080 reads for local assembly' b' [local_assembler.cpp : 462] Lib 4: total 143722488 reads, aligned 85957664, added 26608423 reads for local assembly' b' [local_assembler.cpp : 462] Lib 5: total 172333246 reads, aligned 103001504, added 29572312 reads for local assembly' b' [utils.h : 126] Real: 3784.1006\tuser: 102301.1608\tsys: 115.3267\tmaxrss: 46954152' --- [Thu Jan 3 01:38:26 2019] Extracting iterative edges from k = 59 to 79 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core iterate -c ./megahit_out/intermediate_contigs/k59.contigs.fa -b ./megahit_out/intermediate_contigs/k59.bubble_seq.fa -t 28 -k 59 -s 20 -o ./megahit_out/tmp/k79/79 -r ./megahit_out/tmp/reads.lib.bin -f binary b' [iterate_edges.cpp : 570] Number of crusial kmers: 6366764' b' [iterate_edges.cpp : 570] Number of crusial kmers: 15931215' b' [iterate_edges.cpp : 443] Processed: 67108864, aligned: 9761783. Iterative edges: 42001330' b' [iterate_edges.cpp : 443] Processed: 134217728, aligned: 19426655. Iterative edges: 59487092' b' [iterate_edges.cpp : 443] Processed: 201326592, aligned: 29474998. Iterative edges: 74654124' b' [iterate_edges.cpp : 443] Processed: 268435456, aligned: 39428189. Iterative edges: 86085457' b' [iterate_edges.cpp : 443] Processed: 335544320, aligned: 49025301. Iterative edges: 96747858' b' [iterate_edges.cpp : 443] Processed: 402653184, aligned: 56432416. Iterative edges: 110758195' b' [iterate_edges.cpp : 443] Processed: 469762048, aligned: 63715312. Iterative edges: 118214932' b' [iterate_edges.cpp : 443] Processed: 536870912, aligned: 73422365. Iterative edges: 123326822' b' [iterate_edges.cpp : 443] Processed: 603979776, aligned: 83995930. Iterative edges: 127098105' b' [iterate_edges.cpp : 443] Processed: 671088640, aligned: 95371389. Iterative edges: 131122161' b' [iterate_edges.cpp : 443] Processed: 738197504, aligned: 106937468. Iterative edges: 134799418' b' [iterate_edges.cpp : 443] Processed: 805306368, aligned: 118301036. Iterative edges: 138053493' b' [iterate_edges.cpp : 443] Processed: 872415232, aligned: 129081916. Iterative edges: 141031369' b' [iterate_edges.cpp : 443] Processed: 939524096, aligned: 139841658. Iterative edges: 143338414' b' [iterate_edges.cpp : 447] Total: 948158512, aligned: 141325268. Iterative edges: 143624088' b' [iterate_edges.cpp : 451] Writing iterative edges...' b' [utils.h : 126] Real: 1153.1330\tuser: 29203.8868\tsys: 32.9892\tmaxrss: 6569452' --- [Thu Jan 3 01:57:40 2019] Building graph for k = 79 --- /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_sdbg_build seq2sdbg --host_mem 486315100569 --mem_flag 1 --gpu_mem 0 --output_prefix ./megahit_out/tmp/k79/79 --num_cpu_threads 28 -k 79 --kmer_from 59 --num_edge_files 1 --input_prefix ./megahit_out/tmp/k79/79 --addi_contig ./megahit_out/intermediate_contigs/k59.addi.fa --local_contig ./megahit_out/intermediate_contigs/k59.local.fa --contig ./megahit_out/intermediate_contigs/k59.contigs.fa --bubble ./megahit_out/intermediate_contigs/k59.bubble_seq.fa b' [sdbg_builder.cpp : 339] Host memory to be used: 486315100569' b' [sdbg_builder.cpp : 340] Number CPU threads: 28' b' [cx1.h : 450] Preparing data...' b' [cx1_seq2sdbg.cpp : 394] Number edges: 143624088' b' [cx1_seq2sdbg.cpp : 434] Bases to reserve: 18144134709, number contigs: 19965059, number multiplicity: 163589147' b' [cx1_seq2sdbg.cpp : 440] Before reading, sizeof seq_package: 4695754160, multiplicity vector: 163589147' b' [cx1_seq2sdbg.cpp : 529] After reading, sizeof seq_package: 9231787840, multiplicity vector: 163589147' b' [cx1.h : 457] Preparing data... Done. Time elapsed: 40.4877' b' [cx1.h : 464] Preparing partitions and initialing global data...' b' [cx1_seq2sdbg.cpp : 740] Memory for sequence: 9695152072' b' [cx1_seq2sdbg.cpp : 741] max # lv.1 items = 1557705068' b' [cx1.h : 480] Preparing partitions and initialing global data... Done. Time elapsed: 20.4590' b' [cx1.h : 486] Start main loop...' b' [cx1.h : 515] Lv1 scanning from bucket 0 to 735' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 19.1970' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 20.0612' b' [cx1.h : 515] Lv1 scanning from bucket 735 to 3165' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 23.8737' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 32.0368' b' [cx1.h : 515] Lv1 scanning from bucket 3165 to 7340' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 26.2866' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 33.1168' b' [cx1.h : 515] Lv1 scanning from bucket 7340 to 13422' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 28.3113' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 33.3521' b' [cx1.h : 515] Lv1 scanning from bucket 13422 to 21784' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 29.5631' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 31.6847' b' [cx1.h : 515] Lv1 scanning from bucket 21784 to 33050' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 31.5522' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 31.1249' b' [cx1.h : 515] Lv1 scanning from bucket 33050 to 48619' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 29.9907' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 32.0266' b' [cx1.h : 515] Lv1 scanning from bucket 48619 to 65536' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 23.7583' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 22.3544' b' [cx1.h : 607] Main loop done. Time elapsed: 448.2916' b' [cx1.h : 613] Postprocessing...' b' [cx1_seq2sdbg.cpp :1139] Number of $ A C G T A- C- G- T-:' b' [cx1_seq2sdbg.cpp :1142] 11590402 1523213554 1002332753 1001597550 1524399165 2248302 1244393 1550099 2103543' b' [cx1_seq2sdbg.cpp :1151] Total number of edges: 5070279761' b' [cx1_seq2sdbg.cpp :1152] Total number of ONEs: 5051543022' b' [cx1_seq2sdbg.cpp :1153] Total number of $v edges: 11590402' b' [cx1.h : 621] Postprocess done. Time elapsed: 0.0476' b' [utils.h : 126] Real: 509.3001\tuser: 7184.8578\tsys: 40.4597\tmaxrss: 11573652' --- [Thu Jan 3 02:06:09 2019] Assembling contigs from SdBG for k = 79 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core assemble -s ./megahit_out/tmp/k79/79 -o ./megahit_out/intermediate_contigs/k79 -t 28 --min_standalone 300.0 --prune_level 2 --merge_len 20 --merge_similar 0.95 --low_local_ratio 0.2 --min_depth 2 --bubble_level 2 --max_tip_len -1 --careful_bubble b' [assembler.cpp : 148] Loading succinct de Bruijn graph: ./megahit_out/tmp/k79/79 Done. Time elapsed: 827.703410' b' [assembler.cpp : 152] Number of Edges: 5070279761; K value: 79' b' [assembler.cpp : 162] Number of CPU threads: 28' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 2; Accumulated tips removed: 105428; time elapsed: 13.5695' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 4; Accumulated tips removed: 386399; time elapsed: 14.5770' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 8; Accumulated tips removed: 853566; time elapsed: 17.7735' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 16; Accumulated tips removed: 1677339; time elapsed: 24.8764' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 32; Accumulated tips removed: 2687047; time elapsed: 37.4215' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 64; Accumulated tips removed: 4759388; time elapsed: 60.8589' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 128; Accumulated tips removed: 8555011; time elapsed: 90.4885' b' [assembly_algorithms.cpp : 170] Removing tips with length less than 158; Accumulated tips removed: 9866108; time elapsed: 82.1380' b' [assembler.cpp : 179] Tips removal done! Time elapsed(sec): 341.886600' b' [assembler.cpp : 188] unitig graph size: 8853247, time for building: 300.441624' b' [assembler.cpp : 211] Number of bubbles removed: 1016221, Time elapsed(sec): 46.873223' b' [assembler.cpp : 225] Number of complex bubbles removed: 243097, Time elapsed(sec): 17.217286' b' [assembler.cpp : 243] Number unitigs disconnected: 1044474, time: 2.193614' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 359844, time: 2.324396' b' [assembler.cpp : 200] Tips removed: 18165, time: 2.642370' b' [assembler.cpp : 211] Number of bubbles removed: 60547, Time elapsed(sec): 3.452972' b' [assembler.cpp : 225] Number of complex bubbles removed: 48095, Time elapsed(sec): 4.368816' b' [assembler.cpp : 243] Number unitigs disconnected: 58075, time: 1.190859' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 12430, time: 1.280679' b' [assembler.cpp : 200] Tips removed: 30290, time: 2.499504' b' [assembler.cpp : 211] Number of bubbles removed: 2015, Time elapsed(sec): 0.744100' b' [assembler.cpp : 225] Number of complex bubbles removed: 2972, Time elapsed(sec): 1.835836' b' [assembler.cpp : 243] Number unitigs disconnected: 5583, time: 1.109549' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 2396, time: 1.223728' b' [assembler.cpp : 200] Tips removed: 1725, time: 2.371841' b' [assembler.cpp : 211] Number of bubbles removed: 80, Time elapsed(sec): 0.600951' b' [assembler.cpp : 225] Number of complex bubbles removed: 138, Time elapsed(sec): 1.556909' b' [assembler.cpp : 243] Number unitigs disconnected: 1002, time: 1.104370' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 537, time: 1.219646' b' [assembler.cpp : 200] Tips removed: 243, time: 2.371206' b' [assembler.cpp : 211] Number of bubbles removed: 14, Time elapsed(sec): 0.602443' b' [assembler.cpp : 225] Number of complex bubbles removed: 21, Time elapsed(sec): 1.549567' b' [assembler.cpp : 243] Number unitigs disconnected: 211, time: 1.105409' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 119, time: 1.221618' b' [assembler.cpp : 132] Total length: 2459897327, N50: 823, Mean: 507, number of contigs: 4843321' b' [assembler.cpp : 133] Maximum length: 891948' b' [assembler.cpp : 290] Time to output: 84.840775' b' [assembler.cpp : 317] Number of local low depth unitigs removed: 81460, complex bubbles removed: 7249, time: 107.372977' b' [assembler.cpp : 132] Total length: 1039291793, N50: 1070, Mean: 430, number of contigs: 2413986' b' [assembler.cpp : 133] Maximum length: 891948' b' [utils.h : 126] Real: 1799.5299\tuser: 25291.9767\tsys: 62.0163\tmaxrss: 12875508' --- [Thu Jan 3 02:36:10 2019] Local assembling k = 79 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core local -c ./megahit_out/intermediate_contigs/k79.contigs.fa -l ./megahit_out/tmp/reads.lib -t 28 -o ./megahit_out/intermediate_contigs/k79.local.fa --kmax 99 b' [local_assembler.cpp : 89] Number of contigs: 3202831, Mapper size: 266998293' b' [read_lib_functions-inl.h : 256] Before reading, sizeof seq_package: 34014649232' b' [read_lib_functions-inl.h : 260] After reading, sizeof seq_package: 34014649232' b' [local_assembler.cpp : 330] Lib 0, insert size: 270.80 sd: 103.62' b' [local_assembler.cpp : 330] Lib 1, insert size: 275.67 sd: 108.50' b' [local_assembler.cpp : 330] Lib 2, insert size: 266.36 sd: 106.83' b' [local_assembler.cpp : 330] Lib 3, insert size: 275.98 sd: 108.97' b' [local_assembler.cpp : 330] Lib 4, insert size: 271.68 sd: 108.22' b' [local_assembler.cpp : 330] Lib 5, insert size: 264.94 sd: 105.25' b' [local_assembler.cpp : 462] Lib 0: total 163236894 reads, aligned 86699510, added 16673817 reads for local assembly' b' [local_assembler.cpp : 462] Lib 1: total 160288380 reads, aligned 83428538, added 16677540 reads for local assembly' b' [local_assembler.cpp : 462] Lib 2: total 167467174 reads, aligned 85927620, added 14660972 reads for local assembly' b' [local_assembler.cpp : 462] Lib 3: total 141110330 reads, aligned 79016830, added 14481300 reads for local assembly' b' [local_assembler.cpp : 462] Lib 4: total 143722488 reads, aligned 77851822, added 16603220 reads for local assembly' b' [local_assembler.cpp : 462] Lib 5: total 172333246 reads, aligned 93546124, added 18252645 reads for local assembly' b' [utils.h : 126] Real: 2577.7226\tuser: 69023.8216\tsys: 78.1537\tmaxrss: 46995248' --- [Thu Jan 3 03:19:07 2019] Extracting iterative edges from k = 79 to 99 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core iterate -c ./megahit_out/intermediate_contigs/k79.contigs.fa -b ./megahit_out/intermediate_contigs/k79.bubble_seq.fa -t 28 -k 79 -s 20 -o ./megahit_out/tmp/k99/99 -r ./megahit_out/tmp/reads.lib.bin -f binary b' [iterate_edges.cpp : 570] Number of crusial kmers: 3206802' b' [iterate_edges.cpp : 570] Number of crusial kmers: 7808113' b' [iterate_edges.cpp : 443] Processed: 67108864, aligned: 4607342. Iterative edges: 17726070' b' [iterate_edges.cpp : 443] Processed: 134217728, aligned: 9155272. Iterative edges: 25986526' b' [iterate_edges.cpp : 443] Processed: 201326592, aligned: 13927221. Iterative edges: 33259690' b' [iterate_edges.cpp : 443] Processed: 268435456, aligned: 18639896. Iterative edges: 38733825' b' [iterate_edges.cpp : 443] Processed: 335544320, aligned: 23077765. Iterative edges: 43230823' b' [iterate_edges.cpp : 443] Processed: 402653184, aligned: 25998323. Iterative edges: 48240280' b' [iterate_edges.cpp : 443] Processed: 469762048, aligned: 28853634. Iterative edges: 51359256' b' [iterate_edges.cpp : 443] Processed: 536870912, aligned: 33368758. Iterative edges: 54026420' b' [iterate_edges.cpp : 443] Processed: 603979776, aligned: 38506520. Iterative edges: 56096245' b' [iterate_edges.cpp : 443] Processed: 671088640, aligned: 44001148. Iterative edges: 58236303' b' [iterate_edges.cpp : 443] Processed: 738197504, aligned: 49520767. Iterative edges: 60152714' b' [iterate_edges.cpp : 443] Processed: 805306368, aligned: 54917225. Iterative edges: 61798340' b' [iterate_edges.cpp : 443] Processed: 872415232, aligned: 60018498. Iterative edges: 63246791' b' [iterate_edges.cpp : 443] Processed: 939524096, aligned: 65085492. Iterative edges: 64400614' b' [iterate_edges.cpp : 447] Total: 948158512, aligned: 65805342. Iterative edges: 64551254' b' [iterate_edges.cpp : 451] Writing iterative edges...' b' [utils.h : 126] Real: 777.4945\tuser: 19775.4565\tsys: 24.2879\tmaxrss: 3436548' --- [Thu Jan 3 03:32:05 2019] Building graph for k = 99 --- /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_sdbg_build seq2sdbg --host_mem 486315100569 --mem_flag 1 --gpu_mem 0 --output_prefix ./megahit_out/tmp/k99/99 --num_cpu_threads 28 -k 99 --kmer_from 79 --num_edge_files 1 --input_prefix ./megahit_out/tmp/k99/99 --addi_contig ./megahit_out/intermediate_contigs/k79.addi.fa --local_contig ./megahit_out/intermediate_contigs/k79.local.fa --contig ./megahit_out/intermediate_contigs/k79.contigs.fa --bubble ./megahit_out/intermediate_contigs/k79.bubble_seq.fa b' [sdbg_builder.cpp : 339] Host memory to be used: 486315100569' b' [sdbg_builder.cpp : 340] Number CPU threads: 28' b' [cx1.h : 450] Preparing data...' b' [cx1_seq2sdbg.cpp : 394] Number edges: 64551254' b' [cx1_seq2sdbg.cpp : 434] Bases to reserve: 11745266097, number contigs: 10792969, number multiplicity: 75344223' b' [cx1_seq2sdbg.cpp : 440] Before reading, sizeof seq_package: 3022660288, multiplicity vector: 75344223' b' [cx1_seq2sdbg.cpp : 529] After reading, sizeof seq_package: 5958976816, multiplicity vector: 75344223' b' [cx1.h : 457] Preparing data... Done. Time elapsed: 27.6147' b' [cx1.h : 464] Preparing partitions and initialing global data...' b' [cx1_seq2sdbg.cpp : 740] Memory for sequence: 6203257694' b' [cx1_seq2sdbg.cpp : 741] max # lv.1 items = 1299105064' b' [cx1.h : 480] Preparing partitions and initialing global data... Done. Time elapsed: 13.1347' b' [cx1.h : 486] Start main loop...' b' [cx1.h : 515] Lv1 scanning from bucket 0 to 605' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 11.1823' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 14.8677' b' [cx1.h : 515] Lv1 scanning from bucket 605 to 2820' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 14.0941' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 26.3099' b' [cx1.h : 515] Lv1 scanning from bucket 2820 to 6765' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 16.4419' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 27.4025' b' [cx1.h : 515] Lv1 scanning from bucket 6765 to 12600' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 17.9122' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 27.4767' b' [cx1.h : 515] Lv1 scanning from bucket 12600 to 20717' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 18.3540' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 26.2246' b' [cx1.h : 515] Lv1 scanning from bucket 20717 to 31770' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 19.0714' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 26.2632' b' [cx1.h : 515] Lv1 scanning from bucket 31770 to 47158' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 18.7728' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 27.7260' b' [cx1.h : 515] Lv1 scanning from bucket 47158 to 65536' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 15.1736' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 21.5334' b' [cx1.h : 607] Main loop done. Time elapsed: 328.8074' b' [cx1.h : 613] Postprocessing...' b' [cx1_seq2sdbg.cpp :1139] Number of $ A C G T A- C- G- T-:' b' [cx1_seq2sdbg.cpp :1142] 7622870 1367654129 887655791 887116272 1368550356 1249054 668354 664859 1250470' b' [cx1_seq2sdbg.cpp :1151] Total number of edges: 4522432155' b' [cx1_seq2sdbg.cpp :1152] Total number of ONEs: 4510976548' b' [cx1_seq2sdbg.cpp :1153] Total number of $v edges: 7622870' b' [cx1.h : 621] Postprocess done. Time elapsed: 0.0474' b' [utils.h : 126] Real: 369.6143\tuser: 6008.7834\tsys: 30.4256\tmaxrss: 8824560' --- [Thu Jan 3 03:38:15 2019] Assembling contigs from SdBG for k = 99 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core assemble -s ./megahit_out/tmp/k99/99 -o ./megahit_out/intermediate_contigs/k99 -t 28 --min_standalone 300.0 --prune_level 2 --merge_len 20 --merge_similar 0.95 --low_local_ratio 0.2 --min_depth 2 --bubble_level 2 --max_tip_len -1 --careful_bubble b' [assembler.cpp : 148] Loading succinct de Bruijn graph: ./megahit_out/tmp/k99/99 Done. Time elapsed: 775.795940' b' [assembler.cpp : 152] Number of Edges: 4522432155; K value: 99' b' [assembler.cpp : 162] Number of CPU threads: 28' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 2; Accumulated tips removed: 65602; time elapsed: 11.8340' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 4; Accumulated tips removed: 231715; time elapsed: 12.5604' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 8; Accumulated tips removed: 529107; time elapsed: 14.6683' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 16; Accumulated tips removed: 1052595; time elapsed: 19.1460' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 32; Accumulated tips removed: 1668796; time elapsed: 27.3809' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 64; Accumulated tips removed: 2572323; time elapsed: 42.6039' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 128; Accumulated tips removed: 4161472; time elapsed: 69.3538' b' [assembly_algorithms.cpp : 170] Removing tips with length less than 198; Accumulated tips removed: 5644890; time elapsed: 84.2026' b' [assembler.cpp : 179] Tips removal done! Time elapsed(sec): 281.914510' b' [assembler.cpp : 188] unitig graph size: 5104898, time for building: 265.168966' b' [assembler.cpp : 211] Number of bubbles removed: 478068, Time elapsed(sec): 31.621578' b' [assembler.cpp : 225] Number of complex bubbles removed: 114506, Time elapsed(sec): 11.230867' b' [assembler.cpp : 243] Number unitigs disconnected: 583235, time: 1.246592' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 196219, time: 1.433460' b' [assembler.cpp : 200] Tips removed: 8921, time: 1.608982' b' [assembler.cpp : 211] Number of bubbles removed: 27979, Time elapsed(sec): 2.270221' b' [assembler.cpp : 225] Number of complex bubbles removed: 28081, Time elapsed(sec): 3.367536' b' [assembler.cpp : 243] Number unitigs disconnected: 29112, time: 0.629575' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 4969, time: 0.580323' b' [assembler.cpp : 200] Tips removed: 14253, time: 1.515922' b' [assembler.cpp : 211] Number of bubbles removed: 1094, Time elapsed(sec): 0.560529' b' [assembler.cpp : 225] Number of complex bubbles removed: 1857, Time elapsed(sec): 1.226559' b' [assembler.cpp : 243] Number unitigs disconnected: 2184, time: 0.605866' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 798, time: 0.558601' b' [assembler.cpp : 200] Tips removed: 623, time: 1.436067' b' [assembler.cpp : 211] Number of bubbles removed: 28, Time elapsed(sec): 0.367320' b' [assembler.cpp : 225] Number of complex bubbles removed: 95, Time elapsed(sec): 1.070798' b' [assembler.cpp : 243] Number unitigs disconnected: 310, time: 0.595472' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 100, time: 0.550872' b' [assembler.cpp : 200] Tips removed: 62, time: 1.424716' b' [assembler.cpp : 211] Number of bubbles removed: 4, Time elapsed(sec): 0.361698' b' [assembler.cpp : 225] Number of complex bubbles removed: 7, Time elapsed(sec): 1.032235' b' [assembler.cpp : 243] Number unitigs disconnected: 46, time: 0.586484' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 23, time: 0.550829' b' [assembler.cpp : 132] Total length: 2272011740, N50: 1134, Mean: 736, number of contigs: 3083690' b' [assembler.cpp : 133] Maximum length: 893393' b' [assembler.cpp : 290] Time to output: 78.651084' b' [assembler.cpp : 317] Number of local low depth unitigs removed: 38852, complex bubbles removed: 3763, time: 40.043448' b' [assembler.cpp : 132] Total length: 764671888, N50: 1890, Mean: 673, number of contigs: 1135607' b' [assembler.cpp : 133] Maximum length: 893393' b' [utils.h : 126] Real: 1534.5610\tuser: 19551.5018\tsys: 55.7185\tmaxrss: 12674476' --- [Thu Jan 3 04:03:50 2019] Local assembling k = 99 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core local -c ./megahit_out/intermediate_contigs/k99.contigs.fa -l ./megahit_out/tmp/reads.lib -t 28 -o ./megahit_out/intermediate_contigs/k99.local.fa --kmax 119 b' [local_assembler.cpp : 89] Number of contigs: 2479907, Mapper size: 260318233' b' [read_lib_functions-inl.h : 256] Before reading, sizeof seq_package: 34014649232' b' [read_lib_functions-inl.h : 260] After reading, sizeof seq_package: 34014649232' b' [local_assembler.cpp : 330] Lib 0, insert size: 273.95 sd: 104.79' b' [local_assembler.cpp : 330] Lib 1, insert size: 279.19 sd: 109.47' b' [local_assembler.cpp : 330] Lib 2, insert size: 269.08 sd: 108.07' b' [local_assembler.cpp : 330] Lib 3, insert size: 279.18 sd: 110.34' b' [local_assembler.cpp : 330] Lib 4, insert size: 275.95 sd: 109.70' b' [local_assembler.cpp : 330] Lib 5, insert size: 269.69 sd: 107.07' b' [local_assembler.cpp : 462] Lib 0: total 163236894 reads, aligned 81028122, added 10037302 reads for local assembly' b' [local_assembler.cpp : 462] Lib 1: total 160288380 reads, aligned 78021992, added 10179687 reads for local assembly' b' [local_assembler.cpp : 462] Lib 2: total 167467174 reads, aligned 83236953, added 10128085 reads for local assembly' b' [local_assembler.cpp : 462] Lib 3: total 141110330 reads, aligned 73988040, added 8621705 reads for local assembly' b' [local_assembler.cpp : 462] Lib 4: total 143722488 reads, aligned 71647934, added 9595380 reads for local assembly' b' [local_assembler.cpp : 462] Lib 5: total 172333246 reads, aligned 86679801, added 10756432 reads for local assembly' b' [utils.h : 126] Real: 2191.9689\tuser: 58061.2496\tsys: 71.1012\tmaxrss: 44596048' --- [Thu Jan 3 04:40:22 2019] Extracting iterative edges from k = 99 to 119 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core iterate -c ./megahit_out/intermediate_contigs/k99.contigs.fa -b ./megahit_out/intermediate_contigs/k99.bubble_seq.fa -t 28 -k 99 -s 20 -o ./megahit_out/tmp/k119/119 -r ./megahit_out/tmp/reads.lib.bin -f binary b' [iterate_edges.cpp : 570] Number of crusial kmers: 1404993' b' [iterate_edges.cpp : 570] Number of crusial kmers: 3456390' b' [iterate_edges.cpp : 443] Processed: 67108864, aligned: 1464032. Iterative edges: 4342738' b' [iterate_edges.cpp : 443] Processed: 134217728, aligned: 2907657. Iterative edges: 6939113' b' [iterate_edges.cpp : 443] Processed: 201326592, aligned: 4444515. Iterative edges: 9325587' b' [iterate_edges.cpp : 443] Processed: 268435456, aligned: 5957546. Iterative edges: 11277930' b' [iterate_edges.cpp : 443] Processed: 335544320, aligned: 7361483. Iterative edges: 12788548' b' [iterate_edges.cpp : 443] Processed: 402653184, aligned: 8169285. Iterative edges: 13982168' b' [iterate_edges.cpp : 443] Processed: 469762048, aligned: 8951342. Iterative edges: 14860374' b' [iterate_edges.cpp : 443] Processed: 536870912, aligned: 10399121. Iterative edges: 15988933' b' [iterate_edges.cpp : 443] Processed: 603979776, aligned: 12092535. Iterative edges: 17005270' b' [iterate_edges.cpp : 443] Processed: 671088640, aligned: 13905161. Iterative edges: 18039652' b' [iterate_edges.cpp : 443] Processed: 738197504, aligned: 15713526. Iterative edges: 18966125' b' [iterate_edges.cpp : 443] Processed: 805306368, aligned: 17472643. Iterative edges: 19754391' b' [iterate_edges.cpp : 443] Processed: 872415232, aligned: 19136294. Iterative edges: 20444084' b' [iterate_edges.cpp : 443] Processed: 939524096, aligned: 20771845. Iterative edges: 21013434' b' [iterate_edges.cpp : 447] Total: 948158512, aligned: 21015042. Iterative edges: 21091746' b' [iterate_edges.cpp : 451] Writing iterative edges...' b' [utils.h : 126] Real: 398.2421\tuser: 9986.1930\tsys: 18.0246\tmaxrss: 1572132' --- [Thu Jan 3 04:47:00 2019] Building graph for k = 119 --- /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_sdbg_build seq2sdbg --host_mem 486315100569 --mem_flag 1 --gpu_mem 0 --output_prefix ./megahit_out/tmp/k119/119 --num_cpu_threads 28 -k 119 --kmer_from 99 --num_edge_files 1 --input_prefix ./megahit_out/tmp/k119/119 --addi_contig ./megahit_out/intermediate_contigs/k99.addi.fa --local_contig ./megahit_out/intermediate_contigs/k99.local.fa --contig ./megahit_out/intermediate_contigs/k99.contigs.fa --bubble ./megahit_out/intermediate_contigs/k99.bubble_seq.fa b' [sdbg_builder.cpp : 339] Host memory to be used: 486315100569' b' [sdbg_builder.cpp : 340] Number CPU threads: 28' b' [cx1.h : 450] Preparing data...' b' [cx1_seq2sdbg.cpp : 394] Number edges: 21091746' b' [cx1_seq2sdbg.cpp : 434] Bases to reserve: 6349246360, number contigs: 5724531, number multiplicity: 26816277' b' [cx1_seq2sdbg.cpp : 440] Before reading, sizeof seq_package: 1633107848, multiplicity vector: 26816277' b' [cx1_seq2sdbg.cpp : 529] After reading, sizeof seq_package: 3266215696, multiplicity vector: 53632554' b' [cx1.h : 457] Preparing data... Done. Time elapsed: 19.0110' b' [cx1.h : 464] Preparing partitions and initialing global data...' b' [cx1_seq2sdbg.cpp : 740] Memory for sequence: 3376714230' b' [cx1_seq2sdbg.cpp : 741] max # lv.1 items = 1040165435' b' [cx1.h : 480] Preparing partitions and initialing global data... Done. Time elapsed: 7.4350' b' [cx1.h : 486] Start main loop...' b' [cx1.h : 515] Lv1 scanning from bucket 0 to 443' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 5.8061' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 11.4274' b' [cx1.h : 515] Lv1 scanning from bucket 443 to 2382' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 7.7689' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 22.2476' b' [cx1.h : 515] Lv1 scanning from bucket 2382 to 6069' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 8.7054' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 24.3465' b' [cx1.h : 515] Lv1 scanning from bucket 6069 to 11655' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 9.1557' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 24.5996' b' [cx1.h : 515] Lv1 scanning from bucket 11655 to 19519' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 9.7634' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 23.4170' b' [cx1.h : 515] Lv1 scanning from bucket 19519 to 30297' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 10.1665' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 22.2914' b' [cx1.h : 515] Lv1 scanning from bucket 30297 to 45276' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 10.1862' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 23.3149' b' [cx1.h : 515] Lv1 scanning from bucket 45276 to 65536' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 8.6873' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 20.1678' b' [cx1.h : 607] Main loop done. Time elapsed: 242.0530' b' [cx1.h : 613] Postprocessing...' b' [cx1_seq2sdbg.cpp :1139] Number of $ A C G T A- C- G- T-:' b' [cx1_seq2sdbg.cpp :1142] 6030595 1250170586 807366271 806987900 1250893229 507954 378729 299662 549760' b' [cx1_seq2sdbg.cpp :1151] Total number of edges: 4123184686' b' [cx1_seq2sdbg.cpp :1152] Total number of ONEs: 4115417986' b' [cx1_seq2sdbg.cpp :1153] Total number of $v edges: 6030595' b' [cx1.h : 621] Postprocess done. Time elapsed: 0.0448' b' [utils.h : 126] Real: 268.5502\tuser: 5186.6744\tsys: 24.3981\tmaxrss: 6451452' --- [Thu Jan 3 04:51:29 2019] Assembling contigs from SdBG for k = 119 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core assemble -s ./megahit_out/tmp/k119/119 -o ./megahit_out/intermediate_contigs/k119 -t 28 --min_standalone 300.0 --prune_level 2 --merge_len 20 --merge_similar 0.95 --low_local_ratio 0.2 --min_depth 2 --bubble_level 2 --max_tip_len 182.0 --careful_bubble b' [assembler.cpp : 148] Loading succinct de Bruijn graph: ./megahit_out/tmp/k119/119 Done. Time elapsed: 681.955593' b' [assembler.cpp : 152] Number of Edges: 4123184686; K value: 119' b' [assembler.cpp : 162] Number of CPU threads: 28' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 2; Accumulated tips removed: 57282; time elapsed: 10.7322' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 4; Accumulated tips removed: 203632; time elapsed: 11.2281' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 8; Accumulated tips removed: 479956; time elapsed: 12.9223' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 16; Accumulated tips removed: 1039640; time elapsed: 16.2955' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 32; Accumulated tips removed: 1384786; time elapsed: 22.5822' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 64; Accumulated tips removed: 1834235; time elapsed: 34.6366' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 128; Accumulated tips removed: 2784163; time elapsed: 55.5684' b' [assembly_algorithms.cpp : 170] Removing tips with length less than 182; Accumulated tips removed: 3052225; time elapsed: 69.6736' b' [assembler.cpp : 179] Tips removal done! Time elapsed(sec): 233.788917' b' [assembler.cpp : 188] unitig graph size: 3254455, time for building: 253.908648' b' [assembler.cpp : 211] Number of bubbles removed: 178519, Time elapsed(sec): 7.883386' b' [assembler.cpp : 225] Number of complex bubbles removed: 38737, Time elapsed(sec): 4.186138' b' [assembler.cpp : 243] Number unitigs disconnected: 245597, time: 0.640519' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 71465, time: 0.667706' b' [assembler.cpp : 200] Tips removed: 3186, time: 1.167614' b' [assembler.cpp : 211] Number of bubbles removed: 6705, Time elapsed(sec): 1.214121' b' [assembler.cpp : 225] Number of complex bubbles removed: 10007, Time elapsed(sec): 2.183638' b' [assembler.cpp : 243] Number unitigs disconnected: 14576, time: 0.393839' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 1268, time: 0.343393' b' [assembler.cpp : 200] Tips removed: 4557, time: 1.131221' b' [assembler.cpp : 211] Number of bubbles removed: 403, Time elapsed(sec): 0.319053' b' [assembler.cpp : 225] Number of complex bubbles removed: 934, Time elapsed(sec): 0.973187' b' [assembler.cpp : 243] Number unitigs disconnected: 760, time: 0.378710' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 165, time: 0.337831' b' [assembler.cpp : 200] Tips removed: 158, time: 1.096040' b' [assembler.cpp : 211] Number of bubbles removed: 6, Time elapsed(sec): 0.266964' b' [assembler.cpp : 225] Number of complex bubbles removed: 31, Time elapsed(sec): 0.786789' b' [assembler.cpp : 243] Number unitigs disconnected: 80, time: 0.379913' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 15, time: 0.340309' b' [assembler.cpp : 200] Tips removed: 15, time: 1.103494' b' [assembler.cpp : 211] Number of bubbles removed: 0, Time elapsed(sec): 0.269086' b' [assembler.cpp : 225] Number of complex bubbles removed: 1, Time elapsed(sec): 0.802737' b' [assembler.cpp : 243] Number unitigs disconnected: 8, time: 0.379081' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 2, time: 0.339138' b' [assembler.cpp : 132] Total length: 2242011318, N50: 1318, Mean: 897, number of contigs: 2499251' b' [assembler.cpp : 133] Maximum length: 968571' b' [assembler.cpp : 290] Time to output: 77.271163' b' [assembler.cpp : 317] Number of local low depth unitigs removed: 10000, complex bubbles removed: 992, time: 20.207106' b' [assembler.cpp : 132] Total length: 544214638, N50: 3484, Mean: 1077, number of contigs: 505173' b' [assembler.cpp : 133] Maximum length: 968571' b' [utils.h : 126] Real: 1314.1376\tuser: 16155.7255\tsys: 44.8081\tmaxrss: 11632048' --- [Thu Jan 3 05:13:24 2019] Local assembling k = 119 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core local -c ./megahit_out/intermediate_contigs/k119.contigs.fa -l ./megahit_out/tmp/reads.lib -t 28 -o ./megahit_out/intermediate_contigs/k119.local.fa --kmax 141 b' [local_assembler.cpp : 89] Number of contigs: 2311758, Mapper size: 263504019' b' [read_lib_functions-inl.h : 256] Before reading, sizeof seq_package: 34014649232' b' [read_lib_functions-inl.h : 260] After reading, sizeof seq_package: 34014649232' b' [local_assembler.cpp : 330] Lib 0, insert size: 275.09 sd: 105.07' b' [local_assembler.cpp : 330] Lib 1, insert size: 280.58 sd: 110.30' b' [local_assembler.cpp : 330] Lib 2, insert size: 269.78 sd: 108.59' b' [local_assembler.cpp : 330] Lib 3, insert size: 280.33 sd: 111.05' b' [local_assembler.cpp : 330] Lib 4, insert size: 277.98 sd: 110.64' b' [local_assembler.cpp : 330] Lib 5, insert size: 271.08 sd: 107.70' b' [local_assembler.cpp : 462] Lib 0: total 163236894 reads, aligned 78039381, added 6961374 reads for local assembly' b' [local_assembler.cpp : 462] Lib 1: total 160288380 reads, aligned 75047701, added 6998339 reads for local assembly' b' [local_assembler.cpp : 462] Lib 2: total 167467174 reads, aligned 81530333, added 7919179 reads for local assembly' b' [local_assembler.cpp : 462] Lib 3: total 141110330 reads, aligned 71040851, added 5598766 reads for local assembly' b' [local_assembler.cpp : 462] Lib 4: total 143722488 reads, aligned 68149243, added 6128218 reads for local assembly' b' [local_assembler.cpp : 462] Lib 5: total 172333246 reads, aligned 82779917, added 7014324 reads for local assembly' b' [utils.h : 126] Real: 1793.1308\tuser: 47093.8832\tsys: 64.8944\tmaxrss: 44348536' --- [Thu Jan 3 05:43:17 2019] Extracting iterative edges from k = 119 to 141 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core iterate -c ./megahit_out/intermediate_contigs/k119.contigs.fa -b ./megahit_out/intermediate_contigs/k119.bubble_seq.fa -t 28 -k 119 -s 22 -o ./megahit_out/tmp/k141/141 -r ./megahit_out/tmp/reads.lib.bin -f binary b' [iterate_edges.cpp : 570] Number of crusial kmers: 336419' b' [iterate_edges.cpp : 570] Number of crusial kmers: 1111423' b' [iterate_edges.cpp : 443] Processed: 67108864, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 134217728, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 201326592, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 268435456, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 335544320, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 402653184, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 469762048, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 536870912, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 603979776, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 671088640, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 738197504, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 805306368, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 872415232, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 443] Processed: 939524096, aligned: 0. Iterative edges: 0' b' [iterate_edges.cpp : 447] Total: 948158512, aligned: 0. Iterative edges: 0' b' [utils.h : 126] Real: 96.2699\tuser: 126.0897\tsys: 16.2811\tmaxrss: 496068' --- [Thu Jan 3 05:44:54 2019] Building graph for k = 141 --- /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_sdbg_build seq2sdbg --host_mem 486315100569 --mem_flag 1 --gpu_mem 0 --output_prefix ./megahit_out/tmp/k141/141 --num_cpu_threads 28 -k 141 --kmer_from 119 --num_edge_files 1 --addi_contig ./megahit_out/intermediate_contigs/k119.addi.fa --local_contig ./megahit_out/intermediate_contigs/k119.local.fa --contig ./megahit_out/intermediate_contigs/k119.contigs.fa --bubble ./megahit_out/intermediate_contigs/k119.bubble_seq.fa b' [sdbg_builder.cpp : 339] Host memory to be used: 486315100569' b' [sdbg_builder.cpp : 340] Number CPU threads: 28' b' [cx1.h : 450] Preparing data...' b' [cx1_seq2sdbg.cpp : 434] Bases to reserve: 2370526493, number contigs: 3101048, number multiplicity: 3101048' b' [cx1_seq2sdbg.cpp : 440] Before reading, sizeof seq_package: 617440016, multiplicity vector: 3101048' b' [cx1_seq2sdbg.cpp : 529] After reading, sizeof seq_package: 1234880032, multiplicity vector: 6202096' b' [cx1.h : 457] Preparing data... Done. Time elapsed: 13.1567' b' [cx1.h : 464] Preparing partitions and initialing global data...' b' [cx1_seq2sdbg.cpp : 740] Memory for sequence: 1271620260' b' [cx1_seq2sdbg.cpp : 741] max # lv.1 items = 775031968' b' [cx1.h : 480] Preparing partitions and initialing global data... Done. Time elapsed: 1.9056' b' [cx1.h : 486] Start main loop...' b' [cx1.h : 515] Lv1 scanning from bucket 0 to 359' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 1.3102' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 7.6236' b' [cx1.h : 515] Lv1 scanning from bucket 359 to 2131' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 1.6338' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 16.4432' b' [cx1.h : 515] Lv1 scanning from bucket 2131 to 5673' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 1.8371' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 18.0074' b' [cx1.h : 515] Lv1 scanning from bucket 5673 to 11146' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 2.0167' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 18.0374' b' [cx1.h : 515] Lv1 scanning from bucket 11146 to 18911' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 2.1800' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 17.9835' b' [cx1.h : 515] Lv1 scanning from bucket 18911 to 29572' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 2.4230' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 19.1604' b' [cx1.h : 515] Lv1 scanning from bucket 29572 to 44288' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 2.3523' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 19.6390' b' [cx1.h : 515] Lv1 scanning from bucket 44288 to 65536' b' [cx1.h : 528] Lv1 scanning done. Large diff: 0. Time elapsed: 2.0893' b' [cx1.h : 594] Lv1 fetching & sorting done. Time elapsed: 18.2538' b' [cx1.h : 607] Main loop done. Time elapsed: 150.9918' b' [cx1.h : 613] Postprocessing...' b' [cx1_seq2sdbg.cpp :1139] Number of $ A C G T A- C- G- T-:' b' [cx1_seq2sdbg.cpp :1142] 4722855 1199001006 773869813 773526760 1199632465 246372 131826 131290 245302' b' [cx1_seq2sdbg.cpp :1151] Total number of edges: 3951507689' b' [cx1_seq2sdbg.cpp :1152] Total number of ONEs: 3946030044' b' [cx1_seq2sdbg.cpp :1153] Total number of $v edges: 4722855' b' [cx1.h : 621] Postprocess done. Time elapsed: 0.0390' b' [utils.h : 126] Real: 166.1002\tuser: 4057.3495\tsys: 18.7900\tmaxrss: 4394552' --- [Thu Jan 3 05:47:40 2019] Assembling contigs from SdBG for k = 141 --- cmd: /gscratch/srlab/programs/megahit_v1.1.4_LINUX_CPUONLY_x86_64-bin/megahit_asm_core assemble -s ./megahit_out/tmp/k141/141 -o ./megahit_out/intermediate_contigs/k141 -t 28 --min_standalone 300.0 --prune_level 2 --merge_len 20 --merge_similar 0.95 --low_local_ratio 0.2 --min_depth 2 --bubble_level 2 --max_tip_len 160.0 --is_final_round b' [assembler.cpp : 148] Loading succinct de Bruijn graph: ./megahit_out/tmp/k141/141 Done. Time elapsed: 650.534853' b' [assembler.cpp : 152] Number of Edges: 3951507689; K value: 141' b' [assembler.cpp : 162] Number of CPU threads: 28' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 2; Accumulated tips removed: 10484; time elapsed: 10.2255' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 4; Accumulated tips removed: 34278; time elapsed: 10.7519' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 8; Accumulated tips removed: 79516; time elapsed: 12.1132' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 16; Accumulated tips removed: 164872; time elapsed: 15.0468' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 32; Accumulated tips removed: 312018; time elapsed: 20.6097' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 64; Accumulated tips removed: 546608; time elapsed: 31.7843' b' [assembly_algorithms.cpp : 162] Removing tips with length less than 128; Accumulated tips removed: 896004; time elapsed: 53.2063' b' [assembly_algorithms.cpp : 170] Removing tips with length less than 160; Accumulated tips removed: 979302; time elapsed: 62.7244' b' [assembler.cpp : 179] Tips removal done! Time elapsed(sec): 216.605876' b' [assembler.cpp : 188] unitig graph size: 2748807, time for building: 248.527225' b' [assembler.cpp : 211] Number of bubbles removed: 102873, Time elapsed(sec): 0.698809' b' [assembler.cpp : 225] Number of complex bubbles removed: 15774, Time elapsed(sec): 1.083130' b' [assembler.cpp : 243] Number unitigs disconnected: 132260, time: 0.551315' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 32154, time: 0.536288' b' [assembler.cpp : 200] Tips removed: 1593, time: 0.999743' b' [assembler.cpp : 211] Number of bubbles removed: 867, Time elapsed(sec): 0.245191' b' [assembler.cpp : 225] Number of complex bubbles removed: 3299, Time elapsed(sec): 0.771356' b' [assembler.cpp : 243] Number unitigs disconnected: 8786, time: 0.367845' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 631, time: 0.387801' b' [assembler.cpp : 200] Tips removed: 1568, time: 0.983982' b' [assembler.cpp : 211] Number of bubbles removed: 82, Time elapsed(sec): 0.237150' b' [assembler.cpp : 225] Number of complex bubbles removed: 378, Time elapsed(sec): 0.676794' b' [assembler.cpp : 243] Number unitigs disconnected: 313, time: 0.351013' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 69, time: 0.383936' b' [assembler.cpp : 200] Tips removed: 39, time: 0.983018' b' [assembler.cpp : 211] Number of bubbles removed: 0, Time elapsed(sec): 0.236803' b' [assembler.cpp : 225] Number of complex bubbles removed: 6, Time elapsed(sec): 0.585978' b' [assembler.cpp : 243] Number unitigs disconnected: 33, time: 0.352410' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 3, time: 0.384034' b' [assembler.cpp : 200] Tips removed: 2, time: 0.972645' b' [assembler.cpp : 211] Number of bubbles removed: 0, Time elapsed(sec): 0.235904' b' [assembler.cpp : 225] Number of complex bubbles removed: 0, Time elapsed(sec): 0.619824' b' [assembler.cpp : 243] Number unitigs disconnected: 3, time: 0.349577' b' [assembler.cpp : 265] Unitigs removed in excessive pruning: 0, time: 0.247406' b' [assembler.cpp : 317] Number of local low depth unitigs removed: 2632, complex bubbles removed: 105, time: 23.869613' b' [assembler.cpp : 132] Total length: 2257716734, N50: 1396, Mean: 957, number of contigs: 2358869' b' [assembler.cpp : 133] Maximum length: 2035937' b' [utils.h : 126] Real: 1228.2709\tuser: 14739.6696\tsys: 45.1473\tmaxrss: 11253400' --- [Thu Jan 3 06:08:08 2019] Merging to output final contigs --- --- [STAT] 2276153 contigs, total 2243059881 bp, min 200 bp, max 2035937 bp, avg 985 bp, N50 1413 bp --- [Thu Jan 3 06:08:19 2019] ALL DONE. Time elapsed: 56535.269958 seconds ---