This is an old revision of the document!
edser2:~> ./DiscovarDeNovo READS="frac:0.1,sample:18H::/scratch/bananaSlugBAMS/SW018_S1_L007_001.bam+frac:0.1,sample:19M::/scratch/bananaSlugBAMS/SW019 _S1_L001_001.bam+frac:0.1,sample:19H::/scratch/bananaSlugBAMS/SW019_S2_L008_001.bam" OUT_DIR=/scratch/bananaSlugAssemblies/10run/ Performing re-exec to adjust stack size. -------------------------------------------------------------------------------- Mon Apr 27 22:01:35 2015 run on edser2, pid=31590 [Apr 10 2015 12:24:57 R52415 ] DiscovarDeNovo \ READS="frac:0.1,sample:18H::/scratch/bananaSlugBAMS/SW018_S1_L0 \ 07_001.bam+frac:0.1,sample:19M::/scratch/bananaSlugBAMS/SW019_S \ 1_L001_001.bam+frac:0.1,sample:19H::/scratch/bananaSlugBAMS/SW0 \ 19_S2_L008_001.bam" \ OUT_DIR=/scratch/bananaSlugAssemblies/10run/ -------------------------------------------------------------------------------- SYSTEM INFO - OS: Linux :: 3.5.0-54-generic :: #81~precise1-Ubuntu SMP Tue Jul 15 04:02:22 UTC 2014 - node name: edser2 - hardware type: x86_64 - cache size: 30720 KB - cpu MHz: 2699.924 - cpu model name: Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz - physical memory: 314.88 GB Omitting memory check. If you run into problems with memory, you might try rerunning with MEMORY_CHECK=True. Mon Apr 27 22:01:36 2015: finding input files Mon Apr 27 22:01:36 2015: reading 3 files (which may take a while) Mon Apr 27 22:01:36 2015: processing /scratch/bananaSlugBAMS/SW018_S1_L007_001.bam. Mon Apr 27 22:01:36 2015: memory in use = 0.01 GB, peak = 0.01 GB Mon Apr 27 22:24:52 2015: there are 28,603,868 reads of mean length 100 Mon Apr 27 22:24:52 2015: memory in use = 42.71 GB, peak = 43.92 GB Mon Apr 27 22:25:28 2015: reads sorted Mon Apr 27 22:25:28 2015: memory in use = 43.78 GB, peak = 43.92 GB Mon Apr 27 22:27:33 2015: data stashed in output structures Mon Apr 27 22:27:33 2015: memory in use = 2.64 GB, peak = 43.92 GB Mon Apr 27 22:27:33 2015: processing /scratch/bananaSlugBAMS/SW019_S1_L001_001.bam. Mon Apr 27 22:27:33 2015: memory in use = 2.64 GB, peak = 43.92 GB Mon Apr 27 22:46:24 2015: there are 5,453,744 reads of mean length 301 Mon Apr 27 22:46:24 2015: memory in use = 18.61 GB, peak = 43.92 GB Mon Apr 27 22:46:32 2015: reads sorted Mon Apr 27 22:46:32 2015: memory in use = 18.81 GB, peak = 43.92 GB Mon Apr 27 22:47:17 2015: data stashed in output structures Mon Apr 27 22:47:17 2015: memory in use = 3.86 GB, peak = 43.92 GB Mon Apr 27 22:47:17 2015: processing /scratch/bananaSlugBAMS/SW019_S2_L008_001.bam. Mon Apr 27 22:47:17 2015: memory in use = 3.86 GB, peak = 43.92 GB Mon Apr 27 23:02:58 2015: there are 19,727,231 reads of mean length 100 Mon Apr 27 23:02:58 2015: memory in use = 33.52 GB, peak = 43.92 GB Mon Apr 27 23:03:22 2015: reads sorted Mon Apr 27 23:03:22 2015: memory in use = 34.25 GB, peak = 43.92 GB Mon Apr 27 23:04:52 2015: data stashed in output structures Mon Apr 27 23:04:52 2015: memory in use = 5.65 GB, peak = 43.92 GB INPUT FILES: [1,type=frag,sample=18H,lib=1,frac=0.1] /scratch/bananaSlugBAMS/SW018_S1_L007_001.bam [2,type=frag,sample=19M,lib=1,frac=0.1] /scratch/bananaSlugBAMS/SW019_S1_L001_001.bam [3,type=frag,sample=19H,lib=1,frac=0.1] /scratch/bananaSlugBAMS/SW019_S2_L008_001.bam Mon Apr 27 23:04:52 2015: found 3 samples Mon Apr 27 23:04:52 2015: starts = 0,28603868,34057612 Mon Apr 27 23:06:29 2015: using 53,784,844 reads Mon Apr 27 23:06:29 2015: data extraction complete 1.08 hours used extracting reads Mon Apr 27 23:06:31 2015: see total physical memory of 338,094,714,880 bytes Mon Apr 27 23:06:31 2015: 52.59 bytes per read base, assuming max memory available We need 1 passes. Expect 2750583 keys per batch. Provide 5342050 keys per batch. There were 192 buffer overflows. We need 1 passes. Expect 2750583 keys per batch. Provide 5103062 keys per batch. There were 192 buffer overflows. Warning: HashSet initial size too small. Mon Apr 27 23:36:51 2015: back from buildReadQGraph memory in use = 13,995,552,768 checksum_60 = 759152776190314974 Mon Apr 27 23:37:57 2015: constructing places Mon Apr 27 23:38:15 2015: sorting places Mon Apr 27 23:41:26 2015: building all Mon Apr 27 23:43:07 2015: calling LongReadsToPaths Mon Apr 27 23:44:14 2015: writing Mon Apr 27 23:44:21 2015: translating paths Mon Apr 27 23:44:34 2015: final stage of path translation Mon Apr 27 23:58:35 2015: writing paths 9.52 seconds used reloading assembly Tue Apr 28 00:00:26 2015: start walking memory in use = 7,987,261,440 Tue Apr 28 00:01:04 2015: start walking memory in use = 7,825,833,984 1.21 minutes used cleaning 200-mer graph 55.5 minutes used in ReadQGrapher 5.96e-06 seconds used reloading reads checksum_200 = 23687290918476347 1 peak mem usage = 151.05 GB 10.5 seconds used loading stuff 2 peak mem usage = 151.05 GB launching gap assemblies, mem usage = 7,615,533,056 Tue Apr 28 00:02:26 2015: finding unsatisfieds Tue Apr 28 00:02:27 2015: creating multiplicity map Tue Apr 28 00:02:27 2015: economizing links Tue Apr 28 00:02:27 2015: forming neighborhoods Tue Apr 28 00:02:28 2015: forming initial clusters Tue Apr 28 00:02:28 2015: start sort 0.086 seconds used sorting Tue Apr 28 00:02:28 2015: merging clusters xs.size( ) = 98897 7.32 seconds used merging xs.size( ) = 16644 Tue Apr 28 00:02:36 2015: start overlap-based merging Tue Apr 28 00:02:37 2015: start overlap-based merging LR.size( ) = 15709 LR.size( ) = 7868 Tue Apr 28 00:03:02 2015: now processing 7868 blobs Tue Apr 28 00:03:02 2015: memory in use = 7.22 GB, peak = 151.05 GB .......... .......... .......... .......... .......... .......... .......... .......... .......... .......... 10 minutes spent in local assemblies, memory in use = 7.40 GB, peak = 151.05 GB Tue Apr 28 00:13:03 2015: patch reserving space Tue Apr 28 00:13:03 2015: memory in use = 7.40 GB 0.39 seconds used patching, peak mem usage = 151.05 GB new_stuff.size( ) = 51903 Tue Apr 28 00:13:13 2015: building hb2 5.14 seconds used in new stuff 1 test memory in use now = 7,876,759,552 Warning: HashSet initial size too small. Tue Apr 28 00:14:26 2015: back from buildBigKHBVFromReads 1.22 minutes used in new stuff 2 test peak mem usage = 151.05 GB 11.8 seconds used in new stuff 5 Tue Apr 28 00:14:45 2015: finding interesting reads Tue Apr 28 00:14:45 2015: memory in use = 7.29 GB, peak = 151.05 GB Tue Apr 28 00:15:41 2015: building dictionary Tue Apr 28 00:15:41 2015: memory in use = 7.30 GB, peak = 151.05 GB Tue Apr 28 00:15:54 2015: reducing Tue Apr 28 00:15:54 2015: memory in use = 13.40 GB, peak = 151.05 GB We need 1 passes. Expect 57267 keys per batch. Provide 143166 keys per batch. Tue Apr 28 00:16:18 2015: kmerizing Tue Apr 28 00:16:18 2015: memory in use = 13.88 GB, peak = 151.05 GB We need 1 passes. Expect 77877 keys per batch. Provide 194692 keys per batch. Tue Apr 28 00:16:29 2015: cleaning Tue Apr 28 00:16:29 2015: memory in use = 13.88 GB, peak = 151.05 GB Tue Apr 28 00:16:42 2015: finding uniquely aligning edges Tue Apr 28 00:16:42 2015: memory in use = 13.88 GB, peak = 151.05 GB 3.89 minutes used in new phase hb.N( ) = 1180208, hb.EdgeObjectCount( ) = 741914 1867 paths improved by rerouting Sum(invalid) = 898, npids = 26892422 271 edges tamped down Tue Apr 28 00:18:34 2015: checking involution Tue Apr 28 00:18:34 2015: done WARNING: 235 suspicious read-paths. Sum(invalid) = 564, npids = 26892422 216 edges tamped down Tue Apr 28 00:19:44 2015: making paths index for pull apart Tue Apr 28 00:19:48 2015: pulling apart repeats 0.0336 seconds used separating paths 1 2.87 seconds used in fixing mToLeft, mToRight, and mEdgeToPathIds Tue Apr 28 00:19:54 2015: there were 234 repeats pulled apart. Tue Apr 28 00:19:54 2015: there were 2055 read paths removed during separation. Tue Apr 28 00:19:56 2015: improving paths Tue Apr 28 00:21:03 2015: done 44210 paths extended Tue Apr 28 00:21:50 2015: start degloop Tue Apr 28 00:21:50 2015: creating path index Tue Apr 28 00:21:59 2015: starting loop Tue Apr 28 00:22:02 2015: degloop complete Tue Apr 28 00:22:19 2015: unwinding three-edge plasmids Tue Apr 28 00:22:20 2015: removing small components Tue Apr 28 00:22:54 2015: writing a.fin files Tue Apr 28 00:23:48 2015: determining candidates Tue Apr 28 00:23:48 2015: determining candidates Tue Apr 28 00:23:48 2015: determining candidates CN fraction good = 0.18 /bin/sh: 1: set: Illegal option -o pipefail Tue Apr 28 00:24:04 2015: deleting 0 gaps and adding 0 gaps to force symmetry Tue Apr 28 00:24:11 2015: done making gaps, time used = 9.07 seconds Tue Apr 28 00:24:35 2015: determining candidates Tue Apr 28 00:24:35 2015: determining candidates Tue Apr 28 00:24:35 2015: determining candidates 0.204 seconds using setting up final fasta 0.833 seconds using printing final fasta ==================================================================================== assembly has 38195 edges of mean length 187.611 contig line N50: 2,563 scaffold line N50: 2,563 total bases in 1 kb+ scaffolds: 1,476,875 total bases in 10 kb+ scaffolds: 168,543 There are 53,784,844 reads of mean length 119.5 and mean base quality 35.4. MPL1 = mean length of first read in pair up to first error = 3 (normal range is 175-225 for 250 base reads) Estimated chimera rate in read pairs (including mismapping) = 5.92%. WARNING: based on the observed chimera rate, it looks like something may be very wrong with your input data. genomic read coverage, using 1 kb+ scaffolds for genome size estimate: 4353.3 run started Mon Apr 27 22:01:35 2015, completed Tue Apr 28 00:25:19 2015 peak mem usage = 151.05 GB, total time = 2.4 hours final checksum = 376448268699082 DiscovarDeNovo READS="frac:0.1,sample:18H::/scratch/bananaSlugBAMS/SW018_S1_L007_001.bam+frac:0.1,sample:19M::/scratch/bananaSlugBAMS/SW019_S1_L001_001.bam+frac:0.1,sample:19H::/scratch/bananaSlugBAMS/SW019_S2_L008_001.bam" OUT_DIR=/scratch/bananaSlugAssemblies/10run/ Tue Apr 28 00:25:20 2015: done
