Performing re-exec to adjust stack size.
Sun May 03 14:50:05 2015 run on campusrocks2-0-1, pid=55420 [Apr 4 2015 20:59:59 R52415 ]
DiscovarDeNovo \
READS="frac:0.5,sample:19::/campusdata/BME235/Spring2015Data/ba \
ms/SW019_adapterTrimmed_dupRemoved_150424.bam+frac:0.5,sample:1 \
8::/campusdata/BME235/Spring2015Data/bams/SW018_adapterTrimmed_ \
dupRemoved_150424.bam" \
OUT_DIR=/campusdata/rcalef/0.5_dataDDN_150501 NUM_THREADS=50 \
MAX_MEM_GB=220 MEMORY_CHECK=True
SYSTEM INFO
- OS: Linux :: 2.6.32-220.13.1.el6.x86_64 :: #1 SMP Tue Apr 17 23:56:34 BST 2012
- node name: campusrocks2-0-1.local
- hardware type: x86_64
- cache size: 2048 KB
- cpu MHz: 2400.084
- cpu model name: AMD Opteron™ Processor 6278
- physical memory: 252.39 GB
MEMORY CHECK (typically takes several minutes; could cause
machine to become sluggish or result in this job being killed)
- Apparently able to allocate 100% of nominally available memory.
- Can access at least 220 GB.
Sun May 03 14:50:28 2015: finding input files
Sun May 03 14:50:28 2015: reading 2 files (which may take a while)
Sun May 03 14:50:28 2015: processing /campusdata/BME235/Spring2015Data/bams/SW019_adapterTrimmed_dupRemoved_150424.bam.
Sun May 03 14:50:28 2015: memory in use = 0.01 GB, peak = 0.01 GB
Sun May 03 15:13:28 2015: there are 124,970,747 reads of mean length 143
Sun May 03 15:13:28 2015: memory in use = 72.40 GB, peak = 74.21 GB
Sun May 03 15:13:49 2015: reads sorted
Sun May 03 15:13:49 2015: memory in use = 73.34 GB, peak = 74.21 GB
Sun May 03 15:16:17 2015: data stashed in output structures
Sun May 03 15:16:17 2015: memory in use = 12.77 GB, peak = 74.21 GB
Sun May 03 15:16:17 2015: processing /campusdata/BME235/Spring2015Data/bams/SW018_adapterTrimmed_dupRemoved_150424.bam.
Sun May 03 15:16:17 2015: memory in use = 12.77 GB, peak = 74.21 GB
Sun May 03 15:33:24 2015: there are 138,864,653 reads of mean length 99
Sun May 03 15:33:24 2015: memory in use = 62.37 GB, peak = 74.21 GB
Sun May 03 15:33:46 2015: reads sorted
Sun May 03 15:33:46 2015: memory in use = 63.42 GB, peak = 74.21 GB
Sun May 03 15:36:32 2015: data stashed in output structures
Sun May 03 15:36:32 2015: memory in use = 26.67 GB, peak = 74.21 GB
INPUT FILES:
[1,type=frag,sample=19,lib=1,frac=0.5] /campusdata/BME235/Spring2015Data/bams/SW019_adapterTrimmed_dupRemoved_150424.bam
[2,type=frag,sample=18,lib=1,frac=0.5] /campusdata/BME235/Spring2015Data/bams/SW018_adapterTrimmed_dupRemoved_150424.bam
Sun May 03 15:36:32 2015: found 2 samples
Sun May 03 15:36:32 2015: starts = 0,124970748
Sun May 03 15:45:17 2015: using 263,835,402 reads
Sun May 03 15:45:17 2015: data extraction complete
54.8 minutes used extracting reads
Sun May 03 15:52:15 2015: see total physical memory of 270,999,298,048 bytes
Sun May 03 15:52:15 2015: see user-imposed limit on memory of 236,223,201,280 bytes
Sun May 03 15:52:15 2015: 8.96 bytes per read base, assuming max memory available
We need 4 passes.
Expect 2578518 keys per batch.
Provide 3243760 keys per batch.
We need 5 passes.
Expect 2062814 keys per batch.
Provide 2387988 keys per batch.
Sun May 03 16:51:17 2015: back from buildReadQGraph
memory in use = 57,726,877,696
checksum_60 = 1091042242363737796
Sun May 03 16:53:37 2015: constructing places
Sun May 03 16:54:01 2015: sorting places
Sun May 03 17:00:11 2015: building all
Sun May 03 17:06:25 2015: calling LongReadsToPaths
Warning: HashSet initial size too small.
Sun May 03 17:09:17 2015: writing
Sun May 03 17:10:16 2015: translating paths
Sun May 03 17:10:28 2015: final stage of path translation
Sun May 03 17:29:36 2015: writing paths
56.1 seconds used reloading assembly
Sun May 03 17:43:11 2015: start walking
memory in use = 36,509,278,208
Sun May 03 17:44:23 2015: start walking
memory in use = 36,612,886,528
2.86 minutes used cleaning 200-mer graph
1.97 hours used in ReadQGrapher
6.2e-06 seconds used reloading reads
checksum_200 = 529403660712473571
1 peak mem usage = 140.27 GB
3.66 minutes used loading stuff
2 peak mem usage = 140.27 GB
launching gap assemblies, mem usage = 35,265,961,984
Sun May 03 17:54:38 2015: finding unsatisfieds
Sun May 03 17:54:45 2015: creating multiplicity map
Sun May 03 17:54:47 2015: economizing links
Sun May 03 17:54:47 2015: forming neighborhoods
Sun May 03 17:54:47 2015: forming initial clusters
Sun May 03 17:55:04 2015: start sort
0.688 seconds used sorting
Sun May 03 17:55:05 2015: merging clusters
xs.size( ) = 2689147
2.77 minutes used merging
xs.size( ) = 2235783
Sun May 03 17:58:12 2015: start overlap-based merging
Sun May 03 17:59:11 2015: start overlap-based merging
LR.size( ) = 2219803
LR.size( ) = 1109905
Sun May 03 18:03:02 2015: now processing 1109905 blobs
Sun May 03 18:03:03 2015: memory in use = 36.20 GB, peak = 140.27 GB
………. ………. ………. ………. ……….
………. ………. ………. ………. ……….
4.17 hours spent in local assemblies, memory in use = 39.19 GB, peak = 140.27 GB
Sun May 03 22:13:21 2015: patch reserving space
Sun May 03 22:13:21 2015: memory in use = 39.20 GB
25.2 seconds used patching, peak mem usage = 140.27 GB
new_stuff.size( ) = 2202589
Sun May 03 22:14:27 2015: building hb2
22.1 seconds used in new stuff 1 test
memory in use now = 37,405,421,568
Warning: HashSet initial size too small.
Warning: HashSet initial size too small.
Sun May 03 22:19:13 2015: back from buildBigKHBVFromReads
4.77 minutes used in new stuff 2 test
peak mem usage = 140.27 GB
26.2 seconds used in new stuff 5
Sun May 03 22:20:12 2015: finding interesting reads
Sun May 03 22:20:12 2015: memory in use = 32.74 GB, peak = 140.27 GB
Sun May 03 22:20:24 2015: building dictionary
Sun May 03 22:20:24 2015: memory in use = 33.02 GB, peak = 140.27 GB
Sun May 03 22:27:44 2015: reducing
Sun May 03 22:27:44 2015: memory in use = 140.68 GB, peak = 140.27 GB
We need 1 passes.
Expect 1066152 keys per batch.
Provide 1508583 keys per batch.
Sun May 03 22:37:47 2015: kmerizing
Sun May 03 22:37:47 2015: memory in use = 171.42 GB, peak = 220.11 GB
We need 1 passes.
Expect 695406 keys per batch.
Provide 1398894 keys per batch.
There were 209 buffer overflows.
Sun May 03 22:39:04 2015: cleaning
Sun May 03 22:39:04 2015: memory in use = 171.43 GB, peak = 220.11 GB
Sun May 03 22:40:21 2015: finding uniquely aligning edges
Sun May 03 22:40:21 2015: memory in use = 171.44 GB, peak = 220.11 GB
28.2 minutes used in new phase
hb.N( ) = 8067352, hb.EdgeObjectCount( ) = 4464217
26257 paths improved by rerouting
Sum(invalid) = 19494, npids = 131917701
691 edges tamped down
Sun May 03 22:47:10 2015: checking involution
Sun May 03 22:47:10 2015: done
WARNING: 684 suspicious read-paths.
Sum(invalid) = 2233, npids = 131917701
677 edges tamped down
Sun May 03 22:51:06 2015: making paths index for pull apart
Sun May 03 22:51:14 2015: pulling apart repeats
0.0117 seconds used separating paths 1
5.41 seconds used in fixing mToLeft, mToRight, and mEdgeToPathIds
Sun May 03 22:51:28 2015: there were 95 repeats pulled apart.
Sun May 03 22:51:28 2015: there were 8121 read paths removed during separation.
Sun May 03 22:51:33 2015: improving paths
Sun May 03 22:59:39 2015: done
5684 paths extended
Sun May 03 23:02:40 2015: start degloop
Sun May 03 23:02:40 2015: creating path index
Sun May 03 23:02:51 2015: starting loop
Sun May 03 23:02:51 2015: degloop complete
Sun May 03 23:03:22 2015: unwinding three-edge plasmids
Sun May 03 23:03:24 2015: removing small components
Sun May 03 23:04:54 2015: writing a.fin files
Sun May 03 23:13:51 2015: determining candidates
Sun May 03 23:13:52 2015: determining candidates
CN fraction good = 0.33
Sun May 03 23:14:34 2015: deleting 0 gaps and adding 0 gaps to force symmetry
Sun May 03 23:14:54 2015: done making gaps, time used = 30.6 seconds
Sun May 03 23:16:10 2015: determining candidates
Sun May 03 23:16:11 2015: determining candidates
1.7 seconds using setting up final fasta
12.5 seconds using printing final fasta
assembly has 273653 edges of mean length 807.36
contig line N50: 1,489
scaffold line N50: 1,489
total bases in 1 kb+ scaffolds: 101,397,871
total bases in 10 kb+ scaffolds: 151,417
There are 263,835,402 reads of mean length 100.0 and mean base quality 36.7.
MPL1 = mean length of first read in pair up to first error = 7
(normal range is 175-225 for 250 base reads)
Estimated chimera rate in read pairs (including mismapping) = 21.08%.
WARNING: based on the observed chimera rate, it looks like something may be
very wrong with your input data.
genomic read coverage, using 1 kb+ scaffolds for genome size estimate: 260.2
run started Sun May 03 14:50:05 2015, completed Sun May 03 23:21:47 2015
peak mem usage = 220.11 GB, total time = 8.53 hours
final checksum = 63450798033837183
DiscovarDeNovo READS=“frac:0.5,sample:19::/campusdata/BME235/Spring2015Data/bams/SW019_adapterTrimmed_dupRemoved_150424.bam+frac:0.5,sample:18::/campusdata/BME235/Spring2015Data/bams/SW018_adapterTrimmed_dupRemoved_150424.bam” OUT_DIR=/campusdata/rcalef/0.5_dataDDN_150501 NUM_THREADS=50 MAX_MEM_GB=220 MEMORY_CHECK=True
Sun May 03 23:21:48 2015: done
