Banana Slug Genomics

edser2:~> ./DiscovarDeNovo READS=“frac:0.1,sample:18H::/scratch/bananaSlugBAMS/SW018_S1_L007_001.bam+frac:0.1,sample:19M::/scratch/bananaSlugBAMS/SW019

_S1_L001_001.bam+frac:0.1,sample:19H::/scratch/bananaSlugBAMS/SW019_S2_L008_001.bam” OUT_DIR=/scratch/bananaSlugAssemblies/10run/

Performing re-exec to adjust stack size.

---

Mon Apr 27 22:01:35 2015 run on edser2, pid=31590 [Apr 10 2015 12:24:57 R52415 ]

DiscovarDeNovo \

             READS="frac:0.1,sample:18H::/scratch/bananaSlugBAMS/SW018_S1_L0 \

             07_001.bam+frac:0.1,sample:19M::/scratch/bananaSlugBAMS/SW019_S \

             1_L001_001.bam+frac:0.1,sample:19H::/scratch/bananaSlugBAMS/SW0 \

             19_S2_L008_001.bam"                                             \

             OUT_DIR=/scratch/bananaSlugAssemblies/10run/

---

SYSTEM INFO

- OS: Linux :: 3.5.0-54-generic :: #81~precise1-Ubuntu SMP Tue Jul 15 04:02:22 UTC 2014

- node name: edser2

- hardware type: x86_64

- cache size: 30720 KB

- cpu MHz: 2699.924

- cpu model name: Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz

- physical memory: 314.88 GB

Omitting memory check. If you run into problems with memory,

you might try rerunning with MEMORY_CHECK=True.

Mon Apr 27 22:01:36 2015: finding input files

Mon Apr 27 22:01:36 2015: reading 3 files (which may take a while)

Mon Apr 27 22:01:36 2015: processing /scratch/bananaSlugBAMS/SW018_S1_L007_001.bam.

Mon Apr 27 22:01:36 2015: memory in use = 0.01 GB, peak = 0.01 GB

Mon Apr 27 22:24:52 2015: there are 28,603,868 reads of mean length 100

Mon Apr 27 22:24:52 2015: memory in use = 42.71 GB, peak = 43.92 GB

Mon Apr 27 22:25:28 2015: reads sorted

Mon Apr 27 22:25:28 2015: memory in use = 43.78 GB, peak = 43.92 GB

Mon Apr 27 22:27:33 2015: data stashed in output structures

Mon Apr 27 22:27:33 2015: memory in use = 2.64 GB, peak = 43.92 GB

Mon Apr 27 22:27:33 2015: processing /scratch/bananaSlugBAMS/SW019_S1_L001_001.bam.

Mon Apr 27 22:27:33 2015: memory in use = 2.64 GB, peak = 43.92 GB

Mon Apr 27 22:46:24 2015: there are 5,453,744 reads of mean length 301

Mon Apr 27 22:46:24 2015: memory in use = 18.61 GB, peak = 43.92 GB

Mon Apr 27 22:46:32 2015: reads sorted

Mon Apr 27 22:46:32 2015: memory in use = 18.81 GB, peak = 43.92 GB

Mon Apr 27 22:47:17 2015: data stashed in output structures

Mon Apr 27 22:47:17 2015: memory in use = 3.86 GB, peak = 43.92 GB

Mon Apr 27 22:47:17 2015: processing /scratch/bananaSlugBAMS/SW019_S2_L008_001.bam.

Mon Apr 27 22:47:17 2015: memory in use = 3.86 GB, peak = 43.92 GB

Mon Apr 27 23:02:58 2015: there are 19,727,231 reads of mean length 100

Mon Apr 27 23:02:58 2015: memory in use = 33.52 GB, peak = 43.92 GB

Mon Apr 27 23:03:22 2015: reads sorted

Mon Apr 27 23:03:22 2015: memory in use = 34.25 GB, peak = 43.92 GB

Mon Apr 27 23:04:52 2015: data stashed in output structures

Mon Apr 27 23:04:52 2015: memory in use = 5.65 GB, peak = 43.92 GB

INPUT FILES:

[1,type=frag,sample=18H,lib=1,frac=0.1] /scratch/bananaSlugBAMS/SW018_S1_L007_001.bam

[2,type=frag,sample=19M,lib=1,frac=0.1] /scratch/bananaSlugBAMS/SW019_S1_L001_001.bam

[3,type=frag,sample=19H,lib=1,frac=0.1] /scratch/bananaSlugBAMS/SW019_S2_L008_001.bam

Mon Apr 27 23:04:52 2015: found 3 samples

Mon Apr 27 23:04:52 2015: starts = 0,28603868,34057612

Mon Apr 27 23:06:29 2015: using 53,784,844 reads

Mon Apr 27 23:06:29 2015: data extraction complete

1.08 hours used extracting reads

Mon Apr 27 23:06:31 2015: see total physical memory of 338,094,714,880 bytes

Mon Apr 27 23:06:31 2015: 52.59 bytes per read base, assuming max memory available

We need 1 passes.

Expect 2750583 keys per batch.

Provide 5342050 keys per batch.

There were 192 buffer overflows.

We need 1 passes.

Expect 2750583 keys per batch.

Provide 5103062 keys per batch.

There were 192 buffer overflows.

Warning: HashSet initial size too small.

Mon Apr 27 23:36:51 2015: back from buildReadQGraph

memory in use = 13,995,552,768

checksum_60 = 759152776190314974

Mon Apr 27 23:37:57 2015: constructing places

Mon Apr 27 23:38:15 2015: sorting places

Mon Apr 27 23:41:26 2015: building all

Mon Apr 27 23:43:07 2015: calling LongReadsToPaths

Mon Apr 27 23:44:14 2015: writing

Mon Apr 27 23:44:21 2015: translating paths

Mon Apr 27 23:44:34 2015: final stage of path translation

Mon Apr 27 23:58:35 2015: writing paths

9.52 seconds used reloading assembly

Tue Apr 28 00:00:26 2015: start walking

memory in use = 7,987,261,440

Tue Apr 28 00:01:04 2015: start walking

memory in use = 7,825,833,984

1.21 minutes used cleaning 200-mer graph

55.5 minutes used in ReadQGrapher

5.96e-06 seconds used reloading reads

checksum_200 = 23687290918476347

1 peak mem usage = 151.05 GB

10.5 seconds used loading stuff

2 peak mem usage = 151.05 GB

launching gap assemblies, mem usage = 7,615,533,056

Tue Apr 28 00:02:26 2015: finding unsatisfieds

Tue Apr 28 00:02:27 2015: creating multiplicity map

Tue Apr 28 00:02:27 2015: economizing links

Tue Apr 28 00:02:27 2015: forming neighborhoods

Tue Apr 28 00:02:28 2015: forming initial clusters

Tue Apr 28 00:02:28 2015: start sort

0.086 seconds used sorting

Tue Apr 28 00:02:28 2015: merging clusters

xs.size( ) = 98897

7.32 seconds used merging

xs.size( ) = 16644

Tue Apr 28 00:02:36 2015: start overlap-based merging

Tue Apr 28 00:02:37 2015: start overlap-based merging

LR.size( ) = 15709

LR.size( ) = 7868

Tue Apr 28 00:03:02 2015: now processing 7868 blobs

Tue Apr 28 00:03:02 2015: memory in use = 7.22 GB, peak = 151.05 GB

………. ………. ………. ………. ……….

………. ………. ………. ………. ……….

10 minutes spent in local assemblies, memory in use = 7.40 GB, peak = 151.05 GB

Tue Apr 28 00:13:03 2015: patch reserving space

Tue Apr 28 00:13:03 2015: memory in use = 7.40 GB

0.39 seconds used patching, peak mem usage = 151.05 GB

new_stuff.size( ) = 51903

Tue Apr 28 00:13:13 2015: building hb2

5.14 seconds used in new stuff 1 test

memory in use now = 7,876,759,552

Warning: HashSet initial size too small.

Tue Apr 28 00:14:26 2015: back from buildBigKHBVFromReads

1.22 minutes used in new stuff 2 test

peak mem usage = 151.05 GB

11.8 seconds used in new stuff 5

Tue Apr 28 00:14:45 2015: finding interesting reads

Tue Apr 28 00:14:45 2015: memory in use = 7.29 GB, peak = 151.05 GB

Tue Apr 28 00:15:41 2015: building dictionary

Tue Apr 28 00:15:41 2015: memory in use = 7.30 GB, peak = 151.05 GB

Tue Apr 28 00:15:54 2015: reducing

Tue Apr 28 00:15:54 2015: memory in use = 13.40 GB, peak = 151.05 GB

We need 1 passes.

Expect 57267 keys per batch.

Provide 143166 keys per batch.

Tue Apr 28 00:16:18 2015: kmerizing

Tue Apr 28 00:16:18 2015: memory in use = 13.88 GB, peak = 151.05 GB

We need 1 passes.

Expect 77877 keys per batch.

Provide 194692 keys per batch.

Tue Apr 28 00:16:29 2015: cleaning

Tue Apr 28 00:16:29 2015: memory in use = 13.88 GB, peak = 151.05 GB

Tue Apr 28 00:16:42 2015: finding uniquely aligning edges

Tue Apr 28 00:16:42 2015: memory in use = 13.88 GB, peak = 151.05 GB

3.89 minutes used in new phase

hb.N( ) = 1180208, hb.EdgeObjectCount( ) = 741914

1867 paths improved by rerouting

Sum(invalid) = 898, npids = 26892422

271 edges tamped down

Tue Apr 28 00:18:34 2015: checking involution

Tue Apr 28 00:18:34 2015: done

WARNING: 235 suspicious read-paths.

Sum(invalid) = 564, npids = 26892422

216 edges tamped down

Tue Apr 28 00:19:44 2015: making paths index for pull apart

Tue Apr 28 00:19:48 2015: pulling apart repeats

0.0336 seconds used separating paths 1

2.87 seconds used in fixing mToLeft, mToRight, and mEdgeToPathIds

Tue Apr 28 00:19:54 2015: there were 234 repeats pulled apart.

Tue Apr 28 00:19:54 2015: there were 2055 read paths removed during separation.

Tue Apr 28 00:19:56 2015: improving paths

Tue Apr 28 00:21:03 2015: done

44210 paths extended

Tue Apr 28 00:21:50 2015: start degloop

Tue Apr 28 00:21:50 2015: creating path index

Tue Apr 28 00:21:59 2015: starting loop

Tue Apr 28 00:22:02 2015: degloop complete

Tue Apr 28 00:22:19 2015: unwinding three-edge plasmids

Tue Apr 28 00:22:20 2015: removing small components

Tue Apr 28 00:22:54 2015: writing a.fin files

Tue Apr 28 00:23:48 2015: determining candidates

Tue Apr 28 00:23:48 2015: determining candidates

Tue Apr 28 00:23:48 2015: determining candidates

CN fraction good = 0.18

/bin/sh: 1: set: Illegal option -o pipefail

Tue Apr 28 00:24:04 2015: deleting 0 gaps and adding 0 gaps to force symmetry

Tue Apr 28 00:24:11 2015: done making gaps, time used = 9.07 seconds

Tue Apr 28 00:24:35 2015: determining candidates

Tue Apr 28 00:24:35 2015: determining candidates

Tue Apr 28 00:24:35 2015: determining candidates

0.204 seconds using setting up final fasta

0.833 seconds using printing final fasta