---

Wed Jun 03 12:55:08 2015 run on kolossus, pid=15858 [Jun 2 2015 19:08:18 R52488 ]

DiscovarDeNovo \

             READS=sample:MK::/scratch/ceisenhart/bams/BS-MK_seqprep_dupRemo \

             ved.bam+sample:tag::/scratch/ceisenhart/bams/BS_tag_noAdap_noDu \

             p.bam+sample:Mi19::/scratch/ceisenhart/bams/SW019_MiSeq_adapter \

             Trimmed_dupRemoved.bam+sample:UC18::/scratch/ceisenhart/bams/UC \

             SF_SW018_noAdap_noDup.bam+sample:UC19::/scratch/ceisenhart/bams \

             /UCSF_SW019_noAdap_noDup.bam NUM_THREADS=16 MEMORY_CHECK=True   \

             OUT_DIR=/hive/users/ceisenhart/06.03.2015Assembly               \

             MAX_MEM_GB=750

---

SYSTEM INFO

- OS: Linux :: 2.6.32-504.3.3.el6.x86_64 :: #1 SMP Wed Dec 17 01:55:02 UTC 2014

- node name: kolossus

- hardware type: x86_64

- cache size: 24576 KB

- cpu MHz: 2260.913

- cpu model name: Intel(R) Xeon(R) CPU X7560 @ 2.27GHz

- physical memory: 1009.71 GB

MEMORY CHECK (typically takes several minutes; could cause

machine to become sluggish or result in this job being killed)

- Apparently able to allocate 100% of nominally available memory.

- Can access at least 750 GB.

Wed Jun 03 12:57:05 2015: finding input files

Wed Jun 03 12:57:05 2015: reading 5 files (which may take a while)

Wed Jun 03 12:57:05 2015: processing /scratch/ceisenhart/bams/BS-MK_seqprep_dupRemoved.bam.

Wed Jun 03 12:57:05 2015: memory in use = 0.01 GB, peak = 0.01 GB

Wed Jun 03 14:01:01 2015: there are 249,569,918 reads of mean length 250

Wed Jun 03 14:01:01 2015: memory in use = 111.46 GB, peak = 109.00 GB

Wed Jun 03 14:01:19 2015: reads sorted

Wed Jun 03 14:01:19 2015: memory in use = 112.40 GB, peak = 110.68 GB

Wed Jun 03 14:04:02 2015: data stashed in output structures

Wed Jun 03 14:04:02 2015: memory in use = 68.22 GB, peak = 120.57 GB

Wed Jun 03 14:04:02 2015: processing /scratch/ceisenhart/bams/BS_tag_noAdap_noDup.bam.

Wed Jun 03 14:04:02 2015: memory in use = 68.22 GB, peak = 120.57 GB

Wed Jun 03 14:36:30 2015: there are 135,779,818 reads of mean length 249

Wed Jun 03 14:36:30 2015: memory in use = 106.51 GB, peak = 120.57 GB

Wed Jun 03 14:36:40 2015: reads sorted

Wed Jun 03 14:36:40 2015: memory in use = 107.02 GB, peak = 120.57 GB

Wed Jun 03 14:38:13 2015: data stashed in output structures

Wed Jun 03 14:38:13 2015: memory in use = 100.67 GB, peak = 120.57 GB

Wed Jun 03 14:38:13 2015: processing /scratch/ceisenhart/bams/SW019_MiSeq_adapterTrimmed_dupRemoved.bam.

Wed Jun 03 14:38:13 2015: memory in use = 100.67 GB, peak = 120.57 GB

Wed Jun 03 14:55:49 2015: there are 54,509,730 reads of mean length 300

Wed Jun 03 14:55:49 2015: memory in use = 117.09 GB, peak = 120.57 GB

Wed Jun 03 14:55:53 2015: reads sorted

Wed Jun 03 14:55:53 2015: memory in use = 117.29 GB, peak = 120.57 GB

Wed Jun 03 14:56:43 2015: data stashed in output structures

Wed Jun 03 14:56:43 2015: memory in use = 115.79 GB, peak = 120.98 GB

Wed Jun 03 14:56:43 2015: processing /scratch/ceisenhart/bams/UCSF_SW018_noAdap_noDup.bam.

Wed Jun 03 14:56:43 2015: memory in use = 115.79 GB, peak = 120.98 GB

Wed Jun 03 15:21:40 2015: there are 113,862,306 reads of mean length 232

Wed Jun 03 15:21:40 2015: memory in use = 155.64 GB, peak = 152.54 GB

Wed Jun 03 15:21:48 2015: reads sorted

Wed Jun 03 15:21:48 2015: memory in use = 156.06 GB, peak = 152.83 GB

Wed Jun 03 15:23:10 2015: data stashed in output structures

Wed Jun 03 15:23:10 2015: memory in use = 148.62 GB, peak = 163.18 GB

Wed Jun 03 15:23:10 2015: processing /scratch/ceisenhart/bams/UCSF_SW019_noAdap_noDup.bam.

Wed Jun 03 15:23:10 2015: memory in use = 148.62 GB, peak = 163.18 GB

Wed Jun 03 15:59:42 2015: there are 150,162,130 reads of mean length 249

Wed Jun 03 15:59:42 2015: memory in use = 206.37 GB, peak = 202.74 GB

Wed Jun 03 15:59:53 2015: reads sorted

Wed Jun 03 15:59:53 2015: memory in use = 206.93 GB, peak = 202.74 GB

Wed Jun 03 16:01:41 2015: data stashed in output structures

Wed Jun 03 16:01:41 2015: memory in use = 198.18 GB, peak = 222.28 GB

INPUT FILES:

[1,type=frag,sample=MK,lib=1,frac=1] /scratch/ceisenhart/bams/BS-MK_seqprep_dupRemoved.bam

[2,type=frag,sample=tag,lib=1,frac=1] /scratch/ceisenhart/bams/BS_tag_noAdap_noDup.bam

[3,type=frag,sample=Mi19,lib=1,frac=1] /scratch/ceisenhart/bams/SW019_MiSeq_adapterTrimmed_dupRemoved.bam

[4,type=frag,sample=UC18,lib=1,frac=1] /scratch/ceisenhart/bams/UCSF_SW018_noAdap_noDup.bam

[5,type=frag,sample=UC19,lib=1,frac=1] /scratch/ceisenhart/bams/UCSF_SW019_noAdap_noDup.bam

Wed Jun 03 16:01:41 2015: found 5 samples

Wed Jun 03 16:01:41 2015: starts = 0,249569918,385349736,439859466,553721772

Wed Jun 03 16:11:44 2015: using 703,883,902 reads

Wed Jun 03 16:11:44 2015: data extraction complete, peak mem = 222.28 GB

3.24 hours used extracting reads

Wed Jun 03 16:25:48 2015: see total physical memory of 1,084,169,011,200 bytes

Wed Jun 03 16:25:48 2015: see user-imposed limit on memory of 805,306,368,000 bytes

Wed Jun 03 16:25:48 2015: 4.56 bytes per read base, assuming max memory available

We need 7 passes.

Expect 87893563 keys per batch.

Provide 98925162 keys per batch.

We need 9 passes.

Expect 68361660 keys per batch.

Provide 85294165 keys per batch.

Wed Jun 03 23:16:11 2015: back from buildReadQGraph

Wed Jun 03 23:16:11 2015: memory in use = 249.86 GB, peak = 583.25 GB

checksum_60 = 1500661769504106748

Wed Jun 03 23:21:36 2015: constructing places

Wed Jun 03 23:23:46 2015: sorting places

Wed Jun 03 23:39:21 2015: building all

Thu Jun 04 00:00:20 2015: calling LongReadsToPaths

Thu Jun 04 00:56:43 2015: writing

Thu Jun 04 01:00:50 2015: translating paths

Thu Jun 04 01:12:34 2015: final stage of path translation

Thu Jun 04 02:43:59 2015: writing paths

2.67 minutes used reloading assembly

Thu Jun 04 03:13:06 2015: start walking

memory in use = 217,802,301,440

Thu Jun 04 03:40:40 2015: start walking

memory in use = 206,994,972,672

1.04 hours used cleaning 200-mer graph

11.9 hours used in ReadQGrapher

4.05e-06 seconds used reloading reads

checksum_200 = 1499603233170572930

1 peak mem usage = 583.25 GB

2.56 minutes used loading stuff

2 peak mem usage = 583.25 GB

launching gap assemblies, mem usage = 189,699,788,800

Thu Jun 04 04:25:49 2015: finding unsatisfieds

Thu Jun 04 04:26:44 2015: creating multiplicity map

Thu Jun 04 04:26:53 2015: economizing links

Thu Jun 04 04:26:54 2015: forming neighborhoods

Thu Jun 04 04:28:24 2015: forming initial clusters

Thu Jun 04 04:29:57 2015: start sort

5.34 seconds used sorting

Thu Jun 04 04:30:04 2015: merging clusters

xs.size( ) = 21565962

45.5 minutes used merging

xs.size( ) = 1545255

Thu Jun 04 05:16:33 2015: start overlap-based merging

Thu Jun 04 05:17:51 2015: start overlap-based merging

LR.size( ) = 1327468

LR.size( ) = 665564

Thu Jun 04 05:40:49 2015: now processing 665564 blobs

Thu Jun 04 05:40:49 2015: memory in use = 204.69 GB, peak = 583.25 GB

………. ………. ………. ………. ……….

………. ………. ………. ………. ……….

3.34 days spent in local assemblies, memory in use = 228.88 GB, peak = 583.25 GB

Sun Jun 07 13:44:52 2015: patch reserving space

Sun Jun 07 13:44:52 2015: memory in use = 228.89 GB

35.4 seconds used patching, peak mem usage = 583.25 GB

new_stuff.size( ) = 7741492

Sun Jun 07 13:50:24 2015: building hb2

3.58 minutes used in new stuff 1 test

memory in use now = 211,686,506,496

Warning: HashSet initial size too small.

Sun Jun 07 14:37:56 2015: back from buildBigKHBVFromReads

47.6 minutes used in new stuff 2 test

peak mem usage = 583.25 GB

8 minutes used in new stuff 5

Sun Jun 07 14:51:06 2015: finding interesting reads

Sun Jun 07 14:51:06 2015: memory in use = 179.70 GB, peak = 583.25 GB

Sun Jun 07 15:19:25 2015: building dictionary

Sun Jun 07 15:19:25 2015: memory in use = 179.87 GB, peak = 583.25 GB

Sun Jun 07 15:21:35 2015: reducing

Sun Jun 07 15:21:35 2015: memory in use = 382.82 GB, peak = 583.25 GB

We need 1 passes.

Expect 19835364 keys per batch.

Provide 49588410 keys per batch.

Sun Jun 07 15:26:00 2015: kmerizing

Sun Jun 07 15:26:00 2015: memory in use = 403.09 GB, peak = 583.25 GB

We need 1 passes.

Expect 53444203 keys per batch.

Provide 133610506 keys per batch.

Sun Jun 07 15:31:52 2015: cleaning

Sun Jun 07 15:31:52 2015: memory in use = 403.09 GB, peak = 583.25 GB

Sun Jun 07 15:33:51 2015: finding uniquely aligning edges

Sun Jun 07 15:33:51 2015: memory in use = 403.10 GB, peak = 583.25 GB

1.88 hours used in new phase

hb.N( ) = 63606048, hb.EdgeObjectCount( ) = 48291486

107401 paths improved by rerouting

Sum(invalid) = 1610553, npids = 351941951

22051 edges tamped down

Sun Jun 07 17:00:54 2015: checking involution

Sun Jun 07 17:00:54 2015: done

WARNING: 75298 suspicious read-paths.

Sum(invalid) = 296088, npids = 351941951

11865 edges tamped down

Sun Jun 07 17:34:43 2015: making paths index for pull apart

Sun Jun 07 17:37:11 2015: pulling apart repeats

4.85 seconds used separating paths 1

2.98 minutes used in fixing mToLeft, mToRight, and mEdgeToPathIds

Sun Jun 07 17:44:25 2015: there were 65104 repeats pulled apart.

Sun Jun 07 17:44:25 2015: there were 534710 read paths removed during separation.

Sun Jun 07 17:46:20 2015: improving paths

Sun Jun 07 18:10:36 2015: done

10902131 paths extended

Sun Jun 07 18:26:51 2015: start degloop

Sun Jun 07 18:26:51 2015: creating path index

Sun Jun 07 18:29:43 2015: starting loop

Sun Jun 07 18:34:34 2015: degloop complete

Sun Jun 07 18:40:51 2015: unwinding three-edge plasmids

Sun Jun 07 18:40:59 2015: removing small components

Sun Jun 07 18:50:10 2015: writing a.fin files

Sun Jun 07 19:09:01 2015: determining candidates

Sun Jun 07 19:09:22 2015: determining candidates

Sun Jun 07 19:09:39 2015: determining candidates

Sun Jun 07 19:09:58 2015: determining candidates

Sun Jun 07 19:10:14 2015: determining candidates

CN fraction good = 0.57

Sun Jun 07 19:16:36 2015: deleting 4 gaps and adding 57 gaps to force symmetry

Sun Jun 07 19:20:17 2015: done making gaps, time used = 6 minutes

Sun Jun 07 19:32:19 2015: determining candidates

Sun Jun 07 19:32:35 2015: determining candidates

Sun Jun 07 19:32:47 2015: determining candidates

Sun Jun 07 19:33:01 2015: determining candidates

Sun Jun 07 19:33:14 2015: determining candidates

20.6 seconds using setting up final fasta

3.34 minutes using printing final fasta