User Tools

Site Tools


contributors:team_5:50perrunucsf

IPerforming re-exec to adjust stack size.


Sun May 10 10:10:33 2015 run on campusrocks2-0-0, pid=10320 [Apr 4 2015 20:59:59 R52415 ]

DiscovarDeNovo \

             READS="frac:0.5,sample:19::/campusdata/ndudek/fastq_to_bam/UCSF \
             _SW019_noAdap_noDup.bam+frac:0.5,sample:18::/campusdata/ndudek/ \
             fastq_to_bam/UCSF_SW018_noAdap_noDup.bam"                       \
             OUT_DIR=/campusdata/rcalef/0.5_dataDDN_150501 NUM_THREADS=39    \
             MAX_MEM_GB=220 MEMORY_CHECK=True

SYSTEM INFO

- OS: Linux :: 2.6.32-220.13.1.el6.x86_64 :: #1 SMP Tue Apr 17 23:56:34 BST 2012

- node name: campusrocks2-0-0.local

- hardware type: x86_64

- cache size: 2048 KB

- cpu MHz: 2400.042

- cpu model name: AMD Opteron™ Processor 6278

- physical memory: 252.39 GB

MEMORY CHECK (typically takes several minutes; could cause

machine to become sluggish or result in this job being killed)

- Apparently able to allocate 100% of nominally available memory.

- Can access at least 220 GB.

Sun May 10 10:17:29 2015: finding input files

Sun May 10 10:17:29 2015: reading 2 files (which may take a while)

Sun May 10 10:17:29 2015: processing /campusdata/ndudek/fastq_to_bam/UCSF_SW019_noAdap_noDup.bam.

Sun May 10 10:17:29 2015: memory in use = 0.01 GB, peak = 0.01 GB

Sun May 10 10:54:12 2015: there are 75,081,065 reads of mean length 249

Sun May 10 10:54:12 2015: memory in use = 66.13 GB, peak = 67.76 GB

Sun May 10 10:54:24 2015: reads sorted

Sun May 10 10:54:24 2015: memory in use = 66.70 GB, peak = 67.76 GB

Sun May 10 10:56:44 2015: data stashed in output structures

Sun May 10 10:56:44 2015: memory in use = 15.53 GB, peak = 67.76 GB

Sun May 10 10:56:44 2015: processing /campusdata/ndudek/fastq_to_bam/UCSF_SW018_noAdap_noDup.bam.

Sun May 10 10:56:44 2015: memory in use = 15.53 GB, peak = 67.76 GB

Sun May 10 11:21:24 2015: there are 56,931,153 reads of mean length 232

Sun May 10 11:21:24 2015: memory in use = 49.12 GB, peak = 67.76 GB

Sun May 10 11:21:34 2015: reads sorted

Sun May 10 11:21:34 2015: memory in use = 49.55 GB, peak = 67.76 GB

Sun May 10 11:23:13 2015: data stashed in output structures

Sun May 10 11:23:13 2015: memory in use = 25.97 GB, peak = 67.76 GB

INPUT FILES:

[1,type=frag,sample=19,lib=1,frac=0.5] /campusdata/ndudek/fastq_to_bam/UCSF_SW019_noAdap_noDup.bam

[2,type=frag,sample=18,lib=1,frac=0.5] /campusdata/ndudek/fastq_to_bam/UCSF_SW018_noAdap_noDup.bam

Sun May 10 11:23:13 2015: found 2 samples

Sun May 10 11:23:13 2015: starts = 0,75081066

Sun May 10 11:30:40 2015: using 132,012,220 reads

Sun May 10 11:30:40 2015: data extraction complete

1.22 hours used extracting reads

Sun May 10 11:31:08 2015: see total physical memory of 270,999,298,048 bytes

Sun May 10 11:31:08 2015: see user-imposed limit on memory of 236,223,201,280 bytes

Sun May 10 11:31:08 2015: 7.38 bytes per read base, assuming max memory available

We need 4 passes.

Expect 4797148 keys per batch.

Provide 5352492 keys per batch.

We need 11 passes.

Expect 1744417 keys per batch.

Provide 2097642 keys per batch.

Sun May 10 12:37:53 2015: back from buildReadQGraph

memory in use = 53,805,211,648

checksum_60 = 1093167177637492236

Sun May 10 12:40:09 2015: constructing places

Sun May 10 12:40:38 2015: sorting places

Sun May 10 12:44:01 2015: building all

Sun May 10 12:48:13 2015: calling LongReadsToPaths

Sun May 10 13:07:11 2015: writing

Sun May 10 13:09:42 2015: translating paths

Sun May 10 13:12:35 2015: final stage of path translation

Sun May 10 13:28:00 2015: writing paths

1.06 minutes used reloading assembly

Sun May 10 13:37:55 2015: start walking

memory in use = 41,533,923,328

Sun May 10 13:42:17 2015: start walking

memory in use = 41,263,316,992

9.69 minutes used cleaning 200-mer graph

2.32 hours used in ReadQGrapher

5.01e-06 seconds used reloading reads

checksum_200 = 1086642745387393140

1 peak mem usage = 184.09 GB

1.59 minutes used loading stuff

2 peak mem usage = 184.09 GB

launching gap assemblies, mem usage = 36,172,001,280

Sun May 10 13:53:11 2015: finding unsatisfieds

Sun May 10 13:53:20 2015: creating multiplicity map

Sun May 10 13:53:25 2015: economizing links

Sun May 10 13:53:25 2015: forming neighborhoods

Sun May 10 13:53:41 2015: forming initial clusters

Sun May 10 13:54:09 2015: start sort

1.61 seconds used sorting

Sun May 10 13:54:11 2015: merging clusters

xs.size( ) = 6700008

19.3 minutes used merging

xs.size( ) = 2320189

Sun May 10 14:13:53 2015: start overlap-based merging

Sun May 10 14:14:59 2015: start overlap-based merging

LR.size( ) = 2265143

LR.size( ) = 1132970

Sun May 10 14:20:39 2015: now processing 1132970 blobs

Sun May 10 14:20:39 2015: memory in use = 39.29 GB, peak = 184.09 GB

………. ………. ………. ………. ……….

………. ………. ………. ………. ……….

8.31 hours spent in local assemblies, memory in use = 41.74 GB, peak = 184.09 GB

Sun May 10 22:39:31 2015: patch reserving space

Sun May 10 22:39:31 2015: memory in use = 41.75 GB

35.9 seconds used patching, peak mem usage = 184.09 GB

new_stuff.size( ) = 4344323

Sun May 10 22:41:53 2015: building hb2

55.3 seconds used in new stuff 1 test

memory in use now = 40,122,859,520

Sun May 10 23:11:37 2015: back from buildBigKHBVFromReads

29.8 minutes used in new stuff 2 test

peak mem usage = 184.09 GB

1.23 minutes used in new stuff 5

Sun May 10 23:14:10 2015: finding interesting reads

Sun May 10 23:14:10 2015: memory in use = 34.11 GB, peak = 184.09 GB

Sun May 10 23:24:04 2015: building dictionary

Sun May 10 23:24:04 2015: memory in use = 34.17 GB, peak = 184.09 GB

Sun May 10 23:25:54 2015: reducing

Sun May 10 23:25:54 2015: memory in use = 134.54 GB, peak = 184.09 GB

We need 1 passes.

Expect 1140988 keys per batch.

Provide 2730092 keys per batch.

Sun May 10 23:28:51 2015: kmerizing

Sun May 10 23:28:51 2015: memory in use = 143.29 GB, peak = 184.09 GB

We need 2 passes.

Expect 2094178 keys per batch.

Provide 4706876 keys per batch.

There were 234 buffer overflows.

Sun May 10 23:37:01 2015: cleaning

Sun May 10 23:37:01 2015: memory in use = 143.29 GB, peak = 184.09 GB

Sun May 10 23:38:28 2015: finding uniquely aligning edges

Sun May 10 23:38:28 2015: memory in use = 143.29 GB, peak = 184.09 GB

1 hours used in new phase

hb.N( ) = 18564840, hb.EdgeObjectCount( ) = 14056399

153043 paths improved by rerouting

Sum(invalid) = 241436, npids = 66006110

25876 edges tamped down

Mon May 11 00:06:00 2015: checking involution

Mon May 11 00:06:00 2015: done

WARNING: 15782 suspicious read-paths.

Sum(invalid) = 41027, npids = 66006110

15750 edges tamped down

Mon May 11 00:16:55 2015: making paths index for pull apart

Mon May 11 00:17:19 2015: pulling apart repeats

0.426 seconds used separating paths 1

31.8 seconds used in fixing mToLeft, mToRight, and mEdgeToPathIds

Mon May 11 00:19:01 2015: there were 11047 repeats pulled apart.

Mon May 11 00:19:01 2015: there were 41908 read paths removed during separation.

Mon May 11 00:19:29 2015: improving paths

Mon May 11 00:24:58 2015: done

1306140 paths extended

Mon May 11 00:28:41 2015: start degloop

Mon May 11 00:28:41 2015: creating path index

Mon May 11 00:29:08 2015: starting loop

Mon May 11 00:29:28 2015: degloop complete

Mon May 11 00:31:16 2015: unwinding three-edge plasmids

Mon May 11 00:31:21 2015: removing small components

Mon May 11 00:34:52 2015: writing a.fin files

Mon May 11 00:44:05 2015: determining candidates

Mon May 11 00:44:06 2015: determining candidates

CN fraction good = 0.45

Mon May 11 00:47:00 2015: deleting 0 gaps and adding 8 gaps to force symmetry

Mon May 11 00:48:12 2015: done making gaps, time used = 3.22 minutes

Mon May 11 00:53:16 2015: determining candidates

Mon May 11 00:53:17 2015: determining candidates

18.5 seconds using setting up final fasta

3.39 minutes using printing final fasta

assembly has 3094953 edges of mean length 1062.18

contig line N50: 3,979

scaffold line N50: 3,979

total bases in 1 kb+ scaffolds: 1,528,625,509

total bases in 10 kb+ scaffolds: 137,959,107

There are 132,012,220 reads of mean length 242.4 and mean base quality 33.0.

MPL1 = mean length of first read in pair up to first error = 156

(normal range is 175-225 for 250 base reads)

Estimated chimera rate in read pairs (including mismapping) = 0.07%.

genomic read coverage, using 1 kb+ scaffolds for genome size estimate: 20.9

run started Sun May 10 10:10:33 2015, completed Mon May 11 01:02:09 2015

peak mem usage = 184.09 GB, total time = 14.9 hours

final checksum = 652044377802948288

DiscovarDeNovo READS=“frac:0.5,sample:19::/campusdata/ndudek/fastq_to_bam/UCSF_SW019_noAdap_noDup.bam+frac:0.5,sample:18::/campusdata/ndudek/fastq_to_bam/UCSF_SW018_noAdap_noDup.bam” OUT_DIR=/campusdata/rcalef/0.5_dataDDN_150501 NUM_THREADS=39 MAX_MEM_GB=220 MEMORY_CHECK=True

Mon May 11 01:02:23 2015: done

You could leave a comment if you were logged in.
contributors/team_5/50perrunucsf.txt · Last modified: 2015/09/02 09:22 by ceisenhart