archive:bioinformatic_tools:abyss
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
archive:bioinformatic_tools:abyss [2010/05/19 17:46] jstjohn |
archive:bioinformatic_tools:abyss [2010/05/20 20:48] jstjohn |
||
|---|---|---|---|
| Line 238: | Line 238: | ||
| <code> | <code> | ||
| - | /campus/BME235/programs/abyss_tmp/bin/abyss-pe mpirun="/opt/openmpi/bin/mpirun -machinefile machines -x PATH=/campus/BME235/bin/programs/abyss_tmp/bin:$PATH" np=23 n=8 k=28 name=slugAbyss lib='lane1 lane2 lane3 lane5 lane6 lane7 lane8' lane1='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_1_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_1_2_all_qseq.fastq' lane2='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_2_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_2_2_all_qseq.fastq' lane3='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_3_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_3_2_all_qseq.fastq' lane5='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_5_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_5_2_all_qseq.fastq' lane6='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_6_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_6_2_all_qseq.fastq' lane7='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_7_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_7_2_all_qseq.fastq' lane8='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_8_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_8_2_all_qseq.fastq' | + | /campus/BME235/programs/abyss_tmp/bin/abyss-pe mpirun="/opt/openmpi/bin/mpirun -machinefile machines -x PATH=/campus/BME235/bin/programs/abyss_tmp/bin:$PATH" np=60 n=8 k=28 name=slugAbyss lib='lane1 lane2 lane3 lane5 lane6 lane7 lane8' lane1='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_1_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_1_2_all_qseq.fastq' lane2='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_2_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_2_2_all_qseq.fastq' lane3='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_3_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_3_2_all_qseq.fastq' lane5='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_5_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_5_2_all_qseq.fastq' lane6='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_6_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_6_2_all_qseq.fastq' lane7='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_7_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_7_2_all_qseq.fastq' lane8='/campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_8_1_all_qseq.fastq /campus/BME235/data/slug/Illumina/illumina_run_1/CeleraReads/s_8_2_all_qseq.fastq' |
| </code> | </code> | ||
| Also because the makefile crashed, it didn't get a chance to clean up the output from the previous step. I had to manually delete the lane-x-3.hist files (which were all of size 0 anyway). After doing this the makefile was able to pick up where it left off and re-generate the lane-x-3.hist files. | Also because the makefile crashed, it didn't get a chance to clean up the output from the previous step. I had to manually delete the lane-x-3.hist files (which were all of size 0 anyway). After doing this the makefile was able to pick up where it left off and re-generate the lane-x-3.hist files. | ||
| - | There is some error where the laneX.hist files are empty... | + | Campusrocks-0-6.local is back up so I am re-starting this task. At its peak the KAligner step (where it crashed previously when the -j option was enabled) requires quite a lot of ram. I am hoping that the 30GB available on this node is sufficient. |
| ====Attempt 4==== | ====Attempt 4==== | ||
| I have access to kolossus which has 1.1tb of ram. I will now run the program on kolossus to see if it will assemble there... | I have access to kolossus which has 1.1tb of ram. I will now run the program on kolossus to see if it will assemble there... | ||
| Line 278: | Line 279: | ||
| Near the height I have observed this is eating up about 50G of ram, but the issue appears to be in available space for the sort algorithm in kolossus's /tmp/ directory. I am trying this again so I can copy down the error and send it to cluster-admin because kolossus should have around 400GB free of local HD space on top of its 1.1TB of ram. (kolossus has more ram than HD space: 1.1TB of ram vs 750GB hd) | Near the height I have observed this is eating up about 50G of ram, but the issue appears to be in available space for the sort algorithm in kolossus's /tmp/ directory. I am trying this again so I can copy down the error and send it to cluster-admin because kolossus should have around 400GB free of local HD space on top of its 1.1TB of ram. (kolossus has more ram than HD space: 1.1TB of ram vs 750GB hd) | ||
| + | To get around the issue of sort running out of memory in its temp directory, I found an alternate command where you can supply your own temp directory to sort. Since there is plenty of room left on the hive I issue the following command to generate the files myself. The nice thing is that since this is a makefile, once I have done this I can simply re-start the assembler, and it will see the files I have manually generated and move on to the next step. | ||
| + | <code> | ||
| + | KAligner -j4 -k35 /scratch/galt/bananaSlug/slug_1.fastq /scratch/galt/bananaSlug/slug_2.fastq slugAbyss3-3.fa \ | ||
| + | |ParseAligns -k35 -h lib1-3.hist \ | ||
| + | |sort -T /hive/users/jstjohn/slugAssembly/tmp -nk2,2 \ | ||
| + | |gzip >lib1-3.pair.gz | ||
| + | </code> | ||
archive/bioinformatic_tools/abyss.txt · Last modified: 2015/07/28 06:23 by ceisenhart
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
