BigDataScript report: atac.bds

Script file /users/dskim89/git/atac_dnase_pipelines/atac.bds
Program ID atac.bds.20170521_181459_745
Start time 2017-05-21 18:14:59
Run time 00:14:46.959
Tasks executed 16
Tasks failed 0
Tasks failed names
 
Arguments* [-species, mm9, -fastq1_1, mouse_1Mreads_rep1.1.fastq.gz, -fastq1_2, mouse_1Mreads_rep1.2.fastq.gz, -out_dir, out, -title, mouse_test]
System* sge
Cpus* -1
Exit value 0
* Values in global scope when program finished execution.

Timeline

Task Graph

Parallel (threads) details

Thread ID Parent Tasks
thread_Root None
 atac.bds.20170521_181459_745/task.callpeak_blacklist_filter.blacklist_filter_peak_1.line_27.id_23
atac.bds.20170521_181459_745/task.callpeak_blacklist_filter.blacklist_filter_gpeak_1.line_27.id_24
atac.bds.20170521_181459_745/task.callpeak_bigbed.peak_to_bigbed_peak_1.line_28.id_33
atac.bds.20170521_181459_745/task.callpeak_bigbed.peak_to_bigbed_gpeak_1.line_28.id_34
atac.bds.20170521_181459_745/task.callpeak_bigbed.peak_to_bigbed_idr_peak_pr.line_28.id_41
atac.bds.20170521_181459_745/task.callpeak_bigbed.peak_to_bigbed_peak_overlap.line_28.id_46
atac.bds.20170521_181459_745/task.callpeak_bigbed.peak_to_bigbed_gpeak_overlap.line_28.id_51
atac.bds.20170521_181459_745/task.atac.tar_all_logs.line_1748.id_54
atac.bds.20170521_181459_745/task.report.peak2hammock.line_413.id_57
atac.bds.20170521_181459_745/task.report.peak2hammock.line_413.id_58
atac.bds.20170521_181459_745/task.report.peak2hammock.line_413.id_59
atac.bds.20170521_181459_745/task.report.peak2hammock.line_413.id_60
atac.bds.20170521_181459_745/task.report.peak2hammock.line_413.id_61
atac.bds.20170521_181459_745/task.graphviz.report.line_98.id_62
atac.bds.20170521_181459_745/task.report.pdf2png.line_380.id_79
 
thread_66 thread_Root
 atac.bds.20170521_181459_745_parallel_66/task.ataqc.ataqc_rep1.line_109.id_22
 
thread_41 thread_Root
  

Task details

Task Execution Time Dependencies Task program, Errors, StdOut / StdErr
Num 1
ID task.ataqc.ataqc_rep1.line_109.id_22
Name ataqc rep1
Thread thread_66
PID 55701
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:15:31
End 2017-05-21 18:22:31
Elapsed 00:07:00
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/align/rep1/mouse_1Mreads_rep1.1.trim.fastq.gz /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/align/rep1/mouse_1Mreads_rep1.2.trim.fastq.gz /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/align/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.bam /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.align.log /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.pbc.qc /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.dup.qc /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/align/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.bam /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/align/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.tagAlign.gz /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/signal/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval.signal.bigwig /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE_qc.html /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE_qc.txt
Dependencies
 
# SYS command. line 111

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 114

 export _JAVA_OPTIONS="-Xms256M -Xmx20G -XX:ParallelGCThreads=1"

# SYS command. line 116

 cd /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1

# SYS command. line 119

 if [ -f "$(which picard)" ]; then export PICARDROOT="$(dirname $(which picard))/../share/picard"*; fi

# SYS command. line 121

 /users/dskim89/git/atac_dnase_pipelines/ataqc/run_ataqc.py \
		    --workdir /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1 \
		    --outdir /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1 \
		    --outprefix mouse_1Mreads_rep1.1.trim.PE2SE \
		    --genome mm9 \
		    --chromsizes /mnt/data/pipeline_genome_data/mm9/mm9.chrom.sizes \
		    --ref /mnt/data/pipeline_genome_data/mm9/mm9.fa \
		    --tss /mnt/data/pipeline_genome_data/mm9/ataqc/mm9_gencode_tss_unique.bed.gz \
		    --dnase /mnt/data/pipeline_genome_data/mm9/ataqc/mm9_univ_dhs_ucsc.from_mm10.bed.gz \
		    --blacklist /mnt/data/pipeline_genome_data/mm9/mm9-blacklist.bed.gz \
		    --prom /mnt/data/pipeline_genome_data/mm9/ataqc/tss_mm9_master.from_mm10.bed.gz \
		    --enh /mnt/data/pipeline_genome_data/mm9/ataqc/mm9_enh_dhs_ucsc.from_mm10.bed.gz \
		    --pbc /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.pbc.qc\
		     --fastq1 /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/align/rep1/mouse_1Mreads_rep1.1.trim.fastq.gz --fastq2 /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/align/rep1/mouse_1Mreads_rep1.2.trim.fastq.gz \
		    --alignedbam /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/align/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.bam \
		    --alignmentlog /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.align.log \
		    --coordsortbam /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/align/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.bam \
		    --duplog /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.dup.qc \
		    --finalbam /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/align/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.bam \
		    --finalbed /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/align/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.tagAlign.gz \
		    --bigwig /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/signal/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval.signal.bigwig \
		    --peaks /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.gz  --naive_overlap_peaks /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.gz  --idr_peaks /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.gz  \
		     --reg2map /mnt/data/pipeline_genome_data/mm9/ataqc/dnase_avgs_merged_named.fseq.vals.gz  --reg2map_bed /mnt/data/pipeline_genome_data/mm9/ataqc/mm9_dhs_universal_ucsc_v1.bed.gz  --meta /mnt/data/pipeline_genome_data/mm9/ataqc/accession_to_name.txt

# SYS command. line 145

 TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0; fi
 
   
--------------------Stdout--------------------
30218 (process ID) old priority 0, new priority 19
Run time: 0:05:52

 
--------------------Stderr--------------------
Picked up _JAVA_OPTIONS: -Xms256M -Xmx20G -XX:ParallelGCThreads=1
ERROR	2017-05-21 18:16:50	ProcessExecutor	Warning messages:
ERROR	2017-05-21 18:16:50	ProcessExecutor	1: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2017-05-21 18:16:50	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
ERROR	2017-05-21 18:16:50	ProcessExecutor	2: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2017-05-21 18:16:50	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
ERROR	2017-05-21 18:16:50	ProcessExecutor	3: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2017-05-21 18:16:50	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
ERROR	2017-05-21 18:16:50	ProcessExecutor	4: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2017-05-21 18:16:50	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
ERROR	2017-05-21 18:16:50	ProcessExecutor	5: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2017-05-21 18:16:50	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
ERROR	2017-05-21 18:16:50	ProcessExecutor	6: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2017-05-21 18:16:50	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
ERROR	2017-05-21 18:16:50	ProcessExecutor	7: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2017-05-21 18:16:50	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
Picked up _JAVA_OPTIONS: -Xms256M -Xmx20G -XX:ParallelGCThreads=1
Picked up _JAVA_OPTIONS: -Xms256M -Xmx20G -XX:ParallelGCThreads=1
Picked up _JAVA_OPTIONS: -Xms256M -Xmx20G -XX:ParallelGCThreads=1
processing chromosomes
/users/dskim89/git/atac_dnase_pipelines/ataqc/run_ataqc.py:915: FutureWarning: sort(columns=....) is deprecated, use sort_values(by=.....)
  sorted_results = merged.sort('corr', ascending=True)

 
Num 2
ID task.callpeak_blacklist_filter.blacklist_filter_peak_1.line_27.id_23
Name blacklist_filter peak 1
Thread thread_Root
PID 39007
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:22:33
End 2017-05-21 18:23:41
Elapsed 00:01:08
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.narrowPeak.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.narrowPeak.gz
Dependencies
 
# SYS command. line 29

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 31

 bedtools intersect -v -a <(zcat -f /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.narrowPeak.gz) -b <(zcat -f /mnt/data/pipeline_genome_data/mm9/mm9-blacklist.bed.gz) \
			| awk 'BEGIN{OFS="\t"} {if ($5>1000) $5=1000; print $0}' \
			| grep -P 'chr[\dXY]+[ \t]' | gzip -nc > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.narrowPeak.gz

# SYS command. line 35

 TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0; fi
 
   
--------------------Stdout--------------------
39012 (process ID) old priority 0, new priority 19
Waiting for 59 seconds.

  
Num 3
ID task.callpeak_blacklist_filter.blacklist_filter_gpeak_1.line_27.id_24
Name blacklist_filter gpeak 1
Thread thread_Root
PID 39024
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:22:33
End 2017-05-21 18:23:41
Elapsed 00:01:08
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.gappedPeak.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.gappedPeak.gz
Dependencies
 
# SYS command. line 29

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 31

 bedtools intersect -v -a <(zcat -f /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.gappedPeak.gz) -b <(zcat -f /mnt/data/pipeline_genome_data/mm9/mm9-blacklist.bed.gz) \
			| awk 'BEGIN{OFS="\t"} {if ($5>1000) $5=1000; print $0}' \
			| grep -P 'chr[\dXY]+[ \t]' | gzip -nc > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.gappedPeak.gz

# SYS command. line 35

 TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0; fi
 
   
--------------------Stdout--------------------
39030 (process ID) old priority 0, new priority 19
Waiting for 58 seconds.

  
Num 4
ID task.callpeak_bigbed.peak_to_bigbed_peak_1.line_28.id_33
Name peak_to_bigbed peak 1
Thread thread_Root
PID 39283
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:23:44
End 2017-05-21 18:24:52
Elapsed 00:01:08
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.narrowPeak.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.narrowPeak.bb
Dependencies
 
# SYS command. line 30

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 32

 zcat /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.narrowPeak.gz | sort -k1,1 -k2,2n > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.narrowPeak.bb.tmp

# SYS command. line 33

 cat /mnt/data/pipeline_genome_data/mm9/mm9.chrom.sizes | grep -P 'chr[\dXY]+[ \t]' > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.narrowPeak.bb.chrsz.tmp

# SYS command. line 34

 bedToBigBed -type=bed6+4 -as=/users/dskim89/git/atac_dnase_pipelines/etc/narrowPeak.as /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.narrowPeak.bb.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.narrowPeak.bb.chrsz.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.narrowPeak.bb

# SYS command. line 35

 rm -f /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.narrowPeak.bb.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.narrowPeak.bb.chrsz.tmp

# SYS command. line 39

 TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0; fi
 
   
--------------------Stdout--------------------
39288 (process ID) old priority 0, new priority 19
Waiting for 55 seconds.

 
--------------------Stderr--------------------
pass1 - making usageList (21 chroms): 177 millis
pass2 - checking and writing primary data (582606 records, 10 fields): 1425 millis

 
Num 5
ID task.callpeak_bigbed.peak_to_bigbed_gpeak_1.line_28.id_34
Name peak_to_bigbed gpeak 1
Thread thread_Root
PID 39302
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:23:45
End 2017-05-21 18:24:53
Elapsed 00:01:08
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.gappedPeak.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.gappedPeak.bb
Dependencies
 
# SYS command. line 30

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 32

 zcat /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.gappedPeak.gz | sort -k1,1 -k2,2n > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.gappedPeak.bb.tmp

# SYS command. line 33

 cat /mnt/data/pipeline_genome_data/mm9/mm9.chrom.sizes | grep -P 'chr[\dXY]+[ \t]' > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.gappedPeak.bb.chrsz.tmp

# SYS command. line 34

 bedToBigBed -type=bed12+3 -as=/users/dskim89/git/atac_dnase_pipelines/etc/gappedPeak.as /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.gappedPeak.bb.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.gappedPeak.bb.chrsz.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.gappedPeak.bb

# SYS command. line 35

 rm -f /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.gappedPeak.bb.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.filt.gappedPeak.bb.chrsz.tmp

# SYS command. line 39

 TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0; fi
 
   
--------------------Stdout--------------------
39307 (process ID) old priority 0, new priority 19
Waiting for 54 seconds.

 
--------------------Stderr--------------------
pass1 - making usageList (21 chroms): 163 millis
pass2 - checking and writing primary data (408587 records, 15 fields): 2815 millis

 
Num 6
ID task.callpeak_bigbed.peak_to_bigbed_idr_peak_pr.line_28.id_41
Name peak_to_bigbed idr peak pr
Thread thread_Root
PID 39505
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:24:56
End 2017-05-21 18:26:05
Elapsed 00:01:08
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.bb
Dependencies
 
# SYS command. line 30

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 32

 zcat /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.gz | sort -k1,1 -k2,2n > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.bb.tmp

# SYS command. line 33

 cat /mnt/data/pipeline_genome_data/mm9/mm9.chrom.sizes | grep -P 'chr[\dXY]+[ \t]' > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.bb.chrsz.tmp

# SYS command. line 34

 bedToBigBed -type=bed6+4 -as=/users/dskim89/git/atac_dnase_pipelines/etc/narrowPeak.as /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.bb.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.bb.chrsz.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.bb

# SYS command. line 35

 rm -f /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.bb.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.bb.chrsz.tmp

# SYS command. line 39

 TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0; fi
 
   
--------------------Stdout--------------------
39510 (process ID) old priority 0, new priority 19
Waiting for 60 seconds.

 
--------------------Stderr--------------------
pass1 - making usageList (20 chroms): 1 millis
pass2 - checking and writing primary data (1686 records, 10 fields): 14 millis

 
Num 7
ID task.callpeak_bigbed.peak_to_bigbed_peak_overlap.line_28.id_46
Name peak_to_bigbed peak overlap
Thread thread_Root
PID 39597
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:26:06
End 2017-05-21 18:27:14
Elapsed 00:01:07
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.bb
Dependencies
 
# SYS command. line 30

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 32

 zcat /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.gz | sort -k1,1 -k2,2n > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.bb.tmp

# SYS command. line 33

 cat /mnt/data/pipeline_genome_data/mm9/mm9.chrom.sizes | grep -P 'chr[\dXY]+[ \t]' > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.bb.chrsz.tmp

# SYS command. line 34

 bedToBigBed -type=bed6+4 -as=/users/dskim89/git/atac_dnase_pipelines/etc/narrowPeak.as /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.bb.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.bb.chrsz.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.bb

# SYS command. line 35

 rm -f /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.bb.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.bb.chrsz.tmp

# SYS command. line 39

 TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0; fi
 
   
--------------------Stdout--------------------
39602 (process ID) old priority 0, new priority 19
Waiting for 56 seconds.

 
--------------------Stderr--------------------
pass1 - making usageList (21 chroms): 169 millis
pass2 - checking and writing primary data (478444 records, 10 fields): 1184 millis

 
Num 8
ID task.callpeak_bigbed.peak_to_bigbed_gpeak_overlap.line_28.id_51
Name peak_to_bigbed gpeak overlap
Thread thread_Root
PID 39686
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:27:16
End 2017-05-21 18:28:23
Elapsed 00:01:07
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.bb
Dependencies
 
# SYS command. line 30

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 32

 zcat /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.gz | sort -k1,1 -k2,2n > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.bb.tmp

# SYS command. line 33

 cat /mnt/data/pipeline_genome_data/mm9/mm9.chrom.sizes | grep -P 'chr[\dXY]+[ \t]' > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.bb.chrsz.tmp

# SYS command. line 34

 bedToBigBed -type=bed12+3 -as=/users/dskim89/git/atac_dnase_pipelines/etc/gappedPeak.as /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.bb.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.bb.chrsz.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.bb

# SYS command. line 35

 rm -f /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.bb.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.bb.chrsz.tmp

# SYS command. line 39

 TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0; fi
 
   
--------------------Stdout--------------------
39691 (process ID) old priority 0, new priority 19
Waiting for 57 seconds.

 
--------------------Stderr--------------------
pass1 - making usageList (20 chroms): 43 millis
pass2 - checking and writing primary data (118449 records, 15 fields): 921 millis

 
Num 9
ID task.atac.tar_all_logs.line_1748.id_54
Name tar_all_logs
Thread thread_Root
PID 39767
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:28:24
End 2017-05-21 18:28:24
Elapsed 00:00:00
Timeout 00:00:-1
Wall Timeout 100 days
Input files
Output files
Dependencies
 
# SYS command. line 1749

 cd /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out

# SYS command. line 1750

 find . -type f \
				-name '*.align.log' -or \
				-name '*.dot' -or \
				-name '*.svg' -or \
				-name '*.css' -or \
				-name '*.json' -or \
				-name '*.html' -or \
				-name '*.js' -or \
				-name '*.qc' -or \
				-name '*.pdf' -or \
				-name '*.png' -or \
				-name '*_qc.txt' -or \
				-name '*read_length.txt' \
				-name '*number_of_peaks.txt' \
				| xargs tar -cvf /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/mouse_test.all_quality_metrics.tar
 
   
--------------------Stdout--------------------
./report/jquery.treetable.css
./report/jquery.treetable.js
./report/jquery.treetable.theme.default.css
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.align.log
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.flagstat.qc
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.dup.qc
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.flagstat.qc
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.pbc.qc
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.25M.cc.qc
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.25M.cc.plot.pdf
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE_gcPlot.pdf
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.picardcomplexity.qc
./qc/rep1/test.png
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.inserts.hist_graph.pdf
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE_tss-enrich.png
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE_large_tss-enrich.png
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE_qc.html
./qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE_qc.txt
./qc/mouse_test_IDR_final.qc
./peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.unthresholded-peaks.txt.noalternatesummitpeaks.png
./peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.unthresholded-peaks.txt.png
./peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.narrowPeak.FRiP.qc

  
Num 10
ID task.report.peak2hammock.line_413.id_57
Name peak2hammock
Thread thread_Root
PID 39788
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:28:25
End 2017-05-21 18:28:43
Elapsed 00:00:17
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.hammock.gz
Dependencies
 
# SYS command. line 415

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 416

 zcat /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.gz | sed '/^\(chr\)/!d' | sort -k1,1V -k2,2n > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.tmp

# SYS command. line 418

 /users/dskim89/git/atac_dnase_pipelines/utils/narrowpeak.py /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.hammock

# SYS command. line 419

 rm -f /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.narrowPeak.tmp
 
   
--------------------Stdout--------------------
39793 (process ID) old priority 0, new priority 19

  
Num 11
ID task.report.peak2hammock.line_413.id_58
Name peak2hammock
Thread thread_Root
PID 39862
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:28:44
End 2017-05-21 18:28:56
Elapsed 00:00:12
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.hammock.gz
Dependencies
 
# SYS command. line 415

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 416

 zcat /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.gz | sed '/^\(chr\)/!d' | sort -k1,1V -k2,2n > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.tmp

# SYS command. line 418

 /users/dskim89/git/atac_dnase_pipelines/utils/gappedpeak.py /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.hammock

# SYS command. line 419

 rm -f /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/overlap/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.pval0.1.500K.naive_overlap.filt.filt_thru_npeak.gappedPeak.tmp
 
   
--------------------Stdout--------------------
39867 (process ID) old priority 0, new priority 19

  
Num 12
ID task.report.peak2hammock.line_413.id_59
Name peak2hammock
Thread thread_Root
PID 39937
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:28:57
End 2017-05-21 18:29:06
Elapsed 00:00:08
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/optimal_set/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/optimal_set/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.hammock.gz
Dependencies
 
# SYS command. line 415

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 416

 zcat /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/optimal_set/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.gz | sed '/^\(chr\)/!d' | sort -k1,1V -k2,2n > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/optimal_set/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.tmp

# SYS command. line 418

 /users/dskim89/git/atac_dnase_pipelines/utils/narrowpeak.py /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/optimal_set/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/optimal_set/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.hammock

# SYS command. line 419

 rm -f /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/optimal_set/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.tmp
 
   
--------------------Stdout--------------------
39942 (process ID) old priority 0, new priority 19

  
Num 13
ID task.report.peak2hammock.line_413.id_60
Name peak2hammock
Thread thread_Root
PID 40007
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:29:07
End 2017-05-21 18:29:25
Elapsed 00:00:18
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.narrowPeak.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.narrowPeak.hammock.gz
Dependencies
 
# SYS command. line 415

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 416

 zcat /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.narrowPeak.gz | sed '/^\(chr\)/!d' | sort -k1,1V -k2,2n > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.narrowPeak.tmp

# SYS command. line 418

 /users/dskim89/git/atac_dnase_pipelines/utils/narrowpeak.py /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.narrowPeak.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.narrowPeak.hammock

# SYS command. line 419

 rm -f /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.tn5.pf.narrowPeak.tmp
 
   
--------------------Stdout--------------------
40012 (process ID) old priority 0, new priority 19

  
Num 14
ID task.report.peak2hammock.line_413.id_61
Name peak2hammock
Thread thread_Root
PID 40085
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:29:26
End 2017-05-21 18:29:36
Elapsed 00:00:09
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.gz
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.hammock.gz
Dependencies
 
# SYS command. line 415

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 416

 zcat /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.gz | sed '/^\(chr\)/!d' | sort -k1,1V -k2,2n > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.tmp

# SYS command. line 418

 /users/dskim89/git/atac_dnase_pipelines/utils/narrowpeak.py /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.tmp /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.hammock

# SYS command. line 419

 rm -f /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/peak/macs2/idr/pseudo_reps/rep1/mouse_test_rep1-pr.IDR0.1.filt.12-col.bed.tmp
 
   
--------------------Stdout--------------------
40090 (process ID) old priority 0, new priority 19

  
Num 15
ID task.graphviz.report.line_98.id_62
Name report
Thread thread_Root
PID 40229
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:29:37
End 2017-05-21 18:29:46
Elapsed 00:00:09
Timeout 00:00:-1
Wall Timeout 100 days
Input files
Output files
Dependencies
 
# SYS command. line 99

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 100

 dot -Tsvg /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/report/mouse_test_workflow.dot > /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/report/mouse_test_workflow.svg 2> /dev/null || echo "svg: /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/report/mouse_test_workflow.svg" # to suppress dot font error (exit code=1)
 
   
--------------------Stdout--------------------
40234 (process ID) old priority 0, new priority 19

  
Num 16
ID task.report.pdf2png.line_380.id_79
Name pdf2png
Thread thread_Root
PID 40244
OK true
Exit Code 0
Retries
State FINISHED
Dep. OK
Cpus
Mem
Start 2017-05-21 18:29:37
End 2017-05-21 18:29:46
Elapsed 00:00:09
Timeout 00:00:-1
Wall Timeout 100 days
Input files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.25M.cc.plot.pdf
Output files /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.25M.cc.plot.png
Dependencies
 
# SYS command. line 381

 if [[ -f $(which /software/miniconda3/bin/conda) && $(/software/miniconda3/bin/conda env list | grep bds_atac | wc -l) != "0" ]]; then source /software/miniconda3/bin/activate bds_atac; sleep 5; fi;  export PATH=/users/dskim89/git/atac_dnase_pipelines/.:/users/dskim89/git/atac_dnase_pipelines/modules:/users/dskim89/git/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s); if (( $(nice)<19 )); then renice -n 19 $$; fi

# SYS command. line 382

 gs -dFirstPage=1 -dLastPage=1 -dTextAlphaBits=4 -dGraphicsAlphaBits=4 -r110x110 \
			-dUseCropBox -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -sDEVICE=png16m \
			-sOutputFile=/mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.25M.cc.plot.png \
			-r144 /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse/out/qc/rep1/mouse_1Mreads_rep1.1.trim.PE2SE.nodup.25M.cc.plot.pdf
 
   
--------------------Stdout--------------------
40249 (process ID) old priority 0, new priority 19

  

Global scope

Note: Global scope when program finished execution.
Type Name Value
string _ /usr/bin/bds
string _JAVA_OPTIONS -Xms256M -Xmx1024M -XX:ParallelGCThreads=1
bool allowEmpty false
string[] args [-species, mm9, -fastq1_1, mouse_1Mreads_rep1.1.fastq.gz, -fastq1_2, mouse_1Mreads_rep1.2.fastq.gz, -out_dir, out, -title, mouse_test]
string BASH_FUNC_module%% () { eval `/software/env_module/$MODULE_VERSION/bin/modulecmd sh $*`\n}
bool canFail false
int cpus -1
int cpusLocal 56
string CUDA_VISIBLE_DEVICES 2
int day 86400
string DERBY_HOME /usr/lib/jvm/java-8-oracle/db
real E 2.718281828459045
int G 1073741824
string HOME /users/dskim89/
int hour 3600
string J2REDIR /usr/lib/jvm/java-8-oracle/jre
string J2SDKDIR /usr/lib/jvm/java-8-oracle
string JAVA_HOME /usr/lib/jvm/java-8-oracle
int K 1024
string KRB5CCNAME FILE:/tmp/krb5cc_1060_ssstZu
string LANG en_US.UTF-8
string LD_LIBRARY_PATH /usr/local/cuda-8.0/lib64:
string LOADEDMODULES
string LOGNAME dskim89
int M 1048576
string MAIL /var/mail/dskim89
string MALLOC_ARENA_MAX 4
string MAX_JAVA_MEM 8G
int mem -1
int minute 60
string MODULE_VERSION 3.2.10
string MODULE_VERSION_STACK 3.2.10
string MODULEPATH /usr/local/Modules/versions\t\t\t\t:/usr/local/Modules/$MODULE_VERSION/modulefiles\t:/modules/\t\t\t\t:/software/modulefiles
string MODULESHOME /software/env_module/3.2.10
string NLSPATH /usr/dt/lib/nls/msg/%L/%N.cat
string node
int P 1125899906842624
string PATH /usr/local/cuda-8.0/bin:/usr/local/cuda/bin:/mnt/lab_data/kundaje/users/dskim89/ggr/src:/usr/local/cuda-8.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/software/miniconda3/bin:/snap/bin:/software/miniconda3/bin:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin:/users/leepc12/.bds:/software/miniconda3/bin:/users/dskim89//.bds
real PI 3.141592653589793
string ppwd /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse
string programName atac.bds
string programPath /users/dskim89/git/atac_dnase_pipelines/atac.bds
string PWD /mnt/lab_data/kundaje/users/dskim89/ataqc/data/test_data/mouse
string PYTHONPATH /mnt/lab_data/kundaje/users/dskim89/ggr/src:
string queue
int retry 0
string SHELL /bin/bash
string SHLVL 2
string SSH_CLIENT 171.66.30.161 64736 22
string SSH_CONNECTION 171.66.30.161 64736 171.65.76.90 22
string SSH_TTY /dev/pts/9
string STY 34562.pts-9.indra
string system sge
int T 1099511627776
string TERM screen.xterm-256color
string TERMCAP SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual terminal:\\\n\t:DO=\\E[%dB:LE=\\E[%dD:RI=\\E[%dC:UP=\\E[%dA:bs:bt=\\E[Z:\\\n\t:cd=\\E[J:ce=\\E[K:cl=\\E[H\\E[J:cm=\\E[%i%d;%dH:ct=\\E[3g:\\\n\t:do=^J:nd=\\E[C:pt:rc=\\E8:rs=\\Ec:sc=\\E7:st=\\EH:up=\\EM:\\\n\t:le=^H:bl=^G:cr=^M:it#8:ho=\\E[H:nw=\\EE:ta=^I:is=\\E)0:\\\n\t:li#55:co#229:am:xn:xv:LP:sr=\\EM:al=\\E[L:AL=\\E[%dL:\\\n\t:cs=\\E[%i%d;%dr:dl=\\E[M:DL=\\E[%dM:dc=\\E[P:DC=\\E[%dP:\\\n\t:im=\\E[4h:ei=\\E[4l:mi:IC=\\E[%d@:ks=\\E[?1h\\E=:\\\n\t:ke=\\E[?1l\\E>:vi=\\E[?25l:ve=\\E[34h\\E[?25h:vs=\\E[34l:\\\n\t:ti=\\E[?1049h:te=\\E[?1049l:us=\\E[4m:ue=\\E[24m:so=\\E[3m:\\\n\t:se=\\E[23m:mb=\\E[5m:md=\\E[1m:mh=\\E[2m:mr=\\E[7m:\\\n\t:me=\\E[m:ms:\\\n\t:Co#8:pa#64:AF=\\E[3%dm:AB=\\E[4%dm:op=\\E[39;49m:AX:\\\n\t:vb=\\Eg:G0:as=\\E(0:ae=\\E(B:\\\n\t:ac=\\140\\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:\\\n\t:po=\\E[5i:pf=\\E[4i:Km=\\E[M:k0=\\E[10~:k1=\\EOP:k2=\\EOQ:\\\n\t:k3=\\EOR:k4=\\EOS:k5=\\E[15~:k6=\\E[17~:k7=\\E[18~:\\\n\t:k8=\\E[19~:k9=\\E[20~:k;=\\E[21~:F1=\\E[23~:F2=\\E[24~:\\\n\t:F3=\\E[1;2P:F4=\\E[1;2Q:F5=\\E[1;2R:F6=\\E[1;2S:\\\n\t:F7=\\E[15;2~:F8=\\E[17;2~:F9=\\E[18;2~:FA=\\E[19;2~:kb=:\\\n\t:K2=\\EOE:kB=\\E[Z:kF=\\E[1;2B:kR=\\E[1;2A:*4=\\E[3;2~:\\\n\t:*7=\\E[1;2F:#2=\\E[1;2H:#3=\\E[2;2~:#4=\\E[1;2D:%c=\\E[6;2~:\\\n\t:%e=\\E[5;2~:%i=\\E[1;2C:kh=\\E[1~:@1=\\E[1~:kH=\\E[4~:\\\n\t:@7=\\E[4~:kN=\\E[6~:kP=\\E[5~:kI=\\E[2~:kD=\\E[3~:ku=\\EOA:\\\n\t:kd=\\EOB:kr=\\EOC:kl=\\EOD:km:
int timeout -1
string USER dskim89
int walltimeout 8640000
int week 604800
string WINDOW 0
string XDG_RUNTIME_DIR /run/user/1060
string XDG_SESSION_ID 3129
string XFILESEARCHPATH /usr/dt/app-defaults/%L/Dt