Condor Examples

Sequential programs

The full code of this example is in this public Git repository at GitHub

Consider a simple program which generates just one output file. For simplicity we assume that the input parameters are changed inside the program by generating a random number for example.

Let the a working directory called, let say
benchmark
with all the required files and programs.

Creates the condor directory in gfif
```
$ mkdir condor_example
```
Copy the working directory there and tar and gzip this:
```
$ tar -zcvf benchmark.tar.gz benchmark
```

Creates a bash script to tell condor what to do with the compressed directory. The IMPORTANT point is that this script must have an argument which allows to differentiate the output file of each sequential run

run.sh

#!/bin/bash
Process=$1
tar -xzvf benchmark.tar.gz
cd benchmark/
#run either one system program,
#or one script inside benchmark directory which
#take at least the argument $Process to differentiate 
#the output program:
./main_program.py $Process
#The main program produces an output Output$Process.out
# which must be declared inside the condor submit file: 
# job.submit  as:
#transfer_output_files =  benchmark/Output$(Process).spc
# See job.submit for details

#Finally, sleep for while to be sure that a new random seed
# is generated
sleep 1

The next step is to prepare the condor submit file. For sequantial programming the vainilla condor universe is used. This means that condor will create an independent directory for each queue. Once each queue is finished the output file with their corresponding Process identification is transfered back to the master node:
job.submit
```
Universe        = vanilla
Executable      = run.sh
Output          = tmp/run.out
Error           = tmp/run.error
Log             = tmp/run.log
Arguments       = $(Process)
#requirements = SlotID == 6 (Machine == "particle.udea.edu.co")
#requirements = (Machine != "particle.udea.edu.co")
ShouldTransferFiles = Yes
WhenToTransferOutput = ON_EXIT
transfer_input_files = benchmark.tar.gz
transfer_output_files =  benchmark/Output$(Process).out
queue 10
```
The variable: transfer_output_files, must match the real output file name(s) of the main program in the working directory. Note that some tmp directory must be created for the log files
```
$ mkdir tmp
```
Once the full job is finished you can gather all the results with a command like:
```
$ cat Output* > FullOutput.out
```
Send your job with
```
$ condor_submit job.submit
```
Check the status of your job with
```
$ condor_q
```
and see the logs in the tmp directory if necessary.
If you need kill some queue with some ID, use
```
$ condor_rm ID
```
To kill the full job(s) use:
```
$ condor_rm -a
```
If you change something in the working directory you should need back to the step 2

Similar

Search Help

Non Advanced Search or Natural Search

The documents are returned sorted on relevance depending on order, proximity, frequency of terms.

Advanced Search or Boolean Search

Default search behavior

By default, all search terms are optional. It behaves like an OR logic. Objects that contain the more terms are rated higher in the results and will appear first in their type. For example, wiki forum will find:

objects that include both terms
objects that include the term wiki
objects that include the term forum

Requiring terms

Add a plus sign ( + ) before a term to indicate that the term must appear in results. Example: +wiki forum will find objects containing at least wiki. Objects with both terms and many occurences of the terms will appear first.

Excluding terms

Add a minus sign ( - ) before a term to indicate that the term must not appear in the results. To reduce a term's value without completely excluding it, use a tilde. Example: -wiki forum will find objects that do not contain wiki but contain forum

Grouping terms

Use parenthesis ( ) to group terms into subexpressions. Example: +wiki +(forum blog) will find objects that contain wiki and forum or that contain wiki and blog in any order.

Finding phrases

Use double quotes ( " " ) around a phrase to find terms in the exact order, exactly as typed. Example: "Alex Bell" will not find Bell Alex or Alex G. Bell.

Using wildcards

Add an asterisk ( * ) after a term to find objects that include the root word. For example, run* will find:

objects that include the term run
objects that include the term runner
objects that include the term running

Reducing a term's value

Add a tilde ( ~ ) before a term to reduce its value indicate to the ranking of the results. Objects that contain the term will appear lower than other objects (unlike the minus sign which will completely exclude a term). Example: +wiki ~forum will rate an object with only wiki higher that an object with wiki and forum.

Changing relevance value

Add a less than ( < ) or greater than ( > ) sign before a term to change the term's contribution to the overall relevance value assigned to a object. Example: +wiki +(>forum < blog) will find objects that contain wiki and forum or wiki and blog in any order. wiki forum will be rated higher.