Condor allows users to access a wide variety of machines distributed around the world. The Java Virtual Machine (JVM) provides a uniform platform on any machine, regardless of the machine's architecture or operating system. The Condor Java universe brings together these two features to create a distributed, homogeneous computing environment.
Compiled Java programs can be submitted to Condor, and Condor can execute the programs on any machine in the pool that will run the Java Virtual Machine.
The condor_ status command can be used to see a list of machines in the pool for which Condor can use the Java Virtual Machine.
% condor_status -java Name JavaVendor Ver State Activity LoadAv Mem ActvtyTime coral.cs.wisc Sun Microsy 1.2.2 Unclaimed Idle 0.000 511 0+02:28:04 doc.cs.wisc.e Sun Microsy 1.2.2 Unclaimed Idle 0.000 511 0+01:05:04 dsonokwa.cs.w Sun Microsy 1.2.2 Unclaimed Idle 0.000 511 0+01:05:04 ...
If there is no output from the condor_ status command, then Condor does not know the location details of the Java Virtual Machine on machines in the pool, or no machines have Java correctly installed. In this case, contact your system administrator or see section 3.14 for more information on getting Condor to work together with Java.
Here is a complete, if simple, example. Start with a simple Java program, Hello.java:
public class Hello {
public static void main( String [] args ) {
System.out.println("Hello, world!\n");
}
}
Build this program using your Java compiler. On most platforms, this is accomplished with the command
javac Hello.java
Submission to Condor requires a submit description file. If submitting where files are accessible using a shared file system, this simple submit description file works:
#################### # # Example 1 # Execute a single Java class # #################### universe = java executable = Hello.class arguments = Hello output = Hello.output error = Hello.error queue
The Java universe must be explicitly selected.
The main class of the program is given in the executable statement. This is a file name which contains the entry point of the program. The name of the main class (not a file name) must be specified as the first argument to the program.
If submitting the job where a shared file system is not accessible, the submit description file becomes:
#################### # # Example 1 # Execute a single Java class, # not on a shared file system # #################### universe = java executable = Hello.class arguments = Hello output = Hello.output error = Hello.error should_transfer_files = YES when_to_transfer_output = ON_EXIT queueFor more information about using Condor's file transfer mechanisms, see section 2.5.4.
To submit the job, where the submit description file is named Hello.cmd, execute
condor_submit Hello.cmd
To monitor the job, the commands condor_ q and condor_ rm are used as with all jobs.
For programs that consist of more than one .class file, one option that tells Condor of the additional files adds this line to the submit description file:
transfer_input_files = Larry.class,Curly.class,Moe.class
If the program consists of a large number of class files, it may be easier to collect them all together into a single Java Archive (JAR) file. A JAR can be created with:
% jar cvf Library.jar Larry.class Curly.class Moe.class
Condor must then be told where to find the JAR by adding the following to the submit description file:
jar_files = Library.jar
Note that the JVM must know whether it is receiving JAR files
or class files.
Therefore, Condor must also be informed, in order to pass the
information on to the JVM.
That is why there is a difference in submit description file commands
for the two ways of specifying files (transfer_input_files
and jar_files).
If the program uses Java features found only in certain JVMs, then inform Condor by adding a requirements statement to the submit description file. For example, to require version 3.2, add to the submit description file:
requirements = (JavaVersion=="3.2")
Each machine with Java capability in a Condor pool will execute a benchmark to determine its speed. The benchmark is taken when Condor is started on the machine, and it uses the SciMark2 (http://math.nist.gov/scimark2) benchmark. The result of the benchmark is held as an attribute within the machine ClassAd. The attribute is called JavaMFlops. Jobs that are run under the Java universe (as all other Condor jobs) may prefer or require a machine of a specific speed by setting rank or requirements in the submit description file. As an example, to execute only on machines of a minimum speed:
requirements = (JavaMFlops>4.5)
Options to the Java VM itself can be set in the submit description file:
java_vm_args = -DMyProperty=Value -verbose:gc
These options are those which go after the java command, but before the user's main class. Do not use this to set the classpath, as Condor handles that itself. Setting these options is useful for setting system properties, system assertions and debugging certain kinds of problems.
If a job has more sophisticated I/O requirements that cannot be met by Condor's file transfer mechanism, then the Chirp facility may provide a solution. Chirp has two advantages over simple, whole-file transfers. First, it permits the input files to be decided upon at run-time rather than submit time, and second, it permits partial-file I/O with results than can be seen as the program executes. However, small changes to the program are required in order to take advantage of Chirp. Depending on the style of the program, use either Chirp I/O streams or UNIX-like I/O functions.
Chirp I/O streams are the easiest way to get started. Modify the program to use the objects ChirpInputStream and ChirpOutputStream instead of FileInputStream and FileOutputStream. These classes are completely documented in the Condor Software Developer's Kit (SDK). Here is a simple code example:
import java.io.*;
import edu.wisc.cs.condor.chirp.*;
public class TestChirp {
public static void main( String args[] ) {
try {
BufferedReader in = new BufferedReader(
new InputStreamReader(
new ChirpInputStream("input")));
PrintWriter out = new PrintWriter(
new OutputStreamWriter(
new ChirpOutputStream("output")));
while(true) {
String line = in.readLine();
if(line==null) break;
out.println(line);
}
out.close();
} catch( IOException e ) {
System.out.println(e);
}
}
}
To perform UNIX-like I/O with Chirp, create a ChirpClient object. This object supports familiar operations such as open, read, write, and close. Exhaustive detail of the methods may be found in the Condor SDK, but here is a brief example:
import java.io.*;
import edu.wisc.cs.condor.chirp.*;
public class TestChirp {
public static void main( String args[] ) {
try {
ChirpClient client = new ChirpClient();
String message = "Hello, world!\n";
byte [] buffer = message.getBytes();
// Note that we should check that actual==length.
// However, skip it for clarity.
int fd = client.open("output","wct",0777);
int actual = client.write(fd,buffer,0,buffer.length);
client.close(fd);
client.rename("output","output.new");
client.unlink("output.new");
} catch( IOException e ) {
System.out.println(e);
}
}
}
Regardless of which I/O style, the Chirp library must be specified and included with the job. The Chirp JAR (Chirp.jar) is found in the lib directory of the Condor installation. Copy it into your working directory in order to compile the program after modification to use Chirp I/O.
% condor_config_val LIB /usr/local/condor/lib % cp /usr/local/condor/lib/Chirp.jar .
Rebuild the program with the Chirp JAR file in the class path.
% javac -classpath Chirp.jar:. TestChirp.java
The Chirp JAR file must be specified in the submit description file. Here is an example submit description file that works for both of the given test programs:
universe = java executable = TestChirp.class arguments = TestChirp jar_files = Chirp.jar queue