Every HQueue job is defined by a specification – a JSON structure containing the job’s properties. Here is a simple example: The specification defines a job with exactly one property – the job name. This is a valid specification because only the To execute tasks on the client, a set of command can be added to the specification using the { "name": "Print Hello World", "command" : "echo 'Hello World!'", } When the job is assigned to a client machine, it outputs “Hello World!” using the default shell installed with the machine’s operating system. To execute the command with a particular shell, add the { "name": "Print Hello World", "command" : "echo 'Hello World!'", "shell" : "bash", } Several commands can be stored in a job and executed in sequence by combining them as a single command. For example: { "name": "Multiple Print Commands", "command" : "echo 'The First Command' && echo 'The Second Command'", } How commands are combined depend on the shell but separating commands with For complex commands, it is easier to store the commands in a script file and then point the { "name": "Running a Script", "command" : "/path/to/myScript.sh", } For a complete list of the job properties that can be defined in the specification, see Job Properties. Status ChangesAn HQueue job passes through a sequence of status changes from when it is submitted to HQueue to when it completes. A basic job like the example previously mentioned undergoes this sequence: waiting for machine → running → succeededWhen a job is submitted to HQueue, it is placed into the scheduling queue where it waits for a client machine. Once a machine becomes available and is assigned, then the machine runs the job’s commands. When execution of the job’s commands finishes without errors, the job is marked as succeeded. For a complete list of job statuses, see Job Statuses. Parent-Child RelationshipsA dependency can be set between two jobs so that the first job cannot run until the second job completes. This dependency is called a parent-child relationship in HQueue. For example, if job A depends on job B, then job A is a parent of job B and job B is a child of job A. A Simple Parent-Child ExampleIf you want to create an AVI video from X frames rendered from a Houdini scene. Assuming that IFD files have already been generated for the frames, then you can perform it by doing the following:
For the first step, you can define X HQueue jobs, one job for every frame to be rendered. The job specification for rendering frame 1 looks like: { "name": "Render Frame 1", "command": """cd $HQROOT/houdini_distros/hfs; source houdini_setup; mantra < $HQROOT/path/to/ifds/frame0001.ifd""" "shell": "bash", } The job specifications for the rest of the frames is similar. Now suppose that Mantra renders the images to { "name": "Encode Video", "shell": "bash", "command": "myEncoderApp --input=$HQROOT/path/to/output/frame*.png --output=$HQROOT/path/to/output/myVideo.avi" } To ensure that the encoding job is executed after all the render jobs have completed, you create a dependency by making the encoding job a parent of the render jobs. To create the dependency, you use the So the final specification for the encoding job would look like: { "name": "Encode Video", "shell": "bash", "command": "myEncoderApp --input=$HQROOT/path/to/output/frame*.png --output=$HQROOT/path/to/output/myVideo.avi", "children": [ { "name": "Render Frame 1", "shell": "bash", "command": """cd $HQROOT/houdini_distros/hfs; source houdini_setup; mantra < $HQROOT/path/to/ifds/frame0001.ifd""" }, "name": "Render Frame 2", "shell": "bash", "command": """cd $HQROOT/houdini_distros/hfs; source houdini_setup; mantra < $HQROOT/path/to/ifds/frame0002.ifd""" }, "name": "Render Frame 3", "shell": "bash", "command": """cd $HQROOT/houdini_distros/hfs; source houdini_setup; mantra < $HQROOT/path/to/ifds/frame0003.ifd""" }, ............................. ] } Parent StatusThe parent job’s status depends on the statuses of its child jobs. When the child jobs complete, if at least one child job has failed, then the parent job’s status is set to failed and the parent job’s commands are not executed. Otherwise, if all the child jobs complete successfully, then the parent job’s status is changed to waiting for machine and the parent job is placed into the scheduling queue where it waits for a client machine. Submitting Child Jobs from Within The ParentIt is possible to submit child jobs from within a running job. You may want this if the number of child jobs needed is not known until runtime. In such cases, commands can be added to the parent job that calculate how many child jobs are required and then submits child job specifications to the HQueue server. To submit child jobs, use the Here is an example of a Python script that creates a new job and assigns it as a child to the currently running job: import os import xmlrpclib # Connect to the HQueue server. hq_server = xmlrpclib.ServerProxy("http://hq_server_hostname:5000") # Define the child job. child_job_spec = { "name": "The Child Job", "shell": "bash", "command": "echo 'Hello World!'" } # Get the id of the current job. # It is defined automatically by HQueue in the JOBID environment variable. current_job_id = os.environ["JOBID"] # Submit the job to the server. # newjob() returns a list of job ids (in case multiple jobs are passed in at once). job_ids = hq_server.newjob(child_job_spec, current_job_id) If the script is saved into a file, say createChild.py, then it can be added to the { "name": "The Parent Job", "shell": "bash", "command": "python $HQROOT/path/to/scripts/createChild.py" } Commandless JobsIt is possible to have a job without the Commandless jobs do not perform any work, but they can be useful at times. For example, if you write a Python script that submits jobs to HQueue, you can use commandless jobs to test calls to Job PropertiesHere is a list of properties that can be added to a job specification.
Job Properties ExampleHere is an example of a job specification that demonstrates the use of some properties: { "name": "The Main Job", "shell": "bash", "environment": { "SHOW_MSG": "1", "MSG": "Hello World!" }, "command": """if [ $SHOW_MSG = 1 ]; then echo $MSG; fi""", "tags": [ "single" ], "maxHosts": 1, "minHosts": 1, "priority": 0, "children": [ { "name": "The Child Job", "shell": "bash", "command": "echo 'Hello World!'" } ] } The example above defines a job named 'The Main Job' which has a priority level of 0. It uses the bash shell to execute its command set and defines two environment variables, Job ConditionsJob conditions inform the HQueue scheduler to assign a job to a restricted set of client machines. A job condition is defined by a type, name, operator and value. Together they specify a comparison test that the scheduler uses to determine whether a machine can be assigned to run the job. If a client machine passes ALL of the assigned conditions, then it can run the job. Note If a job does not define its own set of conditions then it automatically inherits its root job’s conditions. Below is a description of each of the condition components:
Job Condition ExamplesHere is an example that demonstrates how to attach a condition to a job specification: { "name": "A Job with Conditions", "shell": "bash", "command": "echo 'I should be running on either machine1 or machine2!'", "conditions": [ { "type" : "client", "name": "hostname", "op": "in", "value": "machine1,machine2" }, ] } The example above defines a job which can only be assigned to a client machine named either “machine1” or “machine2”. Note that the The next example demonstrates how to set a condition where the job can only be assigned to client machines that are members of the “Simulation” group: { "name": "A Job for the Simulation Group", "shell": "bash", "command": "echo 'I should be running on a machine that is a member of the Simulation group!'", "conditions": [ { "type" : "client", "name": "group", "op": "==", "value": "Simulation" }, ] } Tags can be used to describe whether a job requires a dedicated machine or whether it can share a machine with other running jobs. If no tags are specified, then the job is configured to share the machine it is running on as long as the machine has enough CPUs. To declare that a job needs a dedicated machine, add the You can also create custom single tags to control the sets of running jobs that can share machines and the sets that cannot. No two jobs with the same custom single tag will run on the same machine simultaneously. To create a custom single tag, simply prefix the tag name with
For example, suppose you create two jobs, A and B, and assign them a custom single tag named Job StatusesHere is a list of job statuses and their descriptions.
Job VariablesHere is a list of the built-in, runtime variables that are defined in the job’s environment.
Which pair of commands can be used to display the PID of a process?To display a list of running processes use the command ps and this will show you running processes with their PID number.
Which of the following commands will show a list of process names along with their process ID PID?You need to use the ps command. It provides information about the currently running processes, including their process identification numbers (PIDs). Both Linux and UNIX support the ps command to display information about all running process. The ps command gives a snapshot of the current processes.
Which of the following commands can be used to display the filesystem and partition Uuids on a Linux system?You can find the UUID of all the disk partitions on your Linux system with the blkid command. The blkid command is available by default on most modern Linux distributions. As you can see, the filesystems that has UUID are displayed.
What Metacharacter can be used to?A metacharacter is a character that has a special meaning during pattern processing. You use metacharacters in regular expressions to define the search criteria and any text manipulations. Search string metacharacters are different from replacement string metacharacters.
|