In this chapter: Introduction to Running Native Programs The Command-Line Format Return Value, Running Modes and Options » Return Value » Synchronous and Asynchronous Runs » Environment Variables » Start-Up Directory Process Executable Input And/Or Output » User Input for the Executable » Provide User Input from the Script » Read Executable Output into the Script » Write to and Read from the Executable Case Study: Build and Test the Judo Software

Book: The Judo Language 0.9 Chapter 16. Run Executables By James Jianbo Huang non-printer version

Synopsis: Judo provides an exec facility that executes OS native programs in ways that are similar to OS shells and beyond. For instance, you can easily write code to feed input into and read in the output of the running executable at the same time. You can specify the starting directory and environment variables for the new process, and exec supports a sophisticated command format that allows you to pipe multiple executables together and redirect input and output between files.

 
Introduction to Running Native Programs

The exec command provides a flexible way of executing native operating system programs. In a nutshell, you can specify shell-like command lines including pipes and redirections, and set environment variables and start directory; you can pump input to that executable and receive output out of that executable as well. You can receive the return value. By default the executable is run synchronously, but it can be run asynchronously. The complete syntax is:

This looks like a lot of knobs that you can turn. The core elements of running any executables are:

1. the command line
2. the start-up directory, and
3. environment variables for the launched process.

In most cases, these are all you need. The following are some examples that do not need a lot of explanations.

exec 'ls > files.txt';

exec 'ls > ~/tempfiles.txt' from '/temp';

exec && 'java foo.bar.FooBarClass' with CLASSPATH=${mycp};

The && option is to run asynchronously. We will cover all the options and uses of this versatile command later. Before getting into details, beware that Judo ultimately uses Java to run native executables, which has limitations. The JDK documentation for class java.lang.Runtime states:

"The Runtime.exec methods may not work well for special processes on certain native platforms, such as native windowing processes, daemon processes, Win16/DOS processes on Win32, or shell scripts."

Thus far, I have successfully run Unix and Win32 commands, shell scripts and batch files. Nevertheless, always test out the scripts on the intended platforms with the native executables. In this chapter, we will mostly use Unix and Windows commands and the following three Java programs to demonstrate various uses.

Listing 16.1 Hello.java
package test; public class Hello { public static void main(String[] args) { System.out.println("Hello, World!"); if (args.length > 0) System.err.println("Hello, Error!"); System.exit(123); } }

Listing 16.2 Echo.java
package test; public class Echo { public static void main(String[] args) { try { int len; byte[] buf = new byte[256]; while ((len = System.in.read(buf)) > 0) { System.out.write(buf,0,len); } } catch(Exception e) { e.printStackTrace(); } } }

Listing 16.3 ToLower.java
package test; import java.io.*; public class ToLower { public static void main(String[] args) { try { BufferedReader br = new BufferedReader(new InputStreamReader(System.in)); while (true) { String line = br.readLine(); if (line == null) break; System.out.println(line.toLowerCase()); } } catch(Exception e) { e.printStackTrace(); } } }

Let us run test.Hello first.

exec <a> 'java test.Hello';
println a; // prints 123

Here, variable a receives the return code from the program. Since class test.Hello can print to standard output or standard error, let us see how to receive them.

setOut "stdout.txt";      // set standard output to this file
setErr "stderr.txt";      // set standard error to this file
exec 'java test.Hello';   // to stdout
exec 'java test.Hello a'; // to stderr

After running, file stdout.txt contains:

Hello, World!
Hello, World!

and file stderr.txt contains:

Hello, Error!

In the rest of this chapter, we are going to detail on the command-line format, start-up options, return value, execution synchronization, and input/output.

 
The Command-Line Format

The Expr in the syntax rules above is evaluated to a string value representing a command line, and is parsed by Judo to launch executables. The command line is usually a string literal, and it can be multi-line to achieve better readability. For instance,

exec [[* ${j13home}/bin/javac -d ${tmp}/classes
                  ${generated}/*.java
                  ${src}/*judo*.java
                  ${src}/com/judoscript/*.java
                  ${src}/com/judoscript/parser/helper/*.java
                  ${src}/com/judoscript/bio/*.java
                  ${src}/com/judoscript/db/*.java
                  ${src}/com/judoscript/xml/*.java
                  ${src}/com/judoscript/gui/*.java
                  ${src}/com/judoscript/studio/*.java
                  ${src}/com/judoscript/jusp/*.java
                  ${src}/com/judoscript/ext/FactoryUtil.java
     *]]
     with PATH      = j13path,
          JAVA_HOME = j13home,
          CLASSPATH = '${cp}${:}${lib}/j2ee1.3.jar${:}${lib}/bsf-2.3.jar'
;

Judo supports a shell-like, sophisticated command line format, defined in BNF as:

Redirecting and piping are familiar to most computer users. On most operating systems, a program can take input from the system input stream and write to system output and error streams; they are commonly referred to as stdin, stdout and stderr, following the tradition of Unix and the C programming language. In many OS shells, you can redirect stdin, stdout and/or stderr to files; you can also pipe the output of a program into the input of another program. In many Unix shells, you can optionally combine stdout and stderr and send the content into a single file.

The Judo's command-line format is even more flexible: you can pipe up programs, and each program can optionally write output to files at the same time. Only the first program's input can be redirected to a file, as subsequent programs are taking the previous program's output as input. In most Unix shells, only the last program in a pipe can write to a file. In Judo, the last program can redirect output and error into separate files.

When 1>&2, 2>&1, 1>>&2 or 2>>&1 occurs, that should be the only output redirection.

You can redirect the input from a file, and sent the output to a file:

exec 'java test.Echo < lines.txt > alfa';

You can append to a file, redirect output or error to one file or merge them. The following session is recorded on a Windows machine, and we run the Judo software with the -x option which runs the Judo program from the command-line:

c:\>java judo -x "exec 'java test.Hello a 2>&1'"
Hello, World!
Hello, Error!

c:\>java judo -x "exec 'java test.Hello a 1> alfa 2> beta'"

c:\>type alfa
Hello, World!

c:\>type beta
Hello, Error!

c:\>java judo -x "exec 'java test.Hello a 2>&1 alfa'"

c:\>type alfa
Hello, World!
Hello, Error!

c:\>java judo -x "exec 'java test.Hello a 2>>&1 alfa'"

c:\>type alfa
Hello, World!
Hello, Error!
Hello, World!
Hello, Error!

c:\>

The following session tries out piping:

c:\>java judo -x "exec 'java test.Hello a 2>&1 | java test.ToLower | java test.Echo > alfa'"

c:\>type alfa
hello, world!
hello, error!

c:\>

You can send the output to a file and at the same time pipe into another program. With the help of the test.Echo, we can create a tee utility:

Listing 16.4 tee.judo
outfile = #args[0].neverEmpty('output.dump'); exec 'java test.Echo > ${outfile} | java test.Echo' needSystemIn;

The needSystemIn indicates this program needs user input. Here is the result:

c:\>echo ABCDEFG | java judo tee.judo alfa
ABCDEFG

c:\>type alfa
ABCDEFG

 
Return Value, Running Modes and Options

Native executables may return a value, which is typically a string representing an integer. You can specify a variable name in < > following exec to receive this return value. To get the return value, Judo runs the executable synchronously and waits for it to finish.

Programs can be run either synchronously, by default, or asynchronously. To run asynchronously, specify && following the exec keyword. The following program shows the difference.

Listing 16.5 sync_async.judo
exec 'dir'; println "--- I am done with: exec command."; exec && 'dir'; println "--- I am done with: exec && command.";

Its output is:

exit_code.judo  out_err.judo     sync_async.judo
in2script.judo  redir.judo       tee.judo
in_out.judo     script_out.judo  workdir_envvar.judo
--- I am done with: exec command.
--- I am done with: exec && command.
exit_code.judo  out_err.judo     sync_async.judo
in2script.judo  redir.judo       tee.judo
in_out.judo     script_out.judo  workdir_envvar.judo

The second exec is run asynchronously, therefore the following println is run immediately after exec, and the Windows dir command finishes afterwards. In contrast, the first exec waits till the dir commands completes and moves on to the next println statement.

By default, exec command launches the child process with all the environment variables of the current JVM process. You can set new environment variables or change values of existing ones using the with clause. Optionally, you can choose not to inherit any parent environment variables with the <new> option.

The with clause usually takes name-value pairs for environment variable values. It can also take an expression that evaluates either to an Object or a Java instance of java.util.Map, whose elements are used. The following example demonstrates these choices.

Listing 16.6 envvars.judo
cmdline = 'java judo -x "for x in getEnvVars() { println x; }"'; println nl, '--- Run with inherited env variables from parent JVM process:'; exec cmdline; println nl, '--- Run with no env variables except for PATH and CLASSPATH:'; exec cmdline with <new> PATH = ${PATH}, CLASSPATH = ${CLASSPATH} ; println nl, '--- Run with a new env variable:'; exec cmdline with CLASSPATH = 'c:/temp'; println nl, '--- Run with a set of new env variables:'; newEVs = { // an Object FOO = 'foo', BAR = 'bar' }; exec cmdline with newEVs;

This script runs an external program specified in the variable cmdline, which happens to be an in-line execution of a short Judo program. The third exec in this program fails, because the CLASSPATH is not valid, so Judo software is not found anywhere.

Sometimes, it matters where to start the program. For instance, some shell scripts may have used relative directories to access other resources and hence must start from a particular directory structure. The exec command uses the from clause to specify this.

exec 'build.sh' from ${BUILD_HOME};

The following example creates a batch file on a Windows machine and runs it from where it resides.

Listing 16.7 run_batch.judo
testDir = 'c:/temp'; bat_file = '${testDir}${/}alfa.bat'; f = openTextFile(bat_file, 'w'); flush <f> [[* @echo 'This is some garbage.' > alfa.alfa @echo ABCDEFG=%ABCDEFG% @echo PATH=%PATH% *]]; f.close(); exec bat_file from testDir with ABCDEFG = 1234567, PATH = testDir;

So far, we have covered all the basic topics regarding running executables in Judo. Next is a more exciting topic of manipulating the input to and the output from a running executable.

 
Process Executable Input And/Or Output

Executables may take user input and/or write out results. The exec's needSystemIn, input and output clauses provide ways to manipulate these aspects.

If a native program taks user input, things may get interesting when it is launched from within another agent, such as a shell script or Judo. It is not uncommon to see a background job hanging, only waiting to read some input that will never come. In exec, by default Judo sends an EOF (End-Of-File) to the input stream if no input redirection is present. This is to avoid the hang as mentioned above. To allow user input, you need to explicitly so specify via the needSystemIn:

C:\>java judo -x "exec 'java test.ToLower' needSystemIn"
abc
abc
ABC
abc
^Z

C:\>

Judo allows you to provide the user input from within the script. This is achieved by the input block, where you can print to a predefined output destiny, <pipe>. Here is an example:

Listing 16.8 in2script.judo
exec 'java test.ToLower | java test.Echo' input { println <pipe> 'You should see this message in lower case, not UPPER CASE!'; println <pipe> 'Cheers!'; }

Read Executable Output into the Script

Judo also allows you to read in the output from the program:

Listing 16.9 script_out.judo
exec 'java test.ToLower' needSystemIn output { while (line=readPipe()) != eof { println ' #', loopIndex()+1, ': ', line; } }

Write to and Read from the Executable

You can write into and read from a command line at the same time.

Listing 16.10 in_out.judo
exec 'java test.ToLower | java test.Echo' input { println <pipe> 'Hello,'; println <pipe> 'World!'; } output { while (line=readPipe()) != eof { println ' #', loopIndex()+1, ': ', line; } }

The output block is actually run in a separate thread, so your code need to follow thread programming guideline. For details of thread programming, see 8. Threads and Dynamic Evaluation.

 
Case Study: Build and Test the Judo Software   _{to be done}