NAME tester - a program for testing other programs SYNOPSIS tester [ -d [ *sub[,...]* ] ] [ -s ] -f *FILE* [ test ... ] tester --man | --help tester --version DESCRIPTION tester is a program that tests other programs. tester doesn't require special frameworks, programming languages, libraries, or anything like that; it is completely self-contained and uses simple commands in a text file to specify tests and desired results. It it written in Perl but can test programs written in any language, as long as those programs can be run from the command line. The name(s) of one or more tests (as defined in *FILE*, see below) may be given on the command line; if present, only these teste will be run. The default is to run all tests found in *FILE*. OPTIONS -f, --file FILE read test specifications from *FILE* -s, --stop stop after the first failure; the default is to run all tests -d, --debug [ *sub[,...]* ] ] (debug) --man --help show this documentation --version Show version information then exit. INPUT tester reads its configuration from the file specified by the -f switch (above); only one file may be given. In all specifications below, what is always one of "stdout", "stderr", or "exit"; any variant of str is a (quoted) string, and int in a positive integer. Where str1 str2 [ str3 ] is shown, the first string will show the expected value, the second will show the value from the command, and the thrid (if given), will show the status of the test (pass or fail). All three should be "printf()"-style format strings, however, *\n* and *\t* are the only escape sequences recognized. *#* the line will be treated as a comment (and ignored); blank likes are also ignored *COMPARE* what str1 str2 [ str3 ] set the output strings for *compare* directives (below) *FAIL* what str1 str2 [ str3 ] set the output strings for *fail* directives (below) *PATH* str Set $PATH to "str" (which should be a colon-separated list of paths) *QUOTES* lr set the quote characters for all input strings; "lr" must be exactly two characters; default is ""; note that lines in specification files are processed sequentially, so if alternate quotes are desired the *QUOTES* line needs to appear before the alternate quotes are used *SHOW* what str set the output strings for *show* directives (below) *STATUS* pass-str fail-str set the default "pass" and "fail" strings name { start of a test stanza; name must start with a letter, then may contain any combination of letters, digits, dash, and underscore. *command* str run the command specified by *str*, capturing stdout, stderr, and the exit value *compare* what [ str1 str2 [ str3 ] ] compare expected and received values for what; output strings (in order of precedence) are: those (optionally) specified in this directive, those specified in an *COMPARE* directive (above), or the built-n defaults *exit* int expected exit value of *command* (above) *fail* what [ str1 str2 [ str3 ] ] identical to *compare* (above) except that expected and received values are shown only if they do not match (whereas *compare* always shows the values) *quotes* lr set the quote characters for input strings in this test; other tests are not affected; see *QUOTE* above for details *report* report the results (pass/fail) of any tests (*stdout*, *stderr*, or *exit*) *show* what [ str ] show the value received from *command* (above); the precedence of output strings is the same as explained in *compare* and *fail* (above), except the global string comes from *SHOW* (also above) *status* pass-str fail-str set the "pass" and "fail" strings for just this test; other tests are not affected *stderr* str expected results of *command* (above) on stderr; may contain *\n* and *\t* but otherwise must match the results from the command exactly *stdout* str expected results of *command* (above) on stdout; *str* is identical to *stderr* (above) } Every test stanza must contain at least the following directives: name { *command* one of *stdout*, *stderr*, or *exit* } OUTPUT For each test, at least two lines will be printed: *==> name* results: o/e/x where *o*, *e*, and *x* correspond to tests *stdout*, *stderr*, and *exit*, and the results values will be one of 0 for test failed, 1 for test passed, and "-" for not tested. Directives such as *compare*, *report*, etc. will produce additional output. After all tests have run, a final line showing the name of the tests file, the number of tests passed, and the number of tests failed will be printed. EXAMPLES Basic pass/fail tests: true { command /bin/true exit 0 } false { command /bin/false exit 0 } Test C will pass and C will fail. Simple pass/fail test: echo1 { command echo this is a test stdout "this is a test\n" stderr "" exit 0 report } An example of *compare* versus *fail*: echo2 { command echo this is a test stdout "this is a test\n" compare stdout } echo3 { command echo this is a test stdout "this is a test\n" fail stdout } echo4 { command echo this is a test stdout "This is a test.\n" fail stdout } Rather than try to explain the differences here, put those three stanzas in a file and run B; the differences should be obvious. A more complete example: # Global stuff QUOTES {} COMPARE stdout {out\nexp = '%s'\n} {got = '%s'\n} {? = %s\n--\n} FAIL stderr {STDERR FAIL:\ne = '%s'\n}, {g = '%s'\n} PATH bin:/usr/bin:/home/adamm/bin SHOW exit {exit value = %d\n} STATUS {it worked!} {it didn't work :-(} # Test starts here example { # Quotes local to this test only quotes <> command my-command stdout stderr <> exit 0 compare stdout fail stderr show exit } DEBUGGING The debugging facility is mostly intended for finding and fixing problems in the code but it may prove useful for bugs in test specification files. The -d (or --debug) switch will turn on debugging output from all subroutines; for more limited output, give a comma-separated list of one or more subroutine names. For most test problems, using --debug *assertRunTest* will give the most useful output. Not all subroutines produce debugging output, and some should produce more; someday I may get around to fixing this. EXIT STATUS tester may exit with any of the following values: 0 - all tests passed 1 - one or more tests failed 2 - problems related to the command line 3 - problems with the test specifications file AUTHOR Adam Moskowitz. BUGS Please report bugs to .