testing

Tcl has long traditions in testing [put in some general-purpose testing reference here--Beizer, ...]. Especially notable are Expect, Oracle, Sybase [1], NASA [2], and other projects.

The GNU C Compiler uses Tcl scripts and Expect to do regression testing. - tomk

Please use this page to document success stories and information on how Tcl has been used and can be used to build a working test environment.

"All of tcltest, simpletest and DejaGNU have a similar goal, IMHO: that of providing a /framework/ for testing, although both tcltest and simpletest are conceived for Tcl testing while DejaGNU seems more suited towards C/C++. None of them provide a full fledged environment useful for non-programmers, though", writes JCG.

And don't forget about the tcltest package, which is a test harness for writing and running tests in the Tcl test suite or for creating a customized test harness for an extension. It includes both documentation and a huge set of examples (the Tcl and Tk test suites).

SimpleTest is a tcltest work-alike for projects using The Simple Library.

See also Stress testing.

RS: A basic testing skeleton (further developed in e.g.) might look like this:

 proc test {command expected} {
    catch {uplevel 1 $command} result ;# so we can use variables in caller's scope
    if {$result ne $expected} {error "$command->$result, expected $expected"}
 }

Instead of error, you might use puts, so the test suite doesn't stop at the first surprise. Example for testing the test:

 test {expr {2+3}} 5

Jacks is a Java compiler testing framework that was developed for open source Java compiler projects because the official Sun development kits have not been released. Jacks rivals the Sun kits in capability, and may soon even have the same number of compatibility tests.

The Dart framework produces the pretty test status dashboards that one sees for Vtk. More information about Dart is at http://public.kitware.com/Dart/HTML/Index.shtml . Some examples of Dart's output are at [3] and [4]. Dart, like Vtk, is open-source.

The CVS repository for Vtk is

   cvs -d :pserver:anonymous@public.kitware.com:/cvsroot/VTK checkout VTK

Password is "vtk".

The repository for Dart is

   cvs :pserver:anonymous@public.kitware.com:/cvsroot/Dart checkout Dart

Password is "dart"

In response to a question about testing C and Java from Tcl, Cameron Laird responded eloquently:

"Tcl is the single most successful language for test automation of the kind you're describing. That's so undeniable that I'd forgotten it deserves to be documented for new generations of developers. DejaGnu is the testing framework that's made gcc possible. Companies like Oracle and Sybase have MILLIONS of lines of Tcl testing code on which they rely to assure the operation of their 'flagship' products.

Given enough motivation, I could do a whole book on use of Tcl in testing. Tcl tests automobile engines, emergency telephone circuits, chemical sensors, microprocessors, rocket components, industrial ovens, and much, much more. In the absence of any other knowledge, ANY software project should think of Tcl as its first choice for testing."

What more do you need?

RLH How about the book? : )

LV responds - I know what I need - help knowing where and how to get started.

Articles:

CL's article about Mo DeJong, detailing the benefits of a strong regression test policy. [5]
Michael J. Norton's "Network Test Automation with Mac OS X and Tcl" [6]
Ahmet C. Keskin, Till I. Patzchke, and Ernst vonVoight. "TclTk: A Strong Basis for Complex Load Testing Systems." Proc. 7th Intl. Tcl/Tk Conf. Austin, Texas: USENIX, 14--18 February 2000, pp. 53--60. [7]
Carsten H. Lawrenz and Rajkumar Madhuram. "Using Tcl to Build a Buzzword-Compliant Environment That Glues Together Legacy Analysis Programs" Proc. 7th Intl. Tcl/Tk Conf. Austin, Texas: USENIX, 14--18 February 2000, pp. 61--70. [8]
Paul Amaranth. "A Tcl-based Multithreaded Test Harness." Proc. 6th Intl. Tcl/Tk Conf. San Diego, California: USENIX, 14--18 September 1998, pp. 69--78. [9]
C. Allen Flick and S. Dixson. "Using TCL/TK for an Automatic Test Engine." Proc. 6th Intl. Tcl/Tk Conf. San Diego, California: USENIX, 14--18 September 1998, pp. 79--88. [10]
Steven Grady, G. S. Madhusudan, and Marc Sugiyama. "QuaSR: A Large-Scale Automated, Distributed Testing Environment." Proc. 4th Intl. Tcl/Tk Conf. Monterey, Calif.: USENIX, 10--13 July 1998, pp. 61--68. [11]
David E. Smyth. "Tcl and concurrent object-oriented flight software: Tcl on Mars." Proc 2nd Tcl/Tk Workshop. New Orleans: CPU, 23--25 June, 1994, pp. 3--10. [12]
Arjen Markus' paper "Generating test programs with TestMake" [13] Dead link US
...
[ Android, ...]
...
http://www.commsdesign.com/design_corner/OEG20020123S0058

The Open CASCADE 3D modeling application development platform relies on Tcl for its testing framework [14].

GS - Tcl can do more with OCC: it is used for prototyping applications with a tool called Draw.

AutoIt [15] is a Windows-specific DLL that can simulate key presses, mouse movements, and so on. It looks like something that would be fun to wrap with Tcl. I know of no one who's done that, though ... MPJ ~ I just added the Ffidl wrapper code to access all the AutoIt functions here [16].

More testing products: cwind, WinTclSend, ...

MR A good directory of systems... http://www.opensourcetesting.org/ which unfortunately only lists 3 tools for doing tcl unit testing...

"Techniques for 'driving' Windows applications"; FIT; TETware

One product that addresses the same need in the Mac OS X world is Redstone Software's Eggplant.

AM Here is another page on testing: Generating random tests

Joel Spolsky writes on testing sometimes: "Top Five (Wrong) Reasons You Don't Have Testers" [17], ...

There is the tclwebtest package for testing web applications. And the Pipeline programming driver.

An important landmark in the testing vista is "IEEE 829" [18].