Tcl OO Bench

This comparison of OO extensions of Tcl is based on OO benchmarks of the OO shootout [L1 ] and consists of the OO Method calls (source [L2 ]) and the Object Instantiation Test (source [L3 ]).

These tests were performed on a Pentium 4 M Notebook with 1400 MHz under Fedora Core 1.0 with tcl 8.4.5.

XOTcl 1.2.0

  methcall:  0.460u 0.000s 0:00.53 86.7%        0+0k 0+0io 347pf+0w
  objinst:   0.960u 0.000s 0:00.97 98.9%        0+0k 0+0io 348pf+0w

ITcl emulation in XOTcl (see Itcl in XOTcl)

  methcall:  0.590u 0.000s 0:00.62 95.1%        0+0k 0+0io 351pf+0w
  objinst:   1.760u 0.010s 0:01.74 101.7%       0+0k 0+0io 352pf+0w

MIT OTcl 1.0

  methcall:  0.980u 0.000s 0:00.95 103.1%       0+0k 0+0io 329pf+0w
  objinst:   2.370u 0.010s 0:02.37 100.4%       0+0k 0+0io 329pf+0w

itcl 3.2

  methcall:  0.870u 0.020s 0:00.92 96.7%        0+0k 0+0io 347pf+0w
  objinst:   1.790u 0.010s 0:01.84 97.8%        0+0k 0+0io 348pf+0w

stooop 4.4

  methcall:  1.640u 0.040s 0:01.73 97.1%        0+0k 0+0io 326pf+0w
  objinst:   4.030u 0.000s 0:05.75 70.0%        0+0k 0+0io 316pf+0w

classytcl 1.0

  methcall:  0.740u 0.010s 0:00.78 96.1%        0+0k 0+0io 335pf+0w
  objinst:   3.020u 0.030s 0:03.09 98.7%        0+0k 0+0io 335pf+0w

The full source and results are available at [L4 ].


Jean-Luc Fontaine: Please note that for stooop, in the method call benchmark, the value procedure has no reason to be virtual since it only returns the value of a data member in its class. Removing the virtual qualifier results in a huge improvement in speed (on my Xeon 2.4 GHz Fedora 1 Linux machine, with Tcl 8.4.5):

real 0m1.668s, user 0m1.660s, sys 0m0.010s -> real 0m0.394s, user 0m0.390s, sys 0m0.010s

GN It is true that the benchmark for stooop can be improved significantly by removing the virtual qualifier. On the same machine as above, we get the following results:

stooop 4.4

  methcall:  0.480u 0.020s 0:00.52 96.1%        0+0k 0+0io 326pf+0w
  objinst:   4.020u 0.000s 0:04.06 99.0%        0+0k 0+0io 316pf+0w

However, the benchmark specification says that ... the constructor for NthToggle should use the constructor for Toggle to inherit the boolean field and value() method. If virtual is removed, value() is not inherited, therefore the implementation is not correct in the sense of the benchmark (although correct from the results of the methods).

If someone else sees improvments in the tests, please note.


Jean-Luc Fontaine: Also, here are the memory consumption results, which are as interesting in my opinion:

creating 10000 objects

itcl

   Used memory: 4640768
   Memory per itcl object: 464
   Time per object: 9.43

MIT otcl

   Used memory: 2838528
   Memory per otcl object: 283
   Time per object: 17.62

XOTcl

   Used memory: 2162688
   Memory per xotcl object: 216
   Time per object: 7.72

Stooop

   Used memory: 1351680
   Memory per stooop object: 135
   Time per object: 22.36

ClassyTcl (C based version)

   Used memory: 2588672
   Memory per classy tcl object: 258
   Time per object: 4.6312

Snit

   Used memory: 20787200
   Memory per snit object: 2078
   Time per object: 2006.91

There was a problem in this test for ClassyTcl, because the "namespace import xotcl::*" created a Class command in the global namespace, the compiled version of ClassyTcl was not loaded in this test. On my system, the compiled version scores a bit better than XOTcl both on memory and time PDR.

GN: Thank you for noting, the version above is now corrected, the results are again on the same machine with the same tcl version (8.4.5). Compiled ClassyTcl appears to be the fastest in this test on object creation. Memory consumption is between XOTcl and OTcl.


GN: There is certainly a way to speed up the object creation in XOTcl: When an object/class is created in XOTcl, the following (XOTcl) methods are called to configure the object properly:

  • alloc: create the object/class, define heritage etc.
  • searchDefaults: search for rules to define default values
  • configure: evaluate the methods specified via "-name args"
  • init: call the user constructor method

If the task is simply to create objects, the command to create objects could be changed from 'Object create $i' into 'Object alloc $i', leading to the following figures:

   Used memory: 2162688
   Memory per xotcl object: 216
   Time per object: 3.83

DKF: But that would be cheating. :^) Let the test be of creation and initialization of an object; it can be a simple object, but it must be fully initialized.

KJN: similar tests are discussed at Comparing Performance of Tcl OO extensions