Stubs

A stubs table or function table is a C structure whose members are the functions that comprise the interface to a shared object.

See Also

Writing extensions for stubs and pre-stubs Tcl
Extension Stubs Tables
How to provide a stubs table.
How to embed Tcl in C applications
Includes examples that use the stubs mechanism.
How to Use the Tcl Stubs Library
InitStubs TkInitStubs
the important man pages.
Tk ticket 1716117 , improve genstubs/checkstubs
Tcl ticket 1819422 , tclStubsPtr out of libtcl
Tk ticket 1920030 , Ttk stubs table

Documentation

ANNOUNCE: Stubs: A dynamic linking mechanism for Tcl, Tk and extensions , Paul Duffin, 1999-01-14
The original announcement of the stubs mechanism.

Acclaim

For me stubs is simply a brilliant feature of tcl.
Georgios Petasis

Description

The stubs table mechansim is an alternative to using the system runtime loader to resolve symbols in shared objects. It provides a portable, more uniform way to perform symbol resolution betweeen dynamically-linked objects. Tcl and Tk provide several stubs tables for extensions to use, and an extension itself can make stubs tables of its own available to other extensions.

Because stubs table initialization bypasses the symbol resolution features of the system runtime linker, it is not subject to any constraints imposed by that linker. For example, A Tcl extension can call functions in the host Tcl library that even if the system linker doesn't provide this capability, as was historically the case with some runtime linkers.

The name "stubs table" is a little unfortunate because it implies that members of the table structure are filled in at runtime. In reality the stubs table itself exists, already entirely populated, in the shared object that provides the functions. function table is a more fitting name.

A stubs object file provides a global variable to store the stubs table in and an "InitStubs" function that retrieves a pointer to the stubs table from the providing object and stores it in that variable. A corresponding header file rewrites each reference to a relevant function into a reference to the corresponding member in the table.

Instead of linking Tcl directly to an extension's shared library, an extension statically links into itself the stubs object files providing the functions it wants to use: Those provided by Tcl or by other extensions. When Tcl loads the extension and calls the extension's initialization function, the initialization function in turn calls the "InitStubs" function for each desired stubs table, after which the stubs table is available for use. In the case of Tcl, this function is called Tcl_InitStubs().

An extension that uses the Tcl stubs tables must be built with the USE_TCL_STUBS macro defined, which rewrites all Tcl_* function references into references to the corresponding members of the stubs table. Each extension that provides stubs tables provides similar macros.

In the case of Tcl, the stubs table is stored in Tcl_Interp structure because that structure gets passed to the initialization function of an extension, which allows the extension to get at it via Tcl_InitStubs. An extension provides its own stubs table by returning it in the clientData data of Tcl_PkgRequireEx.

When a program embeds Tcl, it can link against the Tcl stubs library and then call Tcl_InitStubs() to get access to the Tcl C interface. It must do some work in order to get a handle on an interpreter it can use to call Tcl_InitStubs. There are examples at How to embed Tcl in C applications.

The stubs mechanism has the following advantages:

  1. Adds support for backlinking (resolving symbols in the loaded library to the the loading library) to all platforms, thus making it possible for static executables to dynamically load extensions on any platform.
  2. Minimises the involvement of the operating system's runtime linker, which provides a more consistent procedure for loading an extension, or any other library which exports a stub interface.
  3. Avoids symbol pollution, which broadens the range of Extensions and libraries built with different compilers will work together even if it is not possible to link them normally. The only requirement for them to be able to work together is that function calls are compatible.
  4. As extensions do not have a hard coded reference to the library it is possible to use them with any library which is compatible. e.g. an extension which was originally built for Tcl 8.0 should work with Tcl 8.1, and an extension built for Tcl 8.1 could work with Tcl 8.0.

A shared object that doesn't use a stubs table to call interface functions of other shared objects such as Tcl, but instead is directly linked to those objects when it is created, is bound to those objects by a particular name and version, and parties to whom the shared object is distributed must provide libraries with the right version numbers. It is then often also necessary to recompile such a shared object when switching to newer version of the shared objects it depends on.

A Tcl extension that uses stubs to call into the Tcl or Tk C API can be loaded by any version of Tcl that supports the API required by the extension. Many extensions use stubs in this way, and any extension that is included in starkit must use stubs.

If it provides a C interface, an extension (or other library) may also provide a stubs table of its own. For example, TclOO provides a stubs table that other Tcl extensions can use. TclOO, Tk, and memchan are all examples of libraries that both use stubs tables and provide stubs tables.

Compare and Contrast

To see the differences take a look at the following which describes what happens when Tcl loads two different extensions.

Loading an extension which is linked directly to Tcl:

    1. Tcl asks the operating system to load the extension.
    2. The operating system loads the extension and and libraries it depends on and tries to resolve any undefined symbols
    3. Tcl then calls the extension's initialisation entry point.

Loading an extension which uses Tcl's stub interface:

    1. Tcl asks the operating system to load the extension.
    2. The operating system loads the extension and and libraries it depends on and tries to resolve any undefined symbols, but there are no Tcl symbols in the extension to resolve.
    3. Tcl calls the extension's initialisation entry point.
    4. The extension calls Tcl_StubsInit() to fill in the stubs table, and then calls Tcl functions through that table.

Who's responsible for the Stubs implementation?

Perhaps Jean-Claude Wippler suggested it to Paul Duffin in 1999, and Paul and Jan Nijtmans, with whom Jean-Claude had also been discussing ideas, implemented it in 2000. Others involved in the first generation were ??? (backlinking details) ...

DKF: I certainly remember suggesting something like the mechanism we ended up with back before Paul produced his initial version, though his initial version was a lot more elegant...

Questions

How can I tell if a binary extension was built with Stubs?

Under Linux and similer Unixes, check with ldd. No Tcl or Tk library references should appear. For Windows, do the same with "dumpbin /dependents".

Where is it most important to use Stubs?

Well, one place is that development against a Tclkit/Starkit/Starpack environment, extensions that are not built using Stubs result in more difficulty. (Or is it actually that they cannot be used at all?)

What are my options if the extension I want to use isn't Stubs-compatible?

Is all this stubs stuff necessary? Why does the build system have to be so difficult. Believe us: this is far less difficult than the alternative. Yes, libtool and autoconf are maddening, but there are things outside the Tcl world that demonstrate we don't have it so bad. DKF briefly defends stubs in a follow-up [L1 ] to a lengthy thread that itself touches several aspects of generation.


Helmut Giese notes that, even when compiler providers agree on object formats, they might still construct libraries in incompatible ways. In particular, under Windows, those who choose to work with gcc-based compilation might need to rebuild tclstub84.lib before Stubs-enabled extensions load correctly.


Georgios Petasis and Michael Schlenker observe that Stubs-less languages need "batteries included'', because reliance on version-specific extensions would otherwise be prohibitively onerous: "the C interface of these languages seems so primitive that you have to recompile everything each time a new version is out."

Phils take on stubs Stubs - Another explanation


KBK: "Even in version lockstep, stubs can be handy if only to keep all of your libraries following the same linkage conventions."


hat0 Please note that, in building a cross-platform stub-enabled extension, that Windows may require this modification to your C code's declaration:

- int Extension_Init(Tcl_Interp *interp)
+ int DLLEXPORT Extension_Init(Tcl_Interp *interp)

Without this, linking may fail in curious ways, with errors such like:

Cannot export ??_C@_03KBFG@Tcl?$AA@: symbol not found
Cannot export ??_C@_03PFGM@?$CJ?3?5?$AA@: symbol not found
Cannot export ??_C@_0BC@JJNH@?0?5actual?5version?5?$AA@: symbol not found