Tcl and threads

The thread model implemented in the Tcl thread extension goes a long way toward alleviating the problems that arise with threads.

See Also

Tcl Threading Model

Sew what - a discussion of Tcl and threads

Concurrency concepts

Tcl Benchmarks with Threads

Threads vs. events

thread

Lifts the threading capabilities of the Tcl C API into the script level.

Thread-safe Tcl Extensions

A list of extensions believed to be safe for use in multi-threaded Tcl applications.

mkextensions

an older thread extension

Modest tool for simulating threads

AM 2008-08-26: its purpose is to provide insight in what a multithreading program (or the relevant synchronising parts of it) is doing. Mind you: a very modest tool.

development targets: Complete pthreads? , last modified 2004-08-13

an open RFE containing information about the development of the Thread extension. Perhaps mostly historical.

call for a function that is not thread safe , last modified 2003-01-24

an open RFE regarding the Thread extension. Perhaps mostly historical.

Tasks

ET 2021-09-21: An extension to tcl threads, designed for simplicity. Primarily used for the two most common uses of threads: (1) to keep the Tk gui responsive while a proc is heavily computing in a separate thread, (2) Use a single queue - multi-server setup to compute the same proc in separate threads for performance gains. Tasks usage looks like the more familiar call/return framework that the beginner tcl programmer might find easier to use, especially for the multi-server case.

Description

see thread for links to documentation on the Tcl script-level usage of threads.

Remember: "You may have more than one interp per thread, but never is one interp shared _across_ threads," as David Gravereaux posted to comp.lang.tcl. In other words, different threads can't share an interp.

Alexandre Ferrieux had this to saw about it way back in 1999-07:

This is related to something I've realized only recently with the Tcl threading model: a purposeful, strong isolations of threads. At the beginning, I believed that "at most one thread by interp" was a dirty hack to hide bad reentrancy in the core. Now I have come to understand that instead, it pushes forward a new (as compared to C, as you mentioned) and also very "Tclish" style: basically, Tcl threads are nearly as isolated as *processes*, which is nice because it means all the modularity and atomic-testing we want, without the fork/exec and address space switch overhead !

-- cl/Tk 8.2 in the works , comp.lang.tcl, 1999-06-26

DKF wrote in comp.lang.tcl:

"When I deal with threading, I work on the principle that the hardware and OS are my enemies and hate me, and so I code accordingly. It probably doesn't make for a maximally-efficient program, but it does reduce debugging pain."

Later, he wrote in comp.lang.tcl:

I reckon it is easier to think about this whole business like this: Processes contain threads (at least one) and threads contain interpreters (at least one.) The current directory is a process-resource, just as the call-stack is a thread-resource (Note that variables on the stack are *also* interpreter-resources.) and commands are interpreter-resources. Interpreters may be either safe or unsafe, and the master interpreter for each thread is always unsafe. IIRC, cd is an untrusted command by default (i.e. it is up to the master to decide whether to provide a vetted version of it.)

...

I operate (even in much more controlled situations than what's being considered in this thread) by never writing code that depends on the current directory, effectively assuming that after the first bit of the startup process the value of pwd can vary wildly. Luckily, file join makes it easy, even when the code is meant to be cross-platform.

Some consolidated rules for embedding Tcl in a threaded application:

• You can NEVER use the same interpreter from more than one thread
• If you only have one Tcl Interpreter:
• You can use either Unthreaded or Threaded Tcl
• No Big Global Mutex Big Global Mutex is required for Unthreaded Tcl Build (Never required for Threaded Tcl build)
• If you have Multiple Tcl Interpreters:
• If you are using an Unthreaded Tcl Big Global Mutex Big Global Mutex is required for ALL calls to functions in libtcl
• If you are using a Threaded Tcl, no mutex locks are required

Example of all three situations (Single Interp, Unthreaded/Threaded Tcl, Multiple Interps Unthreaded Tcl, Multiple Interps Threaded Tcl) can be found in PNFQ, assuming it is released.

David Gravereaux is expert in Tcl threading. Also, Jean-Luc Fontaine has been thinking about this deeply for his TclPython work.

Explain how Tk isn't thread-safe [is it Tk that isn't thread safe, or is it the windowing system underneath Tk?], but GUIs never are, and no one cares (AW actually, we do, see my debugging example below), because they always have a single GUI thread, with others as helpers. Windows specifically only allows a window to receive events from the thread that created the window (can that be true?!?).

One should not expect to use a tcl library compiled with thread support and Tk without thread support compiled in. What you need to use is a tcl and tk both with thread support compiled in - but then be careful to not use tk in more than one thread at a time.

Much more deserves to be Wikified. Consider, for example, these comments of Joe English:

"And David Gravereaux' advice is worth repeating: making extensions multi-interp safe goes a long way towards making them thread-safe.

I find it easiest to make extensions multi-interp safe and thread-oblivious: if you avoid global and static variables, keeping all the extension's internal data in a per-interp data structure, you can often avoid locking altogether."

Explain Tcl (and Tk) generation options.

Internals

CMcC: Each tcl core subsystem has Thread Specific Data.

Misc

Until the generation procedures are cleaned up, workers in this area will need to be familiar with "-DTCL_THREADS=1".

History of Tcl threads: Steve Jankowski released "mttcl" [L1 ] in 19??. D. Richard Hipp ... Jim Davidson's work for NaviServer ...

Zoran Vasiljevic makes the interesting point that "If you have many threads, load Tcl interpreter in each of them and then load a module in each interpreter, then your references to a global hash table will definitely go to the same table instance. Which means you need to protect access to this table with a mutex.

If you need separate tables, i.e. you need not share hash-table data between threads, then you should put the table in thread-specific data structure (TSD), so each thread gets its own copy You would also like to register some thread-exit handler which takes care about finalizing those tables on thread exit, otherwise you'll lose memory."

JMN What sort of things will create 'a global hash table'? I see from another page that arrays are 'implemented internally by hash tables', so does this mean I have to do something fancy just to access a global array from threads? Does this even apply to using threads at the Tcl script level? If not, perhaps it'd be possible for someone knowledgeable to separate out the information on this page that pertains to the C-API to keep such terrifying statements away from those who just want to use the Thread package from script. Ok.. perhaps there are some good reasons to keep people just a little wary of threads - but I'm not sure the above belongs right next to the nice welcoming 'cookbook approach' below.

PYK 2015-04-12: I believe Zoran is referring to data structures that an extension might create at the C level. At the scripting level, All variables, including arrays, local to a specific interpreter. The only resources that are shared between interpreters are those enumerated in the thread page.

tclguy "just wanted to note that I have found the intermixing of thread-enabled and disabled Tcl and extensions to actually be acceptable in most contexts where only one thread was used, or if other Tcl-level threads were created, they were only used for Tcl."

Which statement, while true, requires a lawyer to determine whether it has any significance relative to a project using threads. It sounds like you're saying "if you don't use threads intentionally, the multi-threading Tcl will not bite you otherwise". Is that right? I mean, is that what you meant to say; I don't think you actually know if it's right.