Things holding Tcl back

Too many functions in the Tcl and Tk sources are very long. This makes it difficult to modify or understand some code. I like the idea that RS proposed that a good proc fits between thumb and forefinger. This doesn't apply to some things. Doing switch statements for some things requires large function/procedure bodies.

There is an idea that the Tcl sources are so good that they don't need improvement. This is usually backed up by referencing John Ousterhout winning an ACM award. (is this really a pervasive feeling, or just one person's opinion?)

The idea that we shouldn't be focusing on removing and simplifying code holds us back. I like the work that Miguel (MS) has done to remove bottlenecks.

Arrays need to be integrated more with Tcl. It would be very useful to be able to return an array from a procedure, and pass an array to a procedure.

Please add your own thoughts on what is holding Tcl back, and don't forget that there are things pushing Tcl forward as well.


HJG 2011-02-25: I think the single most important feature Tcl/Tk is still missing, is native print support. As in "included in the standard Tcl installation", working on all supported platforms, not requiring any extra software, drivers or extra administrative work.

E.g. print an 'simple' invoice-form, with several fonts, using some foreign characters, and some graphics, like a company-logo. Invoice-Demo. And do it without any tricks, like ghostscript- or pdf-printer-driver, or calling a browser or other external program to do the printing, etc. Bonus points if it also works from a starkit, without changes, on Win/Linux/Mac.

Wouldn't that be a worthy project for GSoC ? See also Tcl/Tk Printing Support

LV In my opinion, the concept is easier to want than it is to spec out and then implement.

  • Is this going to be cross-platform functionality? If not, perhaps the functionality already present in one of the platform specific extensions will suffice.
  • Will this be a Tk only function? If not, then the whole infrastructure of dealing with fonts will need to be implemented without dependence on Tk. If it is going to be Tk based, then that might limit certain types of applications (embedded devices, etc.)
  • Whether with Tk or without Tk, the output format is going to need to be chosen. Most printers today provide some level of support for PostScript (do all the currently supported OSes provide drivers for PostScript printers?); to me that seems a natural direction.
  • You specifically mention "forms" - if Tk is not going to be the framework, then an extension to manage printed page widgets (frames, check boxes, etc.) will be needed)
  • I forget the rest of the list - it has been discussed in the past. These are not things that prevent what you see as a need from being implemented. These are just a part of what a developer has to consider when taking on a simple task like this.

peterc 2008-06-17: On a technical level, Tcl needs: (1) Add Drag and Drop support for Mac. (2) Printing support! (3) Support for more textured backgrounds in Tk and Tile on Mac. Pinstripe's terribly unfashionable. Ideally, we should be able to define our own background images. (4) Alpha transparency in widget backgrounds. (5) A KDE theme for Tile, incorporated in the base Tile download. (6) A Gnome theme for Tile, also incorporated in base Tile.

On a promotion level: I do think Tcl/Tk has been paying a heavy price for holding onto Classic Tk's look and feel for far too long. When people think of Tcl/Tk, they think of ugly pink squares and diamonds from the dawn of computing prehistory -- and that also coloured their perception of the currency of the rest of the language. Tile's been a great step forward, but, it'll take time for that to sink into the mainstream zeitgeist.

NEM Notes that these are all rather Things holding Tk back rather than Tcl issues, so could go on the Tk 9.0 wishlist. 1-4 seem like nice ideas, if someone has time to write the code. (5) is unlikely to happen due to licensing reasons, IIRC, but there is a standalone Tile Qt.


DKF - TclExecuteByteCode is a bit of a huge monolith and necessarily so, but fortunately most people do not need to look at it. Most other bits in the core (with a couple of exceptions) are small enough to be comprehended comprehensively at a single sitting.

Also, I have an idea for a dictionary value type - implemented as a hash, looking like the result of [array get] - much as we currently have a list value type. Look for a forthcoming TIP when I get the time...

LV note that DKF's dict value type is now in Tcl 8.5.


It seems to me the biggest thing holding tcl back is a lack of manpower. Too many people using tcl to solve real world problems, and not enough people whose sole job (or at least personal quest) is to maintain and extend the language.


"Too many people using tcl to solve real world problems, and not enough people whose sole job (or at least personal quest) is to maintain the language." -- Hmmmm.... the reverse could be said to be the problem of some other languages ;-) -- TAC


Vince - Perhaps we can identify those parts of the Tcl/Tk sources which are too long or hard to understand. Having personally worked on the filesystem code, I can state very strongly that that code is wonderfully clear and well written and if it hadn't been, I wouldn't have been able to complete the vfs code we now have in 8.4. On the other hand, other parts of the core (the shared internals of 'exec' and 'open |' are not easy to understand, and my attempts to abstract those away from the filesystem failed in the end). To that extent I can explicitly list one thing holding back that code: the use of a special 'TclFile' type internally instead of generic Tcl_Channels. The code would be much clearer if channels were used -- so, get rid of TclFile. If anyone wants to pick up the baton, my attempted patch is at ftp://ftp.ucsd.edu/pub/alpha/tcl/chanExec.patch - it passes all except 4 of Tcl's tests.

DKF - For me, lack of a standard object system is probably the single greatest factor. Sure, I know there's Incr Tcl, but there's also stooop and snit and ... We could do with a standard, even if it was just a base-functionality standard that those other packages build on top of.

lv curious - can you think of a single example where someone decided to reject tcl because not one of the oo systems were sufficient for the task? have there been many who said well, there are too many ways to do oo, so we will just go use python or java, where we don't have to use our brains? Hmm - that comes across as being upset or angry, when I read the words. And I don't mean it in an argumentative manner. I'm genuinely curious if people are rejecting tcl because there's too many oo systems.

DKF: The problem is more subtle than that. The lack of a standard makes sure that people are busy working against each other rather than working together. This is resulting in less work being done in new areas because everyone's thinking about the best way to reinvent the wheel instead, and it is the new code (and user-outreach) that makes the language move forward.


Vince - As documented elsewhere on the Wiki, the obfuscated regexp implementation Tcl uses seriously prevents any development or bugfixing in that area.

FW personally thinks Tcl's regexp system, obfuscated or not, is great; behind the scenes, it's a combination of the best features of many popular regexp algorithms, and is by far one of the fastest out there.

Vince Sure, but it still has some bugs, and no-one seems to be capable of fixing them, because the code is so obfuscated.

DKF: Sure, but I don't think that's holding back Tcl.

x: The regexp implementation is an example of obfuscated code, which in general is holding Tcl back. On regex sucking: http://swtch.com/~rsc/regexp/regexp1.html

ares: I don't know whether obfuscated or not, but the source you cited states that TCL indeed uses NFA based regex implementation (they are arguing for NFA based implementations, in contrast to DFA based), so what was your point again ?


The main thing holding tcl back is the limited amount of time the general community contributes to writing new core code. The maintainers do a yeoman's job of working getting new releases ready, with patches, etc. and several people contribute new code on a regular basis. but there's another n-thousand users of tcl whose contributions would enable tcl to move forward faster.


Tcl also hasn't kept up with http 1.1 (and extensions such as WebDAV), which Perl, Python etc have. Or for that matter, the image formats supported within the Tk core distribution is rather limited compared to what one would expect, resulting in dismissal of Tk as a serious toolset.

DKF: There are extensions to deal with this. Why this insistence on the core solving everything?

There are at least two arguments for including things into the core.

The first is that people want functionality directly out of the box, without having to go searching for various tcl extensions, figuring out what dependencies these all have, then discovering the subdependencies, etc. In my (lv) mind, this is an argument for ActiveTcl or some similar Batteries Included solution. The main problem with the current batteries included solutions are their limited platform availability - ActiveTcl is, for example, available as a binary distribution on Intel Windows, Intel and SPARC Solaris, Intel 32 and 64 bit Linux, MacOS X, AIX, and HP-UX. However, for any other combination, they have to go off and do all the work that was and continues to be done for ActiveTcl - reinvent the wheel, so to speak. And there is no single location where one can download the source to build one of these Batteries Included binary distributions for a new platform - instead, one has to download from various locations around the internet, struggle with the incompatibilities in build systems, etc.

The second argument is from people wanting to write all their code in Tcl scripts, so that it is more portable. Having to gather and compile many of the extensions, then write their own glue to various image formats, etc. that are not currently supported, decreases the portability of their systems.

PWQ 10 Sept 2003. I have strong views on this and I have created a page for them here Tcl NG.

JH Note that there are only 8 public platforms of ActiveTcl available. There are numerous more platforms available with full enterprise support (a raft of itanium-based platforms, big iron, and less common platforms) that are not made public because each platform comes with a maintenance cost. It actually takes work to make these available (surprise).


I consider myself a firm member of the "middle ground" of programming/scripting types with an eye towards higher level languages, but I think TCL suffers critically from the following :

Native 00 support, (as mentioned above) and lack of native DB connectivity. Also, why does ActiveState mediate such things as browser plugins? what is that! Talk about shooting itself in the foot. With a plug-in available free of charge built on the latest Tcl ver, I would be inclined to think Activestate would increase revenue by everyone seeing just how fast it is to develop and embed enteprise quality applications into a browser. I think it's a grievous flaw in their whole capitalization/marketing scheme.

edit May 13, 2004 : I went looking for the open source plugin before posting this obviously, but sourceforge doesn't have any files listed under the project, so I assumed (yes (ass)(u)(me)) that AS had stopped releasing the open source version. am I looking in the wrong place? http://sourceforge.net/projects/tclplugin is where I went. also, the only url I associated with keiichi is http://2.csx.jp/users/bitwalk/tcltk_plugin_download.html which now seems to be a Japanese 404.

Why would I buy Komodo if I dont think TCL is cool/useful/quick what-have you? That's what they should be asking themselves.

Basically Tcl core needs the modern components that drive public interest. I want to see a web browser enabled with a Tk widget from which I can then read and write to a DB. Written in 20 lines. THAT'S compelling. Web-enabled applications can then decrease their damn dependence on hassle-filled form variables...

JH May 2004: The above author needs a whack with the clue bat. ActiveState does not mediate anything. We are a (major) participant in many fully open source projects (the core + countless extensions), sometimes the only participant. Although I single-handedly revived the Tcl browser plugin, everything was contributed back to the open source version, as we do with so much else. Every line of code is available to others, which is why folks like Keiichi Takahashi have alternative downloads. The plugin in the Tcl Dev Kit is also unlicensed, it just happens to be bundled with it (it's not a very simple thing to install).


LES on May 12, 2004: I don't program professionally, but I have two takes on the title issue:

- if we're talking about why Tcl isn't as popular as tclers think it should be, I'd say it's the lack of OOP or confusing excess thereof. I couldn't care less about OOP, but it's obviously something without which most programmers will simply cross arms, sulk really hard and refuse to give any step further;

- if we take take the title for its face value, I'd say that nothing holds Tcl back, except the already mentioned lack of manpower. I can't see Tcl becoming popular and widespread to the point of "rivalling" big-leaguers like Java or Perl. It's just too different from most of all that, in a way that will force experienced programmers to let go of a lot that they know and dominate. Children can let go of their values and grasp new ones quite easily, but adults will promptly frown and fight back. I see Tcl as an "alternative" language, as opposed to "the mainstream". This "alternative" position is usually quite palpable and unquestionable, but still subjective. Just because most people don't follow Tcl is doesn't mean it's going in a wrong direction, or that something is "holding it back".


MSW - 2004-05-12

Actually I think the lack of OOP is one of Tcl's great strengths, as it is easy to extend, the programmer has the choice of the Flavour of OOP wanted, and the end-user doesn't realize at all because it's just another package which is required. I don't want to step on someone's toes, but assume [incr tcl] was the standard toolkit, I suppose we'd lose a lot of people complaining we don't need another Java... Given you do have the choice (OTcl, XOTcl, [incr tcl], etc. etc.) you can program in whichever paradigm you like in Tcl. Now if that's no selling point! No one coming along and telling you "Listen, you MUST do OOP in this way!".

RS agrees fully - it is so easy to create an OO framework if one needs it, and tailor it to one's needs. I admit I did many experiments in this way, but never decided that I'd use any of these as my own "standard"... My favorite style still is simple pure-values :)

DKF: I understand that PoV, but disagree. It's one of these subtle cases where I think the costs outweigh the benefits.

MSW at DKF: If you favour one toolkit, nothing hinders you to distribute a custom interpreter which has the specific packages you use(d) builtin ? I mean as an option of local optimisation.


LES I would like to have another two takes:

1: Less is more; so more is less? Many people don't handle freedom too well if they're given too much of it. I'm afraid I am one of those. I often feel very tempted to use customization features like rename, sugar, custom curry and radical language modification. But I always chicken out. Sometimes I have to maintain code that I wrote months before and even if I stick to the regular syntax, I always take a while to figure out what the heck I was thinking when I wrote that code... so I stick to the regular syntax, humbly. Just imagine if I had gone nuts with customizing Tcl. I am afraid I will forget what such customizations mean. I am afraid I will forget how they work and the manual won't be any help because my customizations aren't documented. I am afraid I will not be able to share that code because no one else will understand it. I am afraid I will not be able to reuse someone else's code because my little monster is no longer compatible with standard Tcl... You get the picture. I don't know of any other language that has what, half a dozen different implementations of OOP. Many newcomers are likely to ask themselves - heck, which one is the standard? They'll have no answer because there isn't any, or rather there are many. Some people will just look at that many and take none. And that will only happen provided that they ever run into or trip over all those extensions hither and thither... Wait, and what if they don't pick "the best one"? What if they decide to adopt another one later and, ugh, have to rewrite all the code? What if they hate the Tcl community for not telling them which one was "the best" before they made their ill choice and eventually had to rewrite everything? " - That wouldn't have happened if the community had picked one and commited!" Some people just need a fixed route. A warm and fuzzy standard. An implementation that they can share, reuse and discuss with an entire community, not just the author and half a dozen aficionados while another part of the community discusses some other implementation that may or may not be a lot better - who knows?

This is not my personal view. I am just speculating.

2: To core or not to core - here is the question I once read an interesting discussion about core versus extensions and the lack of a centralization (Tcl Common Library). I think it is a nasty obstacle. I am pretty sure that some newcomers never get past the core and well acquainted with extensions because everything is so scattered all over the WWW. You will find if you look, but Tcl is so often ignored or even scoffed at... who is going to bother? So I believe that many hasten to conclude that "Tcl doesn't have this" and "Tcl can't do that". I would like to suggest that as much as possible of Tcl and related code be made available in one single page that would look like this:

edit by IL :: Great point! maybe we need a CPAN of our own...

    Welcome to Tcl downloads!

    Tcl is divided in two parts: the "core" and "extensions". Extensions are optional, but may come in very handy!
    --------------------------------
    Download the core
    {link to Tcl}
    {link to Tk}
    --------------------------------
    Download extensions
    list of
    extensions
    with 
    brief
    descriptions
    and 
    DIRECT links !!!!
    to their respective packages
    --------------------------------
    Other extensions
    links to extensions that could not be included here because they have some dissonant license or something else
    --------------------------------

If not in the core, at least that: one single page. Not one site. One single page. Make sure that as little as possible ever stands a chance of being missed.

This one is my personal view.


LV If too much freedom paralyzes you, then don't recognize the freedoms! Choose an OO for yourself and restrict yourself to it.

I frankly find the idea of trying to limit the language to one OO, or one widget system, or one anything else to be philosophically offensive.

NEM Clearly, you've not ever had to make this choice then. I find myself sometimes wanting to write code in an OO style (that's not a sin yet, is it?), but it is a tough decision with which extension to go with. XOTcl is perhaps the most flexible, incr Tcl is probably the most popular, but what if I am writing code for tcllib? Then I have to choose between snit or stooop. It becomes a mess - I end up with code written in all of these frameworks, rather than just picking one and restricting myself to it. I personally find the idea of having a wide choice of OO systems seems like a good idea, but really isn't. Ask any good user interface designer - choice is rarely a good thing.

LV From an application point of view, what difference does the choice make? The only place where a choice of OO makes a difference is if one is writing some kind of a component that will need to be used within the OO framework itself. At that point, I know how I make the decision - I choose the framework within which I myself work. Then, if someone wants/needs the component in another OO framework, they are free to create a parallel version for that framework.

NEM Errr... your argument is that reinventing the wheel for each framework is a good thing? So, I should just pick an OO framework I like (say, XOTcl) and implement all the functionality I want, and let others create parallel versions of it for Itcl, and maybe a pure Tcl version for tcllib? That's one solution, I guess, but it doesn't exactly strike me as a good one. Still, perhaps it's a better solution than the lowest-common-denominator style that I find myself writing currently.

LV Think of it this way. In your recommended approach, the wheel never gets invented because there is never an alternative framework. In the way I propose, the wheel gets invented only if there is something of interest that cannot be used as implemented and if there is enough interest by someone to do it. Most of the time, there isn't enough interest, so there isn't any wasted effort in code being reimplemented.

NEM I think perhaps we are talking about different things. I am referring to a common framework allowing things like a libraries/packages of useful functionality to be implemented (e.g. something like tcllib, but with an OO design, for those things where OO makes sense), and easily integrated/extended into/by an application (e.g. through inheritance - something which is difficult to do across frameworks, but is an often-used feature of OO). I don't think this would be impeded by only having one framework -- quite the contrary, I think it is currently being impeded to some degree by the surfeit of OO extensions available. At least, I would be more inclined to work on OO library components if there was a standard framework. The phrase "the wheel never gets invented because there is never an alternative framework" suggests to me that you are talking about something else -- inventing features which are part of the actual framework, and not functionality built on top of the framework?

I think of this as a deployment issue for library authors. When writing an application, you can use a starkit or such to bundle up all the extensions you need. When writing a library, however, it seems better to rely on what's there, as most developers probably wouldn't want each package to include copies of common OO extensions. When writing a library based on an OO framework, that framework forms the basis of your interface (an API rather than a GUI), and so (like when designing a GUI) you try to go for the most accessible, portable, and compatible interface you can, to ensure the widest possible adoption of your code (which in turn, benefits you with increased bug reports and patches). Without a common OO framework, this task becomes much more difficult, in the same way that designing a GUI is complicated by different OS/windowing systems.

LV actually, my fingers typed "the wheel never gets invented" when my mind was saying "the wheel never gets reinvented". If only one OO extension is permitted, as has been proposed, then of course there's never a need to reinvent anything. If there is more than one OO available, the only time some code has to be reinvented for a new OO is if someone who needs the functionality finds that the original author wrote the code in some incompatible manner.

One can make use of BLT, Tix, Tk, and other widgets in one application, even though all those authors used different development methods, differing megawidget methods, etc.

Itcl/Itk/Iwidgets was the only OO for quite some time. The other OO frameworks sprung up often because that combination didn't work for them - either there were missing features, it was too slow, etc.

There's nothing today preventing a software author from just using itcl, if they so wish. Or for that matter, just using snit, or scooop, etc.

I don't see regular complains from developers saying that they had written their marvelous new widget in brand X OO and now they have to worry about brand Y OO. If we were seeing those sorts of comments, then I could understand how having multiple flavors of OO was causing people problems.

NEM OK - I agree with your first point, mostly. Noone ever needs to reinvent the wheel to get access to some functionality. However, it's when you want to extend that functionality that things get more tricky. In OO this is often done through inheritance (I'm not going to debate whether that is a good thing, but it is often done). If functionality X is implemented as an itcl class, then in order to extend it through inheritance, I have to use itcl. I could extend it through explicitly delegation (using snit, or hand rolled in some other framework, or pure Tcl), but then you lose the ability to use inheritance at all (except in those cases where the framework you are using matches the framework that X was written in). The same is true for mega-widgets. I can use all those different mega-widgets easily enough, and I can extend them using explicit delegation. Both of these things only hold true because mega-widget writers, and OO framework writers, go to some effort to try and mimic how Tk widgets work, in terms of interface. There is no standard interface for inheritance though. I'm not sure that there can be, without there being a single common OO framework. Perhaps a minimal framework can be developed, which provides for Tk-widget-a-like interfaces, and inheritance, which other frameworks could build on, but I doubt it's feasible.

Zarutian 3. july 2005: Extending some class through inheritance violates that class encapsulation therefore limiting what that class author implements that class. So is delegating so bad?. If you are extending a class to gain more access to it's internals because the original author didn't provide that access when ver wrote the class then you are extending it's interface rather than it's functionality. In my opinion interfaces should be frozen for each version of a class.


CMCc I'd like to question some of LV's statements. He writes "Itcl/Itk/Iwidgets was the only OO for quite some time". I'm not certain of the timeline, but Tix had its own OO framework, and I was certainly aware of it before I was aware of Itcl. XOtcl's precursor from MIT was also around in the very early days.

LV also writes that he would find "the idea of trying to limit the language to one OO, or one widget system, or one anything else to be philosophically offensive." I think that's a position almost anyone would agree with, but I don't think it's what's being suggested. I think that's what's being suggested is that we support (at least) one OO and/or widget system properly - at least sufficiently to build a critical mass of reusable software.

LV writes "There's nothing today preventing a software author from just using [any OO system]", but I think that what's being suggested is that the lack of a critical mass of reusable software in any one of the OO systems does in fact impede an author from 'just using' it, either because none of the OO systems are 'just usable', or perhaps because all of them are in a different sense :)

There are virtues in all of the OO and megawidget systems we have available, and there are drawbacks. None of the OO systems shines out as a clear winner in all categories which might influence initial choice (say, usability, efficiency, expressive power, code critical mass, etc.) I think it's possible and even desirable to winnow them down to one (or perhaps two - one extension + one pure-tcl) OO system.

I suspect that the reasons this consolidation hasn't occurred is because it doesn't seem to be acceptable to risk stating that this or that feature or design is closer to the perfect OO system, whether because one risks offending the authors of the other OO systems (so that consensus is impossible to achieve) or whether because the very idea of the 'perfect OO system' is anathema to people who quite possibly don't use OO systems anyway.

My A$0.02.

MSW replies to the "None of the OO systems shines out as a clear winner ..." -- well, you are aware that this is always true, no matter the language, no matter the implementation of extensions, that is inherently linked to the different paradigms you can / they do employ. Other languages just have decided that given there's no clear winner they'd toss a coin and include whichever one appears to be lying on the upper side.

I don't think it's a lack of critical mass either, different people using tcl have a different (OO -- if they wanna use it at all)-background, and thus they should be encouraged to code in whatever they are familiar with. It's a division of the community in a way, but we're all under one roof .. just different branches of the same religion :> What we need to do is actually see a sense in using OOP (for more than 5% of all scripts overall :) and the critical mass(es) will build up fast enough. Problem remaining is that tcl itself, without an OOP extension, solves most of the problems quick and reusably enough. Don't reflect that badly on tcl as a lack of decision concerning the OO-flavour. Remember that's tcl's strength.

CMCC I think it's risible to assert that no OO language could possibly shine out as a clear winner, particularly in a language monoculture like tcl community.

Additionally, the contention that no single OO is clearly a front-runner in no way supports the contention that no OO should be supported by the core. I realise itcl is supported in the core, but AFAIK it doesn't even configure/make cleanly from CVS. It seems to me that an appropriate level of support in the core for a reasonable OO platform would be a good thing (rising tide lifts all boats, etc.)

Is there a page somewhere comparing features/flaws of each of the OOTcls?

LES So "Tcl solves most of the problems quick and reusably enough without an OOP extension" and that is a "problem"? I can't believe these OOP'ers...

MSW That is a "problem" for those who want to force (a certain flavor of) OOP on everybody. As I said for the rest of us we're aware that this is tcl's strength :)

CMCc I refer you to Imagine a language for a discussion of whether it's only one of tcl's strengths, or perhaps the source of some of its weaknesses too.

I don't know about "these OOPers", but I do know that slapping a label on something can only go so far to understanding it, so I'll try to address what I believe seems to be a source of misunderstanding.

I don't believe anyone's suggesting that we reduce the functionality of core tcl in order to force LV, LES, MSW (et al) to use only OO techniques. I think some people who would like to use OO are looking for a grand synthesis of the OO models out there to distill out the best, and come up with good OO support within tcl. At least that's my understanding, and my desire. Namespace ensemble is a good step in this direction.

LES The point made above is very clear, but everyone knows that Tcl has been striving to improve several of its aspects and Tcl development is so often said to be understaffed. So I wouldn't like to see effort be spent on something that has always struck me as superfluous. - pause for OOP'ers to recover from the shock and the hot drink that sprayed through their noses - Besides, I don't think OOP is a very good match for Tcl. I think Tcl is of a very procedural nature and I like it. Sometimes my day job forces me to write macros for MS Word - ugh! VBA! - and I hate every minute of it. If I had a crush on OOP, I would be learning Java or VB. I actually wonder why these passionate OOP'ers are coding in Tcl anyway. Wouldn't they be happy as ducks on a rainy day if they were coding in Java instead?


RR 13-May-04

I'd love to contribute to the Tcl maintenance effort, but I don't know any C. That gets me to one of my favorite fantasies: a reliable, robust Tcl/Tk -> C translator. That, I think, would really make Tcl an attractive player.

More to the point of this discussion, however, what is meant by "holding Tcl back"? Is there some notion that more people would use Tcl if only ...? In a previous entry in this thread, the author seemed to imply that Tcl is backward relative to Java and Perl. OK, what have they got that Tcl doesn't? Speaking only for myself (an admitted ignoramus) I only know from Java (I long ago chose Tcl over Perl). After many years of Tcl-only (after many years of Basic only, after a few years of FORTRAN only), I learned Java only because I bought a platform on which there was a native JRM but no X (therefore loading Tcl/Tk would be beyond my meagre abilities). I have to say, after re-writing a lot of my favorite Tcl app's into Java, that Tcl is far and away the better language (at least for what I do with it). So, again, what is it that those languages that aren't held back have that Tcl doesn't?


LV It seems to me, after having read comments about tcl dying, being held back, etc. that what the people posting these comments seem to mean is what is preventing tcl from becoming recognized as the best language (or at least as a valid and respected language) in the development community.


RR It isn't?


LV I didn't say that was my opinion. It just seems to be the mindset of those starting these discussions. Just think of how much documentation could have been written, bugs reported, examples suggested, etc. to the tcl base in the amount of time spent on this page alone, let alone the thousands of other bytes of web/email/usenet/etc. communication channels.

CMCc Larry, speaking personally: I can really choose any language I want to implement in, so I'm not motivated by fads or tech-cool popularity (else I'd be using Python :)

There are compelling reasons for OO, in my view, and they have to do with modularity, maintainability and reusability.

I find myself writing very large programs in tcl, and they suffer eventually from the very ease of authoring which Tcl provides. They benefit from encapsulation (which OO could provide, namespaces do.) They would benefit from clearer APIs, which OO could provide.

Tcl, for all its virtues, lacks complex data structures. Arrays are great, but they don't easily represent the relationships between data. Dicts might help, in that regard, but I think there are times when a programmer has to build complex data structures, and OO would help in that regard.

Personally, I'd like things which would make it easier for me to manipulate complexly structured data, give me clean APIs, and allow me to reuse others' work more easily, and OO inheritance would give do all that as well as being a popular and well known paradigm.

I've put a rant/think-piece under Imagine a language which is germain.

DKF: The key idea of OO is encapsulation. Even more so than inheritance.

LV: Re: complex data structures. I've heard the arguments that lists are sufficient for implementing any other data structure. I wonder whether developers looking at Tcl while having various data structure needs in mind would see the small number available via tcllib as a deterrent.

CMCc Larry, I read a story book when I was a kid in which the characters constructed all number systems up to transfinite using nothing but vertical lines inscribed on surfaces (Peano?). All programs and all possible data can be represented by presence and absence of signals ... for that kind of language, see Befunge, INTERCAL and Brainfuck. Additionally, there's a fascinating book called Laws of Form by G.Spencer-Brown which I strongly recommend.

Meanwhile, stepping outside of flatland, and notwithstanding Mahler's contention that the greatest art arises when the medium is most refractory, even though after Church/Turing all computational systems are able to be simulated by turing machines ... just because a given data structure can represent every other, does not mean it should. I think that carries reductionism a little too far.

Incidentally, I think the idea of OO is to implement abstract data types [L1 ], more than data structures.

AM Then should we not ask: what complex data structures are needed? (I remember creating a wiki page for this very subject ...)

LV: Well, we might take a look at Category Data Structure to see which ones have been discussed here on the Wiki...

NEM No. I don't think it's possible to enumerate all possible complex data structures that one might need. They tend to be specialized. Sure you can do a lot with generic graphs and trees (like the excellent ones in tcllib), but sometimes you just need something custom. (Being able to inherit from a generic implementation would be a big win in this case).

NEM Re-reading, you probably meant "what generic data structures". My answer would be: "as many as possible"! Grab any old data-structures and algorithms text book and implement everything in it ;) It would be good if they were designed with extension and customisation in mind, and in a consistent way (like, say, most OO toolkits would allow you to do).

aa wonders whether MetaKit can be considered to provide the kind of "generic" data structures some people are talking about.

MSW Just to make sure I'm not misunderstood, I'd be all fine with a native OO extension. But obviously there is no perfect one, so I'm just saying that if you want it then use it. Enforcing one style just comes as bad to me, at least I can imagine some people wondering when the first thing you do in your programs is disable and delete half of the host language to load your "nicer" extension. I don't want to treat tcl this way. Please don't make me do that :) [ I'm saying that as I don't expect my fav ending up as default, if there ever was one ]

CMCc I've added a new page Abstract Data Types to hold a discussion about them, and what makes them different from Category Data Structure.


APB 6-Jan-2005

Here are the things holding Tcl back in my mind:

  • Tcl is not a teaching language. People do not get exposure to Tcl in the classroom. This partly goes way back to Richard Stallman's indictment of Tcl but it's primarily because of the fact that Tcl isn't a "pure" language (not a pure functional language, not a pure OO language, etc...) and that it is not derived from C.
  • Tcl is misunderstood. The Perl community have been effective missionaries (zealots) for their language. The perception is that Perl is the most pragmatic and fast of the scripting languages. A Perl user is seen as a rugged individualist with a knack for getting things done. A Tcl user is seen as a misfit with a penchant for the verbose.
  • Tcl is quirky. I have taught Tcl (informally) to many people and I have debugged many of their scripts, none can understand why an unbalanced "#{ blah" would mess up the interpreter. Most complain about the awkwardness of "expr". Many get into quoting hell. All the noise being raised about creating a Tcl/2 is valid -- there are some artifacts that trip people up. Once these folks (all electrical engineers I should add) are bitten, they avoid the language. Perhaps having a better debug facility would help, I'm not certain. The point is that people don't immediately see the beauty and simplicity of Tcl and its initial bumps turn them off.
  • Tcl has too many OO variants. I strongly agree with DKF on this one. I have (at least) three OO packages and I'm not sure what to use, all I'm sure about is that people worldwide are using all three and for what gain? Tcl should embrace one.
  • Tcl needs a primitve dictionary commands/datastore (looks like it's coming dict). I always added this to every program I wrote or I would use TclX key/val routines, but these are both more cumbersome than what ought to be (and have been).
  • Tcl isn't strong interactively. tkcon or something like eltclsh should be more accessible in any given distribution. Giving a new user tclsh interactively is shite -- it needs a readline wrapper (at least) to make someone new happy (1). Again, this comes back to the problem of Tcl's first impressions.
  • Tcl should be easier to debug/profile. Why not release a crappy debugger that can walk through code? Why was profiling (at least up until recently) only available with TclX? Why not have a couple of utilities (I'm not talking about a lot) that make everyone's life easier? (see stepsource.tcl)
  • Tcl is too verbose. I hear that a lot, even though I like it, it does turn off engineers who love the fact that $^ means something special. (2)

Anyhow, those are a few reasons off the top of my head. They are mostly perceptions, but that's mostly what is holding Tcl, rather the adoption of Tcl, back. I know that I work in the EDA world (electronics) and that even though Tcl is the lingua franca of most EDA tools, engineers still wind up using Perl all over the place. I constantly defend Tcl as being a "readable alternative to Perl" and I'm met with stubbornness mostly due to the issues that I have outlined.

Just my 2 cents.

RS (1): On Windows, tclsh has readline functionality built-in - even with history between invocations. On Unixy platforms, this should be fixed indeed.

(2): If you wish, you can name variables and command with special characters, e.g

 set ^ ...; puts ${^}
 interp alias {} ? {} set ::errorInfo; puts [?]

But verbosity often contributes to readability...

DKF: There's been a readline extension around for ages, but it can't go directly in the core because of license issues (which are not the fault of any Tcler). On Win, the readline-like functionality is part of the terminal's standard cooked-mode behaviour.

peterc: Re "... it does turn off engineers who love the fact that $^ means something special", I absolutely love the fact that Tcl never does that. One of the worst things about having to work with Perl after a bit of a break is trying to re-familiarise oneself with those. Ever tried to look up Perl punctuation like "$^" in Google?


SDG: We need to hone our elevator pitch. "Readable alternative to Perl" doesn't excite jaded programmers into opening the book. What does?

How about, "It does everything Perl does, plus:"

  • neat-o VB-style guis for client apps
  • single-file executables, so people can run your apps even if they don't have administrator rights to their desktop
  • an embedded web server, so your app can have a web ui instead of a client gui.

FW: The Perl answer to this would probably be "Perl/Tk, PerlApp and HTTP::Server," but the quality of each of these is all at least slightly lower, since for instance tclhttpd is much richer and more complete and usable than the various server modules available for Perl. Plus Perl doesn't have a JAR-like facility such as TclKits to my knowledge, only more trivial bundling facilities. VK no, perl HAS such ability, called PAR (Perl-ARchive, like JAR), available on CPAN. Perl/Tk, in my opinion, is stagnated version of Tk+Tix bundled into perl, Tcl::Tk has more good moments


Twylite Much of this conversation has been dominated by the question of OO in the core. I think a lot of debate around this topic comes from different intentions for the language. Before asking 'what is holding Tcl back?', we should be asking 'what is Tcl?'

From the comp.lang.tcl FAQ [L2 ]: "Tcl is a simple textual language, intended primarily for issuing commands to interactive programs such as text editors, debuggers, illustrators, and shells ... Tcl is a library package that can be embedded in application programs ... Note that Tcl was designed with the philosophy that one should actually use two or more languages when designing large software systems. One for manipulating complex internal data structures, or where performance is key, and another, such as Tcl, for writing smallish scripts that tie together the other pieces, providing hooks for the user to extend"

This has always been my understanding (and use) of Tcl: a glue language for large systems, and a convenient platform for small utilities. Although some (many?) people are using Tcl alone to develop larger applications, I believe that all functionality being considered for the core must be evaluated against this description of the language's purpose.

I've put my opinions of what is good about Tcl under Things pushing Tcl forward ; it's probably worth reading them before my comments below.

So, what is holding Tcl back? My 2c ...

  • FUD about scripting languages, combined with the marketing efforts of Perl and Python.

Proponents of compiled languages base many of their arguments on twenty year old "facts" about type safety. The truth is that interpreted languages can offer runtime guarantees that are uncommon in compiled languages, and development in dynamic languages is quicker and more reliable. The most in depth study on this matter that I'm aware of is by Prechelt [L3 ] (more detail in [L4 ]).

Python, Perl and Ruby phrase much of their advocacy against other languages rather than for their own. A lot is said about Tcl's performance and lack of thread support. Unfortunately many programmers still consider performance as all-important, irrespective of the real requirements for the application.

  • Lack of Quality Assurance tools.

This is a big one for me, and is IMHO the biggest risk area in adopting Tcl. There is a poor selection of static code checking tools available, and none are maintained as part of the core. I am aware of ActiveState's offerings, but the OSS pickings are slim.

[See Static syntax analysis for a discussion of the currently available tools]

  • Lack of a standard library distribution and integration mechanism.

The largest repository of reusable Tcl packages that I am aware of is the Starkit Distribution Archive [L5 ], but I've encountered a number of people who are unaware that there are easy-to-use packages like this. I was horrified to hear suggestions (even within the last year) that extensions (still) involve downloading the Tcl sources, manually integrating the extension sources, and compiling for your platform (I had to do that about 8 years ago, and it was painful).

I mainly use Tclkit environment, so I am not sure if there are alternatives for packaging libraries for distribution. What I do know is that to ensure widespread and simple reuse, there needs to be a standard for this.

[Of course, several years later the ActiveState teapot repository became available, providing a simple interface to adding new extensions to one's development environment.]

  • Lack of standardised threading support

Tcl's design philosophy rejects threads, but at the same time Tcl is meant to interoperate with other languages. I have had to integrate with a C library that made assumptions about the use of threads (in particular it would use blocking operations without shame). Often this sort of thing is out of the control of the developer who is using Tcl.

I'm not saying that thread support should be in the core; merely that there should be a standardised interface (at Tcl script level) to threads, and ideally that a threads package would be easily included after retrieving it as a standard library package from the Big Tcl Repository :)

I also need to mention that the reason threads are popular is that they lend themselves to developing more readable code. A number of pages on this Wiki discuss appropriate ways to perform lengthy operations without locking the UI or using update, but the changes that are required make the algorithm less obvious (and more prone to errors).

Lars H, 6 July 2005: I'm very much under the impression that the threads package is the standardized threading support, and that it is provided with the core whenever the core is compiled to support threads. Perhaps it's the compiling thing that is the catch, that by default Tcl binary distriutions are compiled without thread support?

JR, 2005-07-06: I wonder if the real complaint is that tcl's standard threading package is a different model of threading than what many people are used to (as in say, java threads) - so-called message passing concurrency vs. shared state concurrency.

[See thread for a standardised interface to threads for Tcl]

  • Modular source code for embedded environments.

I have had occasion to look at porting Tcl to an embedded environment. It was frightening ;) As clear as the Tcl source code is, there was no apparent mechanism to separate out functionality that won't be supported in an embedded environment (like file systems and processes).

Tcl's small size, easy integration with C and powerful command set make it a perfect fit for embedded environments, but I think this is being held back by the difficulty of porting the source code. MicroTcl for Tcl9 looks promising; more info under TinyTCL.

Comments on specific issues raised so far:

  • OO in the core.

First I need to point out that adding OO to the core adds bloat. And I do mean bloat, because it is not near-universally required functionality. If you are using Tcl for its intended purposes you will often not require OO.

Which language has the best OO system? C++ programmers often laugh at Java for its lack of multiple inheritance, while the Java programmers tap their heads at the dangers of multiple inheritance. Sometimes OO syntax makes scripts difficult to read (as can happen in Perl). Most languages used in commercial environments have OO based on inheritance and polymorphism, but in doing so they sacrifice delegation.

Tcl's lack of OO in the core is a strength, not a weakness. It has led to innovation in object systems allowing the developer to choose the most appropriate one for the purpose. IMHO snit is far superior to [incr Tcl] for widgets, for example.

All Tcl's OO systems that are actively maintained should be readily available as packages, in a standard library format. Tcl distributions should be encourages to package multiple OO systems or select the one most appropriate to the environment for which their distribution is intended.

[As of Tcl 8.6, the OO arguments can cease. TWO OO systems will be provided - TclOO as a framework for developing one's own OO ideas, as well as incr tcl. Hopefully the community will cease this opportunity to move Tcl forward.]

  • Database support in the core.

Similar story to OO: it is not a near-universal requirement, it adds bloat, and you will need to pick which databases to support. Should it be MySQL, Postgres, SQL Server, SQLite, Oracle, Interbase, ...?

Again, a standard library format and easily available and integratable libraries will help here. Distributions can do the rest.

[Currently, work towards a standardised database framework, called tdbc, is moving forward. People for whom this is an important issue should join in the develop and testing community.]

  • Tcl is not a teaching language.

I don't see the relevance? Perl, Python, Ruby and PHP are all successful, and are not teaching languages. Java because a teaching language on account of its uptake, not vice versa.

(And Java has given us a legion of programmers who are unable to conceive of not using OO, which gives leaves plenty of opportunities for FUD about Tcl not having OO in the core).

Lars H, 6 July 2005: I know of at least PHP being taught in university classes; probably mostly as a language for generating HTML, but nonetheless taught. I can imagine Perl similarly being taught "because it's part of knowing Unix" or some such.

  • Tcl is quirky.

Too true. Is there a language that isn't? The 'unbalanced brackets in comments' problem is horrible and should be sorted out, but quoting hell is a reality in powerful dynamic languages. In most cases with Tcl you have the option to use alternative constructs to avoid quoting hell (like list instead of just opening quotes or braces).

APB said "The point is that people don't immediately see the beauty and simplicity of Tcl and its initial bumps turn them off." I have to agree with this; but the same is true of many languages. For example, I program in 8 languages, but keep bumping my head every time I try to learn Python because it has some very different concepts.

  • Tcl is too verbose.

Couldn't disagree more (except maybe about OO in the core). It is the readability of Tcl's command set (once you are familiar with the basic, somewhat quirky syntax) that makes Tcl so easy to learn, and easy to maintain. Short-cut symbols that coders love so dearly are what make Perl into 'sentient line noise'.

  • Complex data structures.

According to the purpose of Tcl, that's what other languages are for. Tcl's generic data structures (string, list, array) are incredibly powerful. In my experience people who want more complex structures simply haven't applied their minds to how to encode their data with what Tcl gives them.

Lars H, 6 July 2005: Couldn't agree more on that. - RS, same day, too.


AW Maybe 'Holding back' should be defined a little more clearly...

The one thing I think is holding back more widespread adoption of tcl as a scripting language is that fact that it supports almost nothing 'out of the box'. OO? Well there's (list of 20 OO packages). Ok, but I want OO (and I want to solve my problem, not write Yet Another OO package). So now I have to choose between 20 packages, none of which I really know much about. For my project, I just didn't use OO for that. And the same goes for anything else you want to do... But, on the other hand, the same is true for C/C++ (not the OO bit, but everything else).

A packaging of Tcl with several popular packages (and only one package of each type!) like TclAquaBI does help for this, by selecting one for you. Good introductory docs (i.e. short!) on what each package does and how can also help.

SYStems I would say that most Tcl'ers don't use OO systems, because you can use extend the most popular extensions without having to write OO code. i.e. if Tk depended on itcl for example, you would have seen more itcl code. I think Tk remains the main attraction for Tcl; your best bet to introduce a defacto standard OO system in Tcl is by writing a better Tk in one of them, or if the TCT decide to re-write Tk in one of them, say to allow OO extensions (extending Tk) style for Tk.


LV AW, no language comes with every solution out of the box . Let's see some examples:

Perl
if what you need isn't in the source distribution, you have the CPAN library from which one loads and attempts, often in vain, to build, so as to get additional functionality. And if what you need isn't there, then you are faced with surfing the internet, hoping to find what you need.
Python
if you need something more than vanilla Python, then you can work to configure one of the included extensions, going through the cycle of downloading the various additional libraries needed until you have things built. And, if what you need isn't in the source distribution, then you are faced with surfing the internet, hoping to find what you need.
C
very little comes with a C compiler; if what you need isn't in libc or whatever the compiler of choice calls its default library, then you are faced with surfing the internet, hoping to find what you need.
Java
depending on the distribution, may come with a variety of libraries. If what you need isn't in the distribution you are using, then you are faced with surfing the internet, hoping to find what you need.

And now, let's talk about Tcl:

Tcl
There are 2 configuration time options you might decided you need - threading and memory debugging aid. Other than that, the configure takes care of setting up building and installing all that comes with the Tcl source distribution. And if what you need isn't in the source distribution, then you are faced with surfing the internet, hoping to find what you need.

Gee - I see a pattern here <smile>. Looks like in every language , there are a certain number of functions that come. And in one case, there's an interface to make looking for additional functions easier. But in all cases, you have to go looking, sometimes across the net, until you find what you want.

The primary difference that I see is the number of functions/procedures that come in the source distribution, and, in one case, the organization of a large body of additional functions with an interface to make fetching the source code from that cache of code easier.


I think that TCL's very nature holds it back.

If you look at most languages (Perl, PHP, Java are all examples of this) they clearly are derived from C.

Sarnold agrees : he spent more than 50 hours to *think in Tcl*, after beeing thinking in C/Python/Java a long time.

TCL is not. Linguistically, if I understand correctly, it is most related to LISP.

Lars H suspects Tcl is more related to the Unix shell languages; at least I found I suddenly understood how to *think in Tcl* only after having wrestled with csh a couple of times. And Tcl does right a lot of things that are half-baked in those languages.

EKB I don't know if it was intentional on Ousterhout's part, but Tcl felt very comfortable to me after working with FORTH. They're extensible in a similar way: the command syntax in each language is very uniform, and the programmer can create a new command that acts (almost) like it's part of the core language.

Most people come to languages, in my experience, with a background in C. In that respect, Perl and Java both look more like what they're used to.

Do I think this is a bad thing? No-- I think TCL works wonderfully well. But I think that this is a barrier to entry to people who are using the language for the first time. Am I advocating changing this? Again, no-- there are enough "like C but a scripting language" versions of Perl and its derivatives out there already.


Peter Newman 9 July 2005: The main thing holding Tcl back is academics; the info. science professors and lecturers at the universities who turn 17 year olds into "programmers"? Info. science academics are no different from others of the species. They are (with the sole exception of John Ousterhout,) obsessed with complexity and obscurity.

So academics have grasped C++, Java, and anything object-oriented, with enthusiasm. Simple languages like Tcl - and many of the other scripting languages (other than Perl) - don't get a look in. Because they're what most academics seem to fear and loathe; something simple, understandable and productive.

9 out of 10 programming/info. science students have C++, Java and object-oriented stuff shoved down their throats. It's the way of the future they're told; procedural programming is bug-ridden and unproductive trash.

And when you're 17, you never suspect that the professors and lecturers are a complete bunch of bozos; flogging complex solutions - and impossibly vague - and almost completely non-undertstandable programming concepts - like "code re-use", "polymorphism", "inheritance", blah blah, waffle waffle, crap crap crap, down the student's throats. Simply in order to make it appear as though the academics are the smart ones - as nobody else can understand WTF they're raving on about.

Which is why C++ and Java have 90% of the market. Since these are what 90% of students are taught, it's these tools that they use once they graduate.

Tcl is arguably one of the simplest scripting languages - and consequently is one of the most out of favour. Perl is arguably the most complicated and obscure scripting language. Which is why it's the most popular.

Unfortunately, there's nothing we can do about it. Simplicity and productivity are good. Who cares if the rest of the world is mad. Or if the corporations that hire the university graduates poor literally billions into programming projects that go nowhere (unless of course, you work for one).

EKB -22 July 2005- I deleted the comments I had posted here previously because on reflection they weren't all that enlightening. They're still there in the history if you disagree :-)

NEM As someone currently doing a Computer Science PhD, I have contact with quite a number of academics. At Nottingham, Java is the main programming language taught. Why? I can tell you that it's not because the professors love Java! Quite the reverse: the university has to teach Java (or C# these days) because that is what prospective students want to learn. Funding is tied to the number of students you can coax through the door, so universities are forced to use popular languages. Besides, the languages which are used in commercial projects are certainly not decided on by recent graduates! Other languages are taught. At nottingham, there are courses on Haskell, Prolog, scripting languages (covers Perl, Tcl/Tk and VB last I looked), and others.

Regarding your (Peter's) other comments about professors being "bozos" and loving complex solutions, I find your comments both offensive and inaccurate. If you find "code re-use" a complex concept, then I would suggest that you have little business being a programmer. Many academics spend a great deal of time studying the important problems in language design, and work very hard to provide working solutions to common problems. Sometimes these solutions are indeed difficult to understand, but then software engineering is becoming more ambitious and so the problems are harder. To dismiss the work of academics out of hand is short-sighted and naive. I would suggest that it is this kind of attitude which keeps back the whole field, let alone Tcl.


Peter Newman 9 July 2005: Thanks for your comments guys. But I don't want to get into an argument about these things.

The point I was trying to make is that, on this and many other pages on the Wiki, people ask things like "What's holding Tcl back?" and "How can we make Tcl more popular?". There's nothing wrong with that - but in general, questions like that imply that Tcl's low profile is due to possible defects with the language, or the way it is deveoped, marketed or distibuted, etc.

But that's not (IMHO) the case. Academics love complexity and obscurity. So the popularity of a language in the academic community is directly proportional to its complexity and obscured. So C++ and Java are at the top of the heap, and Tcl is is pretty much near the bottom. Forth, which is even simpler than Tcl, is almost dead.

Since it's the academics who train the programmers, it's they who decide what they shall be taught. So since 90% of students - including not just the programmers - but also the IT types who get into the management positions where they specify the programming languages and tools that the company they work for will be using - have C++ and Java and "object-oriented is good - procedural is bad" thrust down their throats - this is why C++ and Java have roughly 90% of the market. Quite naturally, on graduation, the students and IT managers select and use the tools they've been taught to select and use.

Tcl's low profile then, has nothing to do with Tcl's strengths and weaknesses as a language; it's simply the inevitable result of the academic fascination with complexity and obsurity. And things will get worse; not better for Tcl. Because of the improvements in computer technology, which make even more complex and obscure solutions possible.

Complex, object-oriented solutions will reign supreme, until someone (almost certainly an academic), can come up with an even more complex and obscure programming language.

The only way to make Tcl more popular is to make it much more complex and obscure; the more so, the better. But I doubt many Tcl'ers would want to persue that option - because I'm sure that most of us love Tcl for its simplicity. That's the language's strength. Let's keep it that way.

However, another method of making Tcl more popular would be to start describing Tcl in ways that appeal to the academic fascination with complexity and obscurity. Bjarne Stroustrup - the guy who invented C++ (and, surprise, surprise, an academic) - had great success with this technique. I assume it was he who invented terms like "polymorphism", "classes", "instances", "nodes", "inheritance", "operator overloading", "type safety", "encapsulation", "dynamic binding", "late binding", "early binding", "constructors", "destructors", etc, etc, etc.

If we were to think of similar, impressive sounding, descriptions for Tcl - and then write great long articles about them, that would be the way to attract the attention of the academic community (as Sjoustrup has shown). The fact that it would all be a load of crap and hot-air about nothing, is irrelevant. But those magic catch phrases - and lots of great long articles about what they all mean - is almost certainly an effective way to raise Tcl's profile.

Personally though, I can't be bothered. I like Tcl just the way it is (except I'd prefer it even simpler and more powerful). But the fact is; the simpler a language is; the more obscure it will be. It's tough - because if Tcl were better known; the more work there'd be for Tcl programmers. (However, we're nowhere near as badly off as people, outside of America, who live over large oil deposits. So I'm not complaining.)

NEM Those academic terms are all well-defined and accepted terminology. Many of them do apply to Tcl. They are not meaningless marketing buzz-words. If you do not know what they mean, a quick google or trawl through wikipedia will no doubt enlighten you. By the way, most of the terms you (Peter) mention have been around since before C++. C++ and Java are most definitely not academic languages. If you want a modern academic language, check out Haskell: it'll introduce you to a whole load of new terms, and maybe even broaden your mind at the same time. It is, IMHO, one of the most elegant languages created to date, and genuinely has some tricks that Tcl could benefit from.

As an aside: discussions like this are really much better off on the newsgroup. EKB I agree, so I deleted my comments...

  • Peter Newman ''You miss the point, NEM. It's not whether the terms are meaningful or useful. Only that the language, and discussion of it, is drowned in them. Big, long words and phrases that sound deeply meaningful. It's a pity Bob Dylan isn't a Tcl'er (well, I assume he isn't); he was good at that.

NEM It's not the use of long words (or lack of) that is holding Tcl back. Trying to blame academics for Tcl's (perceived) lack of popularity is absurd.


IL 7-8-2005. Lately I've been of the opinion that TCL needs a facelift. We need to move beyond the high version numbers, and implement T2. Scrap the artifacts, consolidate the command interfaces, make the language consistent within itself, and provide an extensibility format like named parameters etc.

I still think tcl has it over the other languages in terms of its brevity, its gloriously simple syntax; it has so many strengths with regard to its usage. I think if you are forcing people to "think in Tcl" it's done something wrong as an interface, or there's too much clutter.

Spring clean the code, ramp up the advertising, shake off the old, interface with the new, demonstrate hip applications that hook into PDAs, alternative code to Java, Cpp, or Perl, (python even), and work towards exposure. It has to be "cooler". Demonstrate tight integration with new and exciting prospects, reduce the visibility with archaic software that modern developers, the next gen, dont necessarily care about. With every new language I hop to, I appreciate Tcl even more, seeing how easily and gracefully it sails over laborious code structures in other languages. I'd LOVE to see a page with side by side code, entitled "Why you should be Tcl-ish!". I think we all know how impressive such a page could be.

I've completely changed my mind with regard to OO, I think tcl just needs some easy options like named parameters, flags, as a way to add out of the box extensibility. Yeah most long term tcl users can make their own, but why bother, especially when you hope to be able to use someone elses code at some point. I think if people could see beyond the buzzword hell that is OOP they'd give TCL another chance. An xml/dom interface would be nice and increase the versatility of the language; solve it once, solve it well. I've contempted recompiling with a popular parsing mechanism...

[With regards to xml and dom, tdom and tclxml and co-packages are available.]

  • Peter Newman Agreed, that's pretty much the direction to go in.

MG definitely sees virtue in that, but suspects that such extensive changes could mean that the new core would be buggy for quite a while, with so much new or re-written code. I'm not familiar with the Tcl internals, but the benefits of starting largely from scratch could well end up being overshadowed by the problems it would cause.

With regard to OO, I do think one should be added to the core. I don't think -which- one really matters, all that much. Take a poll, or pick one out of a hat ;) With OO in the core, it removes all the problems mentioned above about people having to go looking for extensions to get the basic functionality. If the one included in the core then didn't include all the features someone needed for a particular project (as seems to be the case, at some point or another, with every OO system out there currently), the answer could still be "use X instead of the built-in OO" or "download Y from here", but at least coders would already be into using Tcl, and aware of how simple it is. And it seems most people take one OO system from the start and stick with it, so the chances are whichever was added to the core would serve the vast majority of people for the vast majority of tasks. And no, I've never used any of them, so I may be a little mistaken on some of the details, or over-simplifying, there ;) but that's the impression I've gotten, from reading thousand of lines of discussion on it.

LV Would a language so revamped still be Tcl? - RS doubts it: when everything is a string, you have kind of a classless society... :^)


Lectus Well, to know what is holding Tcl back we need to know what is pushing other languages forward. Let's see:

  • GUI: While Tcl is very known by the Tk extension to be very easy to build GUI's with, other languages like Python also have Tk, and they're moving to WxWidgets. Why? Native widgets! That's right Tk is great because it's easy to write a GUI without the need of a GUI Builder, and it's so integrated into the language. But we need native widgets. Maybe talking more about tile in the media and extending it? Most people think Tk is that old thing with motif-like look. We need to change that.
LV Is it really Native widgets that is driving Python to WxWidgets, or is it widgets which take on the user's selected desktop theme? And, if this is something that developers desire, then why hasn't wxWidgets, which has a Tcl binding for that same widget set, taken off? Also, note that Windows and MacOS have native widgets in the Tk and ttk widgets. What Tk on Windows (and I presume on MacOS) perhaps needs is a set of all widgets possible in Windows (or MacOSX). And of course, Tk on Linux doesn't have native widgets because there isn't a native widget set- there are various native widget sets. I suppose some enterprising young developer out there could create interfaces to make use of GNOME or KDE native widget sets, and somehow attempt to determine when to use said widgets...
  • GUI Builder: While TK is nice and easy, some people like GUI Builders. That's why C# and VB.NET are well known. We need to improve Visual Tcl.
LV Several GUI builders for Tcl/Tk exist, including a very nice one that was spun off from Komodo. What is needed are people who are willing to learn one of applications well enough to provide active development on them.
  • OOP: Lots of people are into Ruby, because of the pure OO nature. We all know Tcl has lots of OO libraries, and so many options. But lots of people won't use a language unless the OO system is highly supported by the language itself. We can do OOP with C, that's right. But is it trivial task? No. That's why people use C++. Any idea on how to fix this without changing Tcl itself? Nearly impossible. Tcl typeless nature doesn't go well with OOP.
LV Certainly there are developers who program in Ruby because of OO, just as there are developers who program in Python because of OO, or Perl because of OO, or C++ because of OO, or Java because of OO, or ... In other words, there are many languages which provide OO. In the majority of these cases, there is a single OO model used. Tcl is one of the few languages of which I am aware where one can develop in a variety of OO models. Some people complain that the large amount of choice is the problem. Pick one. Why worry about the variety? Most of the OO options are not being actively developed. Probably the two major ones which advertise their active development are XOTcl, Snit. Pick one of them. Some people complain that the OO isn't shipped with the Tcl core. Submit a TIP to get it incorporated. [Note that the TIPs were submitted, passed, and have been completed in Tcl 8.6. Time to move on to a new subject!] Some people talk about tcloo, coming in the next version of Tcl. I still am a bit fuzzy as to whether that functionality is going to provide people with the functionality they think of when they use the term OO.
DKF - TclOO will ship as part of Tcl 8.6.
  • Web frameworks: Another reason why everybody is on Ruby. Ruby is only famous because of Rails. A good and popular web framework for Tcl would be nice.
  • Server-side applications: Here Perl and PHP seems to shine. What do we need? More servers running Tcl.
LV What are specific things being thought about here? Something similar to projectforum? AOLserver? Rivet?
  • Real world applications: Lots of big companies are using Python. We also need powerful and good-looking Tcl applications written by big companies. We need a testimonials system at the home page.
This type of thing used to be done at https://www.tcl-lang.org/ , but I've not seen anything recently there.
  • Deployment: Here Tcl shines. No doubt. Starkits/starpacks are really powerful. But we need to show them to the real world. Most people don't know about it.
Having tools and doc gathered together in an easily downloadable form so that one could build and deploy would be a really useful addition. Right now, information is scattered across several web sites concerning how to build a tclkit/starkit/starpack.
  • Teaching programming: The languages more used for teaching are pascal, C and Java. We need to add Tcl to that list, saying that Tcl can be used as a Computer Science playground by being a programable language.
Unless the We in that statement is being addressed to the appropriate administrators at various levels of institutes, there isn't a lot that we can to do add Tcl to the list. What could be done by the community, however, is to join efforts by folks like RS, etc. who have been working on writing material suitable for use in teaching situations. Having materials suitable for teaching students from grade school through university levels, and then tracking down teachers willing to use the materials and provide feedback on how to improve it, would put the community in line to take advantage of the growing desire for open source textbooks.

I could be talking about lots of things here. Really, we can improve Tcl in many ways. But to know what's holding Tcl back we need to look at its history and what today and future languages are doing and try to integrate Tcl in every niche. Enough talking... let's code!

Kevin Walzer I wish more independent developers (shareware developers, micro-ISV's, whatever term you want to use) used Tcl/Tk. I'm baffled that there are no commercial Windows developers using Tcl/Tk. Unlike my platform (Mac OS X), Tcl/Tk applications look 100% native on Windows, especially if you use Tile and extension packages. Why solo developers use VisualBasic and Delphi and .Net when Tcl/Tk is just as powerful is beyond me.

LV I have little experience developing on Windows. But I wonder whether things like VisualBasic's IDE, ability for some of the languages to generate true binary executables, the Microsoft support of some of the languages, etc. factor into why someone might choose something other than Tcl. Plus, I suspect that if you sat down with the languages you mention, and generate a list of capabilities and built-ins that come with the packages, you would see that many times the other languages come with more platform specific functionality than Tcl/Tk provides. This means that the developer has more time to spend solving the problems, as opposed to locating or writing from scratch, basic functionality.

peterc 2008-06-17: Hi Kevin. Crazy, isn't it. I consider Windows to be a major platform for my own Tcl/Tk apps... Not only is Microsoft Windows a significant platform in its own right, but, starpacks for this platform also gives you easy binary platform coverage for all those Unix-like platforms on x86 processors (and a few others) running Wine, plus bonus support for the GPL Windows replacement OS, ReactOS. LV, I really can't think of anything important to me that Windows provides which can't be accessed through Tcl/Tk in some way or other.

DKF: Tcl doesn't access a lot of things directly, which is why there's space for TWAPI to exist on Win (and TclX does a similar role for Unix, as well as a bunch of other stuff). There's nothing wrong in not doing everything so long as the extra bits can be added with package require.

Also check out the functionality in optcl, tcom, eagle, etc. The ability to interact with various OS functions, and to hook into major frameworks such as WSS, MOSS, etc. are major pluses for these packages.


Lectus Another problem I saw when installing Tcl in Linux was that I had to download Tcl, Tk and Tcllib packages. It would be better if it came in one single package like in Python. I know there's ActiveTcl for a complete distribution, but anyway I think it was better if it came that way.

aa See eTcl for exactly the "better way" distribution you're asking for.

Do any of the Linux distributions provide a batteries included type Tcl package that one can load to get a Tcl environment loaded? I certainly understanding having the individual packages available for those who know they only want one piece. But having the ability to do a Debian/etc. package load of TclAndBatteries or whatever, that would list Tcl, Tk, tcllib, tklib, TclX, etc. were all marked as dependencies that should be loaded seems like something that would make life easier for the Linux users who prefer using only managed packages.


GNJ: Tcl's distinctly un-web 2.0 image, and fractured nature of documentation and resources on-line.

CMcC - 2009-07-31 23:55:36

How exactly does a language have or not have a 'Web 2.0 image?' What does this mean?

LV GNJ - what specific functionality do you associate with the term web 2.0. Once that is clarified, perhaps the community can gather pointers for packages which meet the need.

As for the fractured nature of doc and resources, certainly teapot addresses a need for a centralised location for access to actual packages. I would love to see documentation added to teapot's repository, and for said docs to be installed via teacup.

GNJ: Its image is dictated by whats out there on the web, and IMHO most Tcl/Tk related resources look dated from a visual design point of view. I think Tcl has the potential to appeal to a wider audience of designers and developers, Aejaks is an interesting project, starkits on iPhone would be great (dreaming i know, though see www.anscamobile.com using Lua in a similar way, notice sites "web 2.0" look).

CMcC Oh, I agree - it would be great if Tcl was more widely used in modern web applications. To that end, I've written Wub, and given it a lot of support for the jQuery javascript library. I've written some fairly spiffy looking stuff (you know, draggable divs, rounded cornered borders, ajax forms, etc etc.) You won't see it here, because it isn't widely considered consistent with Tcl Wiki's mission, I think. If you want to Web2-ify the wiki, I'm all for it and happy to help.

RLH 2011-03-01: I am not sure how to put this but if you look at any of the "hot" things out there Tcl is not even on the playing field. Web frameworks? Python has them, Ruby has them, Perl has them, but Tcl? That is a hugely visible arena. I would postulate that if Rails hadn't come along Ruby would still be struggling. Tcl has nothing "new and shiny" to bring people to it. I like Tcl so I don't say any of that flippantly.