There are a lot of arguments in favor of keeping as much backward compatibility as possible. This argument trumps a number of discussions the Tcl newsgroup has had over the years. This keeps the language stable but it also stultifies, and makes it much harder to make progress in new directions. When this has happened in other languages, the situation was resolved with a fork of a kind - a second version standard. Modula/2, Oberon/2, and so on provided a relief valve that permitted the language to change and grow but avoiding the need to drag previous users, screaming and kicking, into an incompatible area. I have an ulterior motive for this discussion, of course. With the renewed interest in [weedesk], and the tcl plugin, there is a very real opportunity here to reach a new and very wide audience, few of whom have had any real exposure to programming before. [weedesk] will obviously use Tcl for scripting and extending its capabilities. I would like to be able to expose Tcl in all its glory in this sort of application - but to do that we need to be able to explain it. The recurring discussions and questions we have dealt with in the wiki and in the newsgroup are guides to what people will have trouble with when they are exposed to Tcl for the first time. I would very much like to see these things addressed in as simple and "bulletproof" a manner as possible. This page is to discuss what we'd like to do to Tcl if full backward compatibility was not a factor. The primary objective is to reduce the overall conceptual load of the language (the stuff you need to remember to code efficiently) while increasing its overall power. The bottom line goal: to make Tcl the most powerful and simplest language it can be. I'd like to focus on several areas: * Regularizing the grammar (e.g., "list length" and "list create" instead of "llength" and "list"). * Regularizing object concepts: [cfile] rather than "file", to make the usage of object-like commands match Tk. * Removing hideous warts: the :: is the worst offender in this respect although . from Tk is another example -- since Tcl already has the basic idea of a list built-in, the idea of listing a series of nesting things in order from some common ancestor (global namespace, top window, whatever) should use that same syntax rather than inventing another. Rather than set ::a::b::c foo, set {a b c} foo. Rather than .top.menu.file, it should be { top menu file }. It has been my experience that this item alone confuses more new users than any of the basic Tcl concepts. ::a::b::c looks like C++, but .a.b, which also looks like C++ or C has a completely different meaning. The need to parse and maintain these constructs also adds needless bloat to the language. * Conform arrays and dicts: the distinction between arrays and dicts makes sense to programmers who have used Tcl for ages but they are new landmine for new users. Ideally, these should not be separate, since they are the same concept in two different syntaxes. It might be argued that "dict" is the right way to go and that "array" as such simply dropped. The reason for this will become obvious in the "expr" section following. * expr is a pain in the butt. It was painful enough in its first incarnation as an expression evaluator since it is way verbose and it and its associated [[ ]] obfuscate code. However, since adding bytecode compiling to the language the need for adding yet another layer of { } not only obfuscate further, they introduce a new landmine for newcomers: a { } expression that manifestly ''does'' do $-replacement! Furthermore, whenever expr ''does'' encounter an unknown identifier, it doesn't need the $ to know it needs to look it up. Similarly, the fact that an identifier is suffixed with "(" should be clue enough for expr to look up a tcl proc if the identifier is not in its internal function table. If we get rid of arrays, it becomes simple to use ( ) to delineate things to be evaluated with expr. Thus, something like: "string range $foo [[expr $index-1]] [[expr $index+1]]" becomes just "string range $foo (index-1) (index+1)" - ''much'' easier to read. * Patterns are a similar pain - the overloaded use of both ( ) and [[ ]] make it very difficult to write patterns properly and to read them once you have written them. Ideally some method of delineating patterns - perhaps < > - and the use of @ rather than $ (or some similar choice) for eol, would enable us to embed patterns much more gracefully than we now do. * Macros. It should be possible to define a "macro" - similar to a "proc" - that gets a shot at a command ''before'' it is byte-compiled. These would provide ways to, for example, allow a command kicked off to "unknown" to be resolved and byte-compiled without further performance penalty. * Make the language friendlier. The "set" command is a particular offender here, it is more verbose and different from the "normal" way of doing assignments without offering any great improvement in utility. I offer two possible improvements here: [let] or ->. -> works just like ";" except it notifies the parser that the value of the command it just evaluated should be copied into the variable name that follows. Thus, [[string range $foo (i-1) (i+1)]] -> substring. * And add ''real'' comments - I'm tired of the old "why can't I have unbalanced [[ ]] in comments?" questions. It really isn't that hard to have the parser eliminate "//.*$". I'm sure more ideas in this same vein have occurred to others. I'd like to explore these ideas with an eye toward producing a successor to Tcl that would vastly increase its power, readability, friendliess, and utility. What are your ideas? --------- [XX] From the Marketing Division of the TCL Branch of the Sirrius Cybernetics Corporation: The case for both sides is so compelling that the *ideal* answer for TCL/2 or TCL9 or whatever is to grab the best of both worlds and make it a feature of the language. Some rough possible ideas from the perspective of good marketing possibities follows. keep in mind that while I've tried to leave out the outlandish, I do not intend to make any suggestions on the feasibility of implementation or even technological superiority (yet, anyway). Just trying to get the ball rolling... So if I had to "sell" TCL/2 or TCL9, I'd want to be able to say that the next generation of TCL is unconstrained by previous code and/or design decisions (mistakes?) yet compatibility is close to 100%. How can this be accomplished, bearing in mind some of the strengths of TCL to begin with: 1. Compatibility engines as loadable extensions - the idea being that a programmer can add one line to an old program and have it work with TCL/2 TCL9. Of course, this compatibility can only be guaranteed for the core and for TCL code and not binary extensions. "Emulation libraries" could be provided that would allow TCL/9 to fall back to the language implentation represented by that particular library. One attractive feature about this model is that there is no core bloat for compatibility. One downside is that (presumably) binary extensions would need to be distributed with the core. Key versions would need support in the core. What would be really neat would be a "generator" that could take an existing TCL or TK library and synthesize a compatibility module "on-the-fly" (using Critcl type technolgy, perhaps). Then we could say: "TCL9 supports itself + any version of TCL and TK you have installed on your system or deployment" 2. Compatibility "switch" as part of invoking syntax for an interp or new command. The difference here is that the functionality of previous versions would be "encapsulated" in the core. One could activate such support by invoking a command during execution. This might bloat the core sizewise (though I have NO idea how much) but shouldn't cost much performancewise, and should cost nothing if not invoked. I encourage those that know the mechanics of these things to show me up and come up with better implementation details. My point is that while there are downsides to both breaking old code *and* keeping compatibility syntax; TCL should innovate and do neither. Make any syntax changes necessary in the new language definition, and offer optional compatibility to older software and site users. [PWQ] ''6 Jan 05'', I too wonder why the [TCT] have never discussed adding compatibily options to the interp. Most other languages have this feature. IT would not be much of a ask to type: tclsh -compat 8.0 my_old_script [LV] I don't recall anyone ever submitting a [TIP] for such a feature, with the intent on implementing it, to the TCT for discussion. It would as always require changes to the core, including internal dispatching through the stubs table, but it allows changes to proceed unimpeded. [RS] I think it would be more robust to specify such dependiencies in the script itself: package require -exact Tcl 8.0 [DKF]: It's been discussed, though perhaps not recently. [DGP] is the person to chat to about this; he has a pretty deep understanding of versioning complexities. ----- One other quick suggestion would be the deployment with the new core of a sort of Upgrade Assistant for old code. Since things like this *never* work 100% and very often not without some tweaking of the new code, calling it a translator would be foolish. The "Assistant" would take an older TCL script, and make several levels of suggestions for code changes: 1. TCL9 Syntax Compliance 2. Obsolete command replacement 3. TCL9 Technology advances Thus, the programmer can obtain suggestions for at the very least running his script in a TCL9 interpreter without compatibility options. More experimental suggestions could also be made, encouraging the programmer to use new TCL9 commands and constructs. Either way, a potential win for both the developer and TCL9. ---- This page is to discuss what we'd like to do to Tcl if full backward compatibility was not a factor. [XX] I wonder, then, if i should move my suggestions above to another page, possibly the TCL9 wishlist. I entered them here because it was the original comments about the situation being resolved with a "fork of a kind." I agree with the principle of keeping the best, fixing the worst, and refining the implementation of the language into an elegant implementation of the "TCL idea" without being bound by design decisions of the previous generation. I view my suggestions as a way to make possible what is actually being discussed on this page without compromising the smooth transition path and reliable operation that *needs* to be in effect. So in my opinion, it is relevent to this page, though I am probably biased so I'll leave this bit of housekeeping to the judgement of others. ---- [DKF]: I find it fascinating that someone was asking for a regularized grammar in one breath, and asking for an infix assignment operator in the next. Fascinating, but not compatible. [Larry Smith]: -> is not an infix operator, it is a command separator - exactly like ";" as far as the parser goes. When it is encountered, the interpreter does its "thing" for that command the way it would for ";" but when it is done it scans past the -> to find a (or several?) variable names and assigns the result it is holding in hand to it (them). Going over some of the points in more detail: * "lindex" vs "list index": I agree and wish it could be fixed without massive incompatabilities. :^( * [cfile]: Are you aware of TIP#208? [Larry Smith] I am now. I like it. * The widget pathname "wart": This wart is useful to me ;^) so expect it to be useful to many others too. Namespace paths are more warty though. * Arrays vs dicts: I know, and I'd love to do something on this. No guarantee for 8.5 though. * Expr and REs: You're ''way'' wrong here. These are embedded little languages, and hence building on one of Tcl's key strengths. And $ as EOL-constraint is holy for REs. [Larry Smith] I understand the concept of embedded little languages, it is the means for accessing the one for computation that is irritating. It is far too common to make people do it the way it is done now. And as for the holy $-sign - "$" means "substitute". Using it to also mean "eol" - even in a little embedded language - makes it a wart. A painful wart. * Macros: You can do it now. It's a touch complex, but the code to do it (written by [SS]) is on the wiki somewhere. * Infix '''->''' for assignment: How is the Tcl interpreter to know that a particular -> argument is an assignment operator and not some random string? [Larry Smith] The same way it knows ";" is a command terminator. (And [let] is a very complex command indeed; it does all sorts of things in different circumstances. [Larry Smith] [let]'s exact syntax is open for discussion - yes, the example implementation is complex, but can still compile down to bytecodes. If any of the above ideas is accepted then [let] can become simpler - using ( ) for expressions means let really doesn't need the "arithmetic assign", and the "evaluation assign" := is a wart of a sort (cohort, cohort =) all by itself. The basic objective here is to make assignments more concise.) * "Real" comments: This can be done right now using a customized [source] command, and the problem with '''}''' characters is simply because they are in a quoted context. You might as well ask why you can't have comments in C strings... ---- [Peter Newman] 6 January 2005: Interesting DKF. I also take the view that for Tcl to move forward, we've got to start again. My basic view on how Tcl should be structured is on [If I were to complain] - and your response to that was bang on the money. The interconnections are the problem. But they're not a problem if we start of the assumption that backwards compatability is not required. Take '''set''', which you complained about above. It's surely a good example of (the evil) interconnectivity. Because one can ''set'' global variables, local variables, lists, arrays and god knows what else. So obviously ''set'' has to ''know'' about all these different data types. But in my vision, every data type, and every command (or set of commands) should be a completely separate, stand-alone entity - which the script level programmer can include or exclude from their particular version of the language, as they see fit. But if we throw away the requirement for backward compatability - we can easily eliminate ''set''s interconnection problems. Taking a hint from Tk:- frame .myFrame .myFrame configure .myFrame cget etc etc we could change ''set'' to a method. In other words:- Global myGlobalVarName Local myLocalVarName List myListVarName Array myArrayVarName (to define the variables required) - and then:- myGlobalVarName set # To get the variables existing value myLocalVarName set myListVarName set myArrayVarName set myGlobalVarName set newValue # To set the new value myLocalVarName set newValue myListVarName set newValue myArrayVarName set newValue In other words, we treat data types exactly the same as Tk treats widgets. And set becomes just a method supported by any data type that chooses to support it. There's no more interconnectivity. It's also a technique that Tcl/Tk programmers are familar with (not to forget those familar with the more modern object-oriented languages). And it hopefully eliminates the clumsiness of ''set'', that you mentioned above. Another great thing about it, is that we could easily implement the syntax - and thus the new ''Global'', ''Local'', ''List'' and ''Array'' etc data types that might appear in our new Tcl/2 - using current high-level Tcl. Let's code it up in high-level Tcl (to test and refine the syntax). And make each data type a separate stand-alone '''package'''. And IMHO, we could very quickly and easily sort out ''lists'', ''strings'', ''files'', and the other basic core components of Tcl, in similar fashion. Then it's just a case of:- 1) Sorting out the interpreter; 2) Re-coding the above in C, and then; 3) Ditching everything else (the traditional Tcl bits we don't want, and don't convert over). Then we have '''Tcl/2'''. I like the name too. Cheers. Note also that the syntax suggested above is compatable with the current Tcl syntax. More specifically, both could exist in the same (existing) interpreter with no conflict. Making it easier for people to migrate from Tcl to Tcl/2. [Larry Smith] It took me a bit to understand the direction this idea was proposing, but I think I like it. It does, in fact, regularize the language, which I think is important here. See the 3rd paragraph above for more on why I am pushing this. [RS]: Note however that global or local variables can be arrays or lists, and lists can be reinterpreted as strings and vice versa... My hopes for a newer Tcl would be that it's more systematic, and more powerful, but not more complicated than what we have now :) [Larry Smith] And that is precisely the objective. And I'm prepared to turn any sacred cows (even my own) to ground chuck to accomplish it. ;) [DKF]: Oh well, be sure to let us know once you've got the first prototype going... [Zarutian] 06. jan 2005: I like your idea, Peter, regarding treating variables as objects/procs as it eliminates the need for the "trace * variable" command. And arrays can be easily dropped by useing dictionaries instead. But what about useing the Lisp way of doing artimetric, that would make implementions of bignum, matrixes and other such easier, no? [Larry Smith] It is certainly more consistant, the problem is it is too alien to new users, especially those without prior programming experience. I would personally find rpn to be a better choice, too, but I rejected it out of hand for the same reason. We are, more or less, stuck with "normal"-looking arithmetic expressions. We must make the best of it without letting the seams show too much. ---- [Category Discussion]