Tcl9 internal changes

MS every now and then I trip on some things in the core that I'd like to change and can't. This page is meant to keep a list, additions welcome.

The first batch concerns command invocation:

  • get rid of TCL_ALLOW_EXCEPTIONS: the library shouldn't care about that and always return the actual code; tclsh's main can do whatever it needs, of course. This functionality seems to have fallen on the wrong side of the border.

Lars H: Is the Tcl9-ishness of this purely a matter for extensions calling the Tcl_Eval* functions, or are there also script level implications? (I've noticed that the interp bgerror mechanisms are prepared to handle arbitrary exceptions, but since asynchronous scripts tend to be evaluated without TCL_ALLOW_EXCEPTIONS, the exceptions never get as far as ::tcl::Bgerror — something I'd like to see changed soon.)

  • make explicit the difference between "namespace where the command is looked up" and "namespace where the command is invoked"; separate command lookup from invocation
  • let procs be "named lambdas"; let the context of the proc body NOT be the namespace where the proc's command resides (ie, no changes on renaming to a different namespace)
  • of course .... iPtr->result should die!
  • stop promising that a command receives an empty unshared result in the interpreter: a command has to set its result from scratch, reset it or leave it untouched. This has performance implications, but also enables simpler shell-style pipelining (without nesting [...]). Maybe reserve a variable or have a command to get the interp's result.
  • Tcl_[GS]etCommandInfo and direct calling of commands!! At least rethink thoroughly

Other things

  • hide more implementation details behind opaque pointers: interp, CallFrame (don't let anybody else alloc them, they do not need to know the size nor how the core allocates or frees them), ...
  • change Rule [1] in Tcl.n: let the command be identified after substituting the first word but before substituting the rest. This allows completely bugfree compilation

Do you mean

  proc foo {x y} {expr {2*$x}}
  foo 3 [proc foo {x y} {return 2$x}]

should return 6 rather than 23 as it currently does?

Precisely! Maybe a better illustration: look at this currently unsolvable (at reasonable cost) bug #1812624 [L1 ]

mig@oli:~$ tclsh
% set x 1[proc set args {puts AHA!}]
1
mig@oli:~$ tclsh
% eval {set x 1[proc set args {puts AHA!}]}
AHA!

(Note that that's 8.5, 8.6 may require a different example to show the effect)

  • change Rule [1] in Tcl.n: let the first word be not a command but a command prefix (aka autoexpand first word). Note that this makes it difficult to deal with commands with names that are not proper lists (eg foo{bar}) or that have spaces: they have to be [list]-protected

ferrieux If you allow my $.02, an old dream of mine: ease polymorphism of Tcl_ObjType's. Currently plenty of code checks pointer equality with &someObjType: that's a braindead version of OO programming; this is all the more surprising that the spirit of the implementation is otherwise adequately OO (function pointers). One corollary would be to rip off the List object type's special status (under performance constraints though), so that it becomes easily sub-classable. Maybe a more important one: make it possible for an extension to subclass predefined types.

MS Ah yes ... that reminds me:

  • change the name Tcl_Obj to Tcl_Value (and drop the old one); or Tcl_Thing ... or anything that does not suggest OO. NEM: Tcl_Term?
  • along these lines: review Tcl_ObjType to (a) maybe remove some things, (b) allow a 'toAny' field (which will be tried first before attempting a shimmer via the stringRep). The idea being that some extensions may have an optimized way to (say) convert to listType without passing through the stringRep
  • rethink memory allocation api: we may want to pass an interp in, and get a result back? Anyway: some way to avoid automatic panic on allocation failure, often a simple error would have been enough (sorry Dave, can't create that list for you). Of course, plain malloc and a check for NULL would do too ... Lars H: Isn't there already Tcl_AttemptAlloc? But yes, this would be good.
DKF: The current memory allocation strategy is something I actually like. The attemptalloc stuff is actually what you get with normal malloc, but its name reminds people that it can fail. However, there is no point in failing gracefully if you have no way to recover from the failure; if a small alloc fails, you probably don't have enough memory left to recover and work out how to get back in business. This is why dealing with memory failures is hard. OTOH, the failure policy was set at least as far back as Tcl 7.3, and modern Tcl (while more memory hungry overall) does not churn quite so badly and can hence stand a chance to recover.
  • How could I forget? Fix the varName resolution mess ...

(TBC)