AMG: Executing a script in a different stack frame means only that the script is given local access to that stack frame's variables, plus the attendant change to [info level]. It does not "make the caller do something." I know of only one circumstance where this distinction matters, but it's critical. uplevel return is the same as return with no uplevel; uplevel cannot be used to terminate the caller. (Instead use return's -level option.)

---

When to use [uplevel] edit

AMG: [uplevel] provides access to another stack frame's local variables. In most cases, [upvar] is preferred over [uplevel]: it's simpler and it doesn't interfere with bytecode compilation. However, if the script to be executed comes from elsewhere (e.g. it was passed as an argument), [uplevel] is required.

As DKF points out below, [tailcall] may be a superior alternative, when the level count is 1 (default), and the [uplevel]'ed code is the last thing done by the proc. [tailcall]'s argument is a command, which is different than a script. If you want to execute a script, I suggest using single-argument [try] as a shim. [eval] would also work, but it prevents bytecode compilation [1].

---

Harald Kirsch wrote in c.l.t:

Question: Is it possible to have [macro] as a counterpart to proc such that [macro] defines a "function" which is evaluated in the caller's stack context, i.e., which does not create its own stack context?

So how's this:

 proc macro {name body} {
        proc $name {} [list uplevel 1 $body]
        set name ;# not needed, but maybe helpful
 } ;# RS
 0 % macro now {incr i; incr j}
 now
 0 % set j 10; set i 1
 1
 0 % now
 11

As a variation, you might also have the body executed in global scope, if you want to avoid global commands (not recommended normally, since the variables you use are not cleaned up):

 proc Sub {name body} {
    proc $name {} [list uplevel #0 $body]
 }
 % Sub nenv {array size env}
 % nenv
 44

---

DGP Note that [macro] will not be able to handle a body that includes a return command, due to some fundamental limitations in Tcl itself. See tcllib::control documentation for some more details.

Tcl chatroom transcript:

suchenwi: I'm not sure what you mean with "will not be able to handle a body that includes a return command." If there is one, it will cause the caller to return, I'd think. And that's like "#define BYE return;" in C - expected behavior for a macro.

dgp: No, it won't. Try it.

suchenwi: But Harald Kirsch already followed up, he wants macro to take arguments and have local variables. This is tricky indeed - you can't do a "downlevel"....

dgp: Important to remember that uplevel adjusts only the command and variable substitution context. It can't really do anything to the calling stack.

suchenwi: Ah - I see. Tried "macro foo {incr i; incr j;return;puts "oops"}".

dgp: That doesn't test what needs testing.

suchenwi: foo itself returns (the oops won't come), but does not terminate its caller. One would have to do

 set body [string map {return {return -code return}} $body] ;-)

dgp:

 macro mreturn return
 proc test {} {puts a; mreturn; puts b}
 test

suchenwi:

 proc macro {name body} {
    proc $name {} [list uplevel 1 [string map {
             return {return -code return}
    } $body]]}

dgp: Yes, that string map will handle more cases, but can't achieve [return -code error], etc. One way to fix this limitation would be to add a new status code to Tcl that caused the result to be evaluated in the caller's context by the caller.

 return -code eval {set var val}

Then you would just map all [return $arg $arg ...] to return -code eval {return $arg $arg ...} . Not completely sure what cans of worms get opened by that new feature though. Of course, with that status code available, [macro] would be much easier to write too.

kennykb: I like [return -code eval] -- except for deciding whether Tcl_EvalObj and its friends should interpret it or return it to the C caller.

rmax: Hi Kevin. I think [return -code eval] would also help [do] to become complete for all possible cases.

kennykb: Yes, dgp's comments make it obvious that it could do return -code eval { return -code eval { return -code whatever {... }}} to break out of multiple scopes.

dgp: I still would not recommend that though. Just as it should always be [uplevel 1 ...].

kennykb: Don: I wouldn't recommend it either, but a certain depth of nesting is needed for [control::do] to handle [return -code] within the loop body, right?

rmax: You won't of course create nested [return -code eval] constructs yourself, but they could emerge from nested [do] loops or macro calls.

dgp: I think catch needs extending too.

 catch {return -code error foo} msg --> 2
 set msg --> foo

But where is the '1'? Add another argument to catch to get the '1'.

kennykb: Yes, also need a place to stash the -errorcode and -errorInfo options to [return].

dgp: I think you can just grab them from ::errorInfo and ::errorCode directly.

dgp: ... but yes, status code '1' needs special handling, as always, to deal with ::errorInfo and ::errorCode. For the rest, set code {catch $body msg}

 if {$code == 2} {
 # We caught a return

D'oh! Make that

 set code {catch $body msg returncode}

then...

 return -code eval [list return -code $returncode $msg]
 }

dgp: Ah! I think I see what you were getting at!

kennykb: Hmmmm, it occurs to me that [return -code eval] subsumes all the others.

dgp: If $body itself contains [return -code eval $script].

kennykb: return -code error -errorcode code -errorinfo into message => return -code eval [list error message info code]

return -code break => return -code eval break

kennykb: ... and the same for continue and return.

suchenwi: Looks like it would actually make the language smaller....

dgp: ...except maybe for the differences in ::errorInfo construction.

kennykb: We'd have to keep the other syntax on return for back compatibility, but I think it could simplify the mechanism. Oh, yeah, ::errorInfo construction. OK, make it return -code eval [list error $message [doTheRightThingWith $info] $code].

suchenwi: ..and if eval is the last remaining value for -code, one could shorten that to -eval...

kennykb: Hmm, need a new command for those crazy enough to do [return -code 6].

dgp: How about "-code ok" ? I guess that's the default case where no eval script is done. Note: that's different from eval of an empty script.

Why a new command?

kennykb: I suppose if you wanted to be obsessive about it, you could say return -code ok $value -> return -code eval [list identity $value] where identity is a command that returns its arg, but I wouldn't go that far, myself.

dgp: We still need [return -code 6] if we want ability to create new status codes at the script level.

rmax: Isn't this possible to handle completely on the catch side? Doing it on the return side involves string operations on scripts, which hurts performance.

dgp: proc niftyNewBreak {} {return -code 6}

kennykb: proc niftyNewBreak {} { return -code eval {::tcl::raiseException 6 }} ? 8^)

rmax: ... and there is the danger of substituting a "return" that isn't really a return.

dgp: Ah, I see.

kennykb: Where are we doing string operations on scripts?

dgp: No substitution. catch and re-raise.

rmax: The implementation of [do] would have to substitute "return -code ..." by "return -code eval {return -code ...}" or am I wrong?

kennykb: Reinhard: Ah, but there's no string processing there. The new command is a pure list.

dgp: [do] includes the line:

 set code [[catch { uplevel 1 $body } result]]

Change to:

 set code [catch { uplevel 1 $body } result returnCode]

Add a case to the switch

 2 {return -code eval [list return -code $returnCode $result]}

kennykb: Yes.. and note that the thing to be 'eval'ed is a pure list.

dgp: The use of eval might prevent bytecode, so that's a performance issue, but there should be no need to reparse the strings.

rmax: wouldn't "2 {return -code $returnCode $result}" be sufficient?

dgp: No. The idea is that the return in $body should act as if it was evaluated in the caller. Hmmm. so far have taken care of the first pass through [do]. Remaining passes are handled by an [uplevel 1 {while ...}] That probably needs similar handling. Yes. Same changes there.

rmax: I still think it would be sufficient for [do] to be able to catch and pass on the -code argument to return.

kennykb: Gentleb[e]ings, I think we're excising one of the language's major warts here.

rmax: Can you give an example of usage for [do] where that wouldn't be enough?

* clock : 1011891602 : Jan 24,2002 Thursday 5pm GMT

kennykb: What if the body of the [do] contains [return -code eval] ?

dgp: I think that case is handled above.

rmax: do will pass on [return -code eval].

dgp: rmax: you don't want to return from [do], you want to return from the caller of [do].

rmax: Yes:

 do {
    return -code return
 }
 inside do:
 set code [catch { uplevel 1 $body } result retCode]
 ...
 2 { return -code [list return $retCode] $result }

What about that?

dgp: That will return from the caller of [do] when what is intended is to return from the caller of the caller of [do].

* rmax is totally confused now... has to try some things...

kennykb: That's yet another new syntax for return. How is that preferable to "return -code eval [list return -code $retCode $result]" ?

dgp: Oh. I mis-read part of that last one. Given what we've described, that should be an error:

rmax: it is a bit shorter, and I haven't thought much about it...

dgp: unknown code: "return 2"

kennykb: Since we need [return -code eval] anyway -- in order to implement [macro], I'm reluctant to invent still more syntax just to make [do] a tiny bit shorter.

rmax: Yep, Don. It implied another proposal of changing the syntax of return. OK, Kevin, you win.

dgp: The numeric code for eval should be -1.

kennykb: Why -1?

dgp: #define TCL_EVAL -1

dgp: It's an interesting status code in that it is completely internal. No command will actually return it, it will return the result of eval'ing the result instead. [return -code $integer] has been documented to allow any postive integer. -1 won't interfere with previous extended status codes.

kennykb: OK, we have a compatibility issue with C code that calls commandProc's directly. (There are examples doing stuff like that in Brent's book, IIRC.)

dgp: hmmmm... take that back. I guess "positive" isn't in the docs after all. Are there any example doing that with *ObjCmd's?

kennykb: I don't think we have too much of a problem as long as we avoid whatever code exp_continue uses.

dgp: Any existing code is calling existing command procedures. None of which return the status code TCL_EVAL, so is there really a problem?

bbh: exp_continue seems to be -101

kennykb: Don: Doesn't Brent's code have a Tcl_GetCommandInfo so that it's actually calling a user-supplied command proc?

dgp: Anyhow look for a section "Bypassing Tcl_Eval". His code doesn't handle any extended return codes, so there's no new problem here. His routine is called Tcl_Invoke. Tcl_Invoke("break", NULL); would cause problems now.

* suchenwi has to stop mirroring this debate on http://wiki.tcl.tk/1507 - real work calling...

---

TIP #90 [2] is a formalized follow-up of the above discussion. ...and is now incorporated in Tcl 8.5 development.

17 October 2002: returneval provides a Tcl-only implementation of something very similar to the above [return -code eval].

RHS TIP 90 doesn't cover the -code eval idea, does it? I read through it and it didn't seem to.

Lars H: No, but -code eval was considered as an alternative solution. Maybe it's not in the TIP itself, but in some of the surrounding discussion, however a good deal of thought was put into it.

---

AMG: [uplevel 0 $script] is the same as [eval $script].

---

Possible [uplevel] deficiencies edit

AMG: Here's a nice, simple C-style do ... while loop implemented using [uplevel]. Other implementations are available on the New Control Structures page.

proc do {body while condition} {
    if {$while ne "while"} {
        error "invalid argument \"$while\": must be while"
    }
    uplevel 1 $body
    uplevel 1 [list while $condition $body]
}

I'm curious about the bytecode compilation properties of this control structure. I don't think $condition and $body ever get compiled. Right? If so, is there any way to fix this? Putting the code to be executed into a randomly named proc would enable bytecode compilation, but running the code in a new stack frame defeats the whole purpose of uplevel!

So, does there exist a Tcl_Obj type for bytecode-compiled script snippets which are not given their own stack frame?

I have another question. Can [do] be enhanced such that [return] inside the [do] body (or condition!) will cause [do]'s caller to return? Incrementing return's -level option works, but this is not a change to [do], it is a change to [do]'s arguments.

proc test1 {} {
    do {return success} while {1}
    return failure
}
proc test2 {} {
    do {return -level 2 success} while {1}
    return failure
}
test1 ;# returns failure
test2 ;# returns success

AMG: These are still burning questions... Anyone have answers?

steveb Here is how I do this with Tcl8.5+ and Jim Tcl. Needs some small tweaks if you want to support break and continue properly.

proc do {body while condition} {
    if {$while ne "while"} {
        error "invalid argument \"$while\": must be while"
    }
    set rc [catch {
        uplevel 1 $body
        uplevel 1 [list while $condition $body]
    } msg opts]
    if {$rc} {
            dict incr opts -level
    }
    return {*}$opts $msg
}

DKF: With 8.6 (specifically, tailcall) you can do this:

proc do {body while condition} {
    if {$while ne "while"} {
        error "invalid argument \"$while\": must be while"
    }
    tailcall while 1 $body\n[list if $condition {} else break]
}

AMG: Neat! These both work as-is in Tcl, and they pass the test1/test2 test case I posted above. My remaining question is: how can I ensure that the code is bytecoded? With DKF's approach it might happen, but the bytecodes will be regenerated each time [do] is called.

I need this for [icc catch], currently implemented this way:

proc ::wibble::icc::catch {script} {
    try {
        uplevel 1 $script
    } on 7 events {
        return $events
    }
    return
}

I should be able to change it thusly:

proc ::wibble::icc::catch {script} {
    tailcall try $script on 7 events {set events} on ok "" {list}
}

And hey! This gets bytecoded!

set script {puts before; puts >[wibble::icc get myfeed foo]<; puts after}
tcl::unsupported::representation $script

source wibble.tcl
wibble::icc configure myfeed accept foo
wibble::icc put myfeed foo
wibble::icc put myfeed exception
wibble::icc catch $script

tcl::unsupported::representation $script

The first [tcl::unsupported::representation] says that $script is a pure string; the second says that it's a bytecode. Success!

The script prints "before", then the [wibble::icc catch] returns "foo exception". There are no reverse angle brackets around it, nor is "after" printed, therefore the script gets interrupted inside [wibble::icc get], as expected.

Thanks, guys! The trick was to avoid [uplevel]. ;^)