yieldTo

yieldto, a built-in Tcl command, yields, replacing the current evaluation stack with the evaluation of another command.

See Also

ycl parse tcl stream, by PYK
A Tcl script parser in which components of the parser yield to each other as they read their way through a script. In order to maintain a single point of contact for the client, each component is renamed to the name of the main coroutine when it becomes active.

Documentation

official reference

Synopsis

yieldto command ?arg …?

Exceptions

TCL COROUTINE ILLEGAL_YIELD
If called from an illegal context (i.e., not in a coroutine).
TCL COROUTINE CANT_YIELD
There is a coroutine context, but a non-NRE-aware C command is gumming up the works.

Description

yieldto has the same relation to yield that tailcall has to return. The following two commands are equivalent in that they yield the same value to the same caller, but different in that yield causes the coroutine to accept one argument the next time it is called, and yieldto causes the coroutine to accept multiple arguments the next time it is called, and returns a list of those arguments:

yieldto string cat $result
yield $result

Whereas tailcall replaces the current level with the evaluation of a new command, yieldto replaces the entire execution chain of the current coroutine context, yielding at the same time so that the coroutine context may subsequently be re-entered. Since the execution of command replaces the current execution chain, the caller of command is effectively whatever entered the replaced coroutine context.

Command is resolved in the context of the current coroutine, and often refers to another coroutine, although it may refer to any command.

yieldto may only be called from within a coroutine context. It is an error to call it from elsewhere.

Example: Multiple Arguments

proc accumulate op {
    set agreduce 0
    while 1 {
        set agreduce [$op $agreduce {*}[yieldto lindex $agreduce]]
    }
}
% coroutine cumsum accumulate ::tcl::mathop::+
0
% cumsum 1 2 3
6
% cumsum 7 8 9
30

Example: the Caller of command is Whatever Entered the Coroutine Context

In this example, p1 is the effective caller of return, and the result of p1 is intermission. The coroutine is suspended until it is entered again:

proc p1 {} {
    set res {one two three}
    coroutine c1 apply {{} {
        yield
        yieldto return intermission 
        return {the end} 
    }}
    c1
    puts {execution never makes it to here}
}
% p1
intermission
% c1
the end

Example: uplevel

yieldto can provide the same functionality for a coroutine that uplevel provides for a procedure. In order to do so, arrangements need to be made to call the coroutine back once the script has been called in the other level. Here is an example of a coroutine that implements the same functionality as incr:

coroutine coroincr apply {{} {
    set res {}
    while 1 {
        set name [yield $res] 
        set res [yieldto try "[list [info coroutine]] \[::incr [list $name]]"]
    }
}}

If error handling can be left to the script being evaluated, the coroutine can simply pause until the next time next time script evaluation is desired, and allow normal error handling to occur in the evaluated script:

# [coeval] takes one argument and evaluates it as a script one level up
coroutine coeval apply {{} {
    while 1 {yieldto try [yield] on ok {} [list [info coroutine]]
    }
}}

If the coroutine needs to perform additional work after the script is evaluated, it must make provision to capture any error and return it :

coroutine coeval apply [list {} {
    set ns [namespace eval [info cmdcount] {namespace current}]
    namespace upvar $ns res res options options
    set res {}
    set options {-level 0 -code ok}
    set cumulative 0
    while 1 {
        set start [clock microseconds]
        set script [lindex [yieldto return -options $options $res] 0]
        yieldto if 1 "catch [list $script] [list ${ns}::res] [
            list ${ns}::options]; [list [info coroutine]]"
        incr cumulative [expr {[clock microseconds] - $start}]
        puts [list {cumulative time} $cumulative]
    }
} [namespace current]]

Mutating the Evaluation Stack

In the following example, the error is neve called:

proc p0 {} {
        coroutine c2 p1
        error {error after returning from c2}
}


proc p1 {} {
        yieldto yieldto [list [info coroutine]]
}

coroutine c1 p0
puts {no error}

output:

no error

The reason is that p1 yields not to p0, but to the caller of p0. Here is how:

  • c2 yields
  • and the second yieldto then takes place in the context of c1, so c1 yields back to c2
  • with c1 no longer on the evaluation stack when p1 returns it returns to the next level on the evaluation stack which is now the level that originally called c1

Avoiding Stack Growth

PYK 2017-06-01:

growstack grows the stack and eventually produces the error, too many nested evaluations (infinite loop)?:

coroutine growstack ::apply [list {} {
    while 1 {
        yieldto try {incr i} finally [list [info coroutine]]
    }
} [namespace current]]

The error happens because try never returns. I.e., try remains on the stack, which keeps growing as the calls to try pile up. This can happen with commands such as eval, if, for, or any other command that is used to evaluate a script. One way to avoid the stack growth is to use catch instead:

coroutine nostack ::apply [list {} {
    while 1 {
        yieldto apply {{ns coroutine} {
            catch {incr i} ${ns}::cres ${ns}::copts 
            tailcall $coroutine
        }} [namespace current] [info coroutine]
        # Add error handling here if needed.
    }
} [namespace current]]

Here is another example of the same phenomenon:

proc c {} {
    set script [string map [list @coro@ [list [info coroutine]]] {
        if {[info coroutine] eq {}} {
            @coro@
        } else {
            yieldto @coro@
        }

    }]

    set res {}
    for {set i 0} {$i < 65536} {incr i} {
        puts [list iteration $i $res]
        set res [yieldto {*}$script]
    }
}
coroutine c1 c

To avoid the "too many nested evaluations" error, use the same strategy of yielding to a routine which then uses tailcall:

#! /usr/bin/env tclsh

proc c {} {
    set script [list ::apply [list coro {
        tailcall $coro [::info coroutine]
    }] [info coroutine]]

    #set script [list [info coroutine]]

    # more than enough iterations to trigger "infinite evaluation" error
    set res {}
    for {set i 0} {$i < 65536} {incr i} {
        puts [list iteration $i $res]
        set res [yieldto {*}$script]
    }
}

coroutine c1 c

A Conversation with MS

The following conversation with MS, Tcl Chatroom, 2014-09-25, was very enlightening for me. So preserving for posterity.

[18:55]        apn        miguel, the inject worked fine
[18:56]        apn        You can even do multiple injects though the injected code cannot inject itself for the next call (only tried it to see what would happen)
[18:59]        miguel        apn: by design you can only inject code into a sleeping coro ... head hurts imagining how to insure the alternative works properly
[19:04]        miguel        ... so, if you really need it, you can easily build an [autoinject] out of [yieldto], [info coroutine] and [inject]
[19:07]        apn        I have not progressed to yieldto yet or have a use case for it that is obvious (to me)
[19:07]        miguel        well, autoinject would be a use case 
[19:08]           * jima neither has "visualized" yieldto
[19:08]        miguel        yieldto is to yield as tailcall is to return
[19:08]        apn        Kind of get that, but not quite where it might be used
[19:08]        apn        other than autoinject 
[19:10]        miguel        if you want to "yield" with a non-OK code, you can do eg 'yieldto return -code continue'
[19:10]        miguel        ... the coro yields, but the coro's caller sees a continue return
[19:12]        apn        So for example if I want to return a -break from a Tk bind callback script that invoked a coro ?
[19:13]        miguel        (more examples abound, symmetric coroutines among them)
[19:13]        apn        I was actually wondering about that this afternoon experimenting with integrating Tk with a coro based fiber package I wrote
[19:14]           * miguel answers a generic "yes", without actually being sure about what the question was
[19:15]        apn        Something like
[19:15]        apn        bind MyTkTag <<MyEvent>> "do something; break"
[19:15]        apn        If instead, I was to call a coro like
[19:16]        apn        bind MyTkTag <<MyEvent>> "mycoro %W"
[19:16]        apn        and wanted mycoro to return a break
[19:16]        apn        so further bound scripts would not get execututed
[19:16]        apn        whatever execututed means
[19:17]        apn        If mycoro was a normal proc, it could do a [return -code break]
[19:17]        miguel        ok - if the coro should go to sleep (yield), then it should the above trick of yieldto'ing to [break]
[19:18]        miguel        ... if it should die after doing its stuff, [return -code break] is a possibility (with the appropriate -level if it is returning from some nested proc call)
[19:19]        apn        I was thinking of the first case
[19:20]        miguel        [yieldto break] should do it ... or [yieldto return -code break]
[19:25]        miguel        apn: [yieldto] yields to another command, it doesn't have to be another coro. It just means 'put the present coro to sleep, remove it from Tcl's and C call stacks, and run this other command in it's place (without the coro's caller ever nticing a thing)'
[19:26]        miguel        ... some kind of delegation - let my minion do the job, it's siesta time for me
[19:26]        apn        So what this other command returns is what the original caller of the coro sees ? Cool!
[19:27]        apn        (as the return value I mean)
[19:27]        miguel        yup ... that's the tailcall-like aspect
[19:27]        apn        *now* I get it

Historical

New experimental command added in ::tcl::unsupported on 2009-12-07.

tcl::unsupported::yieldTo command ?arg ...?

Suspends the current coroutine and makes the current coroutine's caller invoke command (with the optional args). Command is resolved prior the suspension in the context of the current coroutine, and can refer to another coroutine (though this is not required). May only be called from within a coroutine; it is an error to call it from elsewhere.

Exceptions:

TCL COROUTINE ILLEGAL_YIELD — if called from an illegal context (i.e., not in a coroutine).

Docs forthcoming, for now just the tclcore thread at [L1 ]


AMG: Why is this command named with an internal capital letter? All other built-in Tcl commands (as opposed to utility procs) are named with multiple words (or abbreviations thereof) jammed together with no capitals or underscores. Examples abound: bgerror, fblocked, foreach, gets, lappend, regexp, uplevel, and vwait.

MS: no special reason

CRC: From the thread, it appears that the author of this code just likes, or is used to, camel case. In the ensuing discussion it was referred to as "yieldto". Not being a fan of camel case myself, I hope that if it gets incorporated, it is "yieldto", or "yield -target" as DKF suggests.

DGP: FWIW, there are precedents in the other direction as well: pkg_mkIndex, tcl_findLibrary, tcl_endOfWord, etc.

AMG: Those are the utility procs I was referring to.


jbr: I'm wondering why we are polluting the global namespace with yield variants. Don't people already have complaints about the many list commands and the file io commands not being grouped together?

AMG: Same question. Some time ago there was a discussion on tcl-core regarding yield, yieldTo, and yieldm. I don't recall how it was resolved, or even if it was resolved at all. I advocated unifying these three commands into a single [yield] command that takes options. In addition to cleaning up the global namespace, this would close the gap between yieldTo and yieldm: how do you instruct the current coroutine's caller to invoke a command, while making the current coroutine's resume command accept any number of arguments?

Page Authors

pyk