Reading and writing to a piped command
The methods for accomplishing some task involving writing to and then reading from a piped command depend primarily on whether the communication will be interactive or non-interactive. The non-interactive case is fairly straight-forward and robust. The interactive case can be problematic, since external programs may buffer incoming and outgoing data in arbitrary ways. Expect is the best all-purpose tool for interactive communication with another program.
proc gzip {buf} { set chan [open "|gzip -c" r+] fconfigure $chan -translation binary -encoding binary puts $chan $buf flush $chan chan close $chan write set buf [read $chan] close $chan return $buf }
The write side of $chan must be closed so that gzip reads any remaining data in the buffers processes it, and sends the result to its stdtout.
For versions of Tcl that don't have chan close ?direction?, try the following workaround:
proc gzip {buf} { return [exec gzip -c << $buf] }
#! /bin/env tclsh proc communicate {chan msg pump respond eof } { fileevent $chan readable [list apply [list {chan respond eof} { set res [gets $chan] if {$res ne {}} { fileevent $chan readable {} {*}$respond $res } if {[eof $chan]} { fileevent $chan readable {} {*}$eof } }] $chan $respond $eof] fileevent $chan writable [list apply [list {chan msg pump} { catch {puts $chan $msg} #something like this may be needed if the child program can be configured #with reasonable output buffering #fileevent $chan writable [list apply [list {chan pump} { # puts $chan $pump #}] $chan $pump] fileevent $chan writable {} }] $chan $msg $pump] } proc process {chan count args} { puts "sed output: $args" if {$count < 2} { communicate $chan hello \n [namespace code [list process $chan [incr count]]] \ [namespace code [list closed $chan]] } else { catch {close $chan} set ::done 0 } } proc closed {chan} { #flush first to catch any pipe error, getting it out of the way in order to grab #the exist status with [close] if {[catch {flush $chan} eres eopts]} { set status [catch {close $chan} eres eopts] set ::done $status } set ::done 0 } #try this line, which causes a sed error, to see how that's handled #set chan [open {|sed -l {s/hello/goodbye} 2>@stderr} r+] set chan [open {|sed -l s/hello/goodbye/ 2>@stderr} r+] fconfigure $chan -buffering none communicate $chan hello \n [namespace code [list process $chan 1]] \ [namespace code [list closed $chan]] vwait ::done return -code $::done
It's up to the individual program when to print out its results. The -l option to BSD sed configures sed to print output whenever at least one line of output is ready. -u accomplishes the same for some other versions of sed. The pump feature of this example can be used pump some program-specific value through the channel until the desired output is collected. It's a hack, but depending on the program, might be the only way to accomplish the task. Expect is the fully-featured tool for this type of task. Another approach would be to fork the current process into a producer and a consumer.
Neil Madden - here is a real-life example of interacting with a program through a pipe. The program in question is ispell - a UNIX spell-checking utility. I use it to spell-check the contents of a text widget containing LaTeX markup. There are a number of issues to deal with:
The example does not use [fileevent], as this would complicate this particular example. The options passed to ispell are -a (which makes it non-interactive) and -t (which makes it recognize TeX input).
set contents [split [$text get 1.0 end] \n] set pipe [open [list | ispell -a -t] r+] fconfigure $pipe -blocking 0 -buffering line set ver [gets $pipe] ;# Ignore the initial version line set linenum 1 foreach line $contents { set wordnum 1 foreach word [split $line] { puts $pipe $word ;# Feed word to ispell while 1 { set len [gets $pipe res] if {$len > 0} { # A valid result # do stuff continue } else { if {[fblocked $pipe]} { # No output break } elseif {[eof $pipe]} { # Pipe closed catch {close $pipe} return } # A blank line - skip } } incr wordnum } incr linenum }
Thanks to Kevin Kenny for helping me figure this out.
DDG - The wiki page Spellcheck Widget using Aspell contains an example on how to use aspell or hunspell to check a given text string and how to use the suggestions by aspell/hunspell to replace the wrong words.
Arjen Markus: I have experimented a bit with plain Tcl driving another program. As this needs to work on Windows 95 (NT, ...) as well as UNIX in four or five flavours, I wanted to use plain Tcl, not Expect (however much I would appreciate the chance to do something really useful with Expect - apart from Android :-).
I think it is worth a page of its own, but here is a summary:
set inout [open |[list myprog] r+] fconfigure $inout -buffering line
Buffering might be out of your hands, though, for "real 16-bit commandline applications", which apparently don't have the ability to flush (reliably), except on close.
DDG - 2022-01-12: To start a python3 pipe you have to select the command line arguments *-iu* for interactive mode and unbuffered output. Here an example to simulate the terminal output of the python3 interpreter
DDG - 2022-01-31: Added error catching and -q option for quiet start.
set code "x=1 print(x) y=x+1 print(y) # error checking print(z) " proc piperead {pipe args} { if {![eof $pipe]} { #puts "read $pipe : $args" set got [regsub {.+> } [gets $pipe] ""] puts "$got" } } # 2022-01-31: adding stderr redirection set pipe [open "|python3 -uiq 2>@1" r+] fconfigure $pipe -buffering none -blocking false fileevent $pipe readable [list piperead $pipe] set pywait [list] foreach pyline [split $code \n] { puts stdout ">>> $pyline" puts $pipe $pyline # need some delay for the pipe after 200 [list append pywait ""] vwait pywait } close $pipe puts "Finished python3 pipe!" exit 0
And this is the output:
>>> x=1 >>> print(x) 1 >>> y=x+1 >>> print(y) 2 >>> # error checking >>> print(z) Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'z' is not defined
DDG - 2022-01-12: To start a R pipe again you have to use the interactive argument *--interactive*, further you can suppress the R welcome message by using the *-q* option. Here an example to mimic the R terminal:
set code "x=1 print(x) y=x+1 print(y) print(R.version.string)" proc piperead {pipe args} { if {![eof $pipe]} { #puts "read $pipe : $args" set got [gets $pipe] if {$got ne ""} { puts "$got" } } } set pipe [open "|R -q --interactive" r+] fconfigure $pipe -buffering none -blocking false fileevent $pipe readable [list piperead $pipe] set rwait [list] foreach rline [split $code \n] { puts $pipe $rline # need some delay for the pipe after 100 [list append rwait ""] vwait rwait } close $pipe puts "Finished R pipe!" exit 0
And here the output:
> x=1 > print(x) [1] 1 > y=x+1 > print(y) [1] 2 > print(R.version.string) [1] "R version 4.0.5 (2021-03-31)" Finished R pipe!