POE::Wheel::Run - portably run blocking code and programs in subprocesses
#!/usr/bin/perl
use warnings;
use strict;
use POE qw( Wheel::Run );
POE::Session->create(
inline_states => {
_start => \&on_start,
got_child_stdout => \&on_child_stdout,
got_child_stderr => \&on_child_stderr,
got_child_close => \&on_child_close,
got_child_signal => \&on_child_signal,
}
);
POE::Kernel->run();
exit 0;
sub on_start {
my $child = POE::Wheel::Run->new(
Program => [ "/bin/ls", "-1", "/" ],
StdoutEvent => "got_child_stdout",
StderrEvent => "got_child_stderr",
CloseEvent => "got_child_close",
);
$_[KERNEL]->sig_child($child->PID, "got_child_signal");
# Wheel events include the wheel's ID.
$_[HEAP]{children_by_wid}{$child->ID} = $child;
# Signal events include the process ID.
$_[HEAP]{children_by_pid}{$child->PID} = $child;
print(
"Child pid ", $child->PID,
" started as wheel ", $child->ID, ".\n"
);
}
# Wheel event, including the wheel's ID.
sub on_child_stdout {
my ($stdout_line, $wheel_id) = @_[ARG0, ARG1];
my $child = $_[HEAP]{children_by_wid}{$wheel_id};
print "pid ", $child->PID, " STDOUT: $stdout_line\n";
}
# Wheel event, including the wheel's ID.
sub on_child_stderr {
my ($stderr_line, $wheel_id) = @_[ARG0, ARG1];
my $child = $_[HEAP]{children_by_wid}{$wheel_id};
print "pid ", $child->PID, " STDERR: $stderr_line\n";
}
# Wheel event, including the wheel's ID.
sub on_child_close {
my $wheel_id = $_[ARG0];
my $child = delete $_[HEAP]{children_by_wid}{$wheel_id};
# May have been reaped by on_child_signal().
unless (defined $child) {
print "wid $wheel_id closed all pipes.\n";
return;
}
print "pid ", $child->PID, " closed all pipes.\n";
delete $_[HEAP]{children_by_pid}{$child->PID};
}
sub on_child_signal {
print "pid $_[ARG1] exited with status $_[ARG2].\n";
my $child = delete $_[HEAP]{children_by_pid}{$_[ARG1]};
# May have been reaped by on_child_close().
return unless defined $child;
delete $_[HEAP]{children_by_wid}{$child->ID};
}POE::Wheel::Run executes a program or block of code in a subprocess, created the usual way: using fork(). The parent process may exchange information with the child over the child's STDIN, STDOUT and STDERR filehandles.
In the parent process, the POE::Wheel::Run object represents the child process. It has methods such as PID() and kill() to query and manage the child process.
POE::Wheel::Run's put() method sends data to the child's STDIN. Child output on STDOUT and STDERR may be dispatched as events within the parent, if requested.
POE::Wheel::Run can also notify the parent when the child has closed its output filehandles. Some programs remain active, but they close their output filehandles to indicate they are done writing.
A more reliable way to detect child exit is to use POE::Kernel's sig_child() method to wait for the wheel's process to be reaped. It is in fact vital to use sig_child() in all circumstances since without it, POE will not try to reap child processes.
Failing to use sig_child() has in the past led to wedged machines. Long-running programs have leaked processes, eventually consuming all available slots in the process table and requiring reboots.
Because process leaks are so severe, POE::Kernel will check for this condition on exit and display a notice if it finds that processes are leaking. Developers should heed these warnings.
POE::Wheel::Run communicates with the child process in a line-based fashion by default. Programs may override this by specifying some other POE::Filter object in "StdinFilter", "StdoutFilter", "StdioFilter" and/or "StderrFilter".
POE::Wheel subclasses tend to perform a lot of setup so that they run lighter and faster. POE::Wheel::Run's constructor is no exception.
new() creates and returns a new POE::Wheel::Run object. If it's successful, the object will represent a child process with certain specified qualities. It also provides an OO- and event-based interface for asynchronously interacting with the process.
Conduit specifies the inter-process communications mechanism that will be used to pass data between the parent and child process. Conduit may be one of "pipe", "socketpair", "inet", "pty", or "pty-pipe". POE::Wheel::Run will use the most appropriate Conduit for the run-time (not the compile-time) operating system, but this varies from one OS to the next.
Internally, POE::Wheel::Run passes the Conduit type to POE::Pipe::OneWay and POE::Pipe::TwoWay. These helper classes were created to make IPC portable and reusable. They do not require the rest of POE.
Three Conduit types use pipes or pipelike inter-process communication: "pipe", "socketpair" and "inet". They determine whether the internal IPC uses pipe(), socketpair() or Internet sockets. These Conduit values are passed through to POE::Pipe::OneWay or POE::Pipe::TwoWay internally.
The "pty" conduit type runs the child process under a pseudo-tty, which is created by IO::Pty. Pseudo-ttys (ptys) convince child processes that they are interacting with terminals rather than pipes. This may be used to trick programs like ssh into believing it's secure to prompt for a password, although passphraseless identities might be better for that.
The "pty" conduit cannot separate STDERR from STDOUT, but the "pty-pipe" mode can.
The "pty-pipe" conduit uses a pty for STDIN and STDOUT and a one-way pipe for STDERR. The additional pipe keeps STDERR output separate from STDOUT.
The IO::Pty module is only loaded if "pty" or "pty-pipe" is used. It's not a dependency until it's actually needed.
Winsize sets the child process' terminal size. Its value should be an arrayref with four elements. The first two elements must be the number of lines and columns for the child's terminal window, respectively. The second pair of elements describe the terminal's X and Y dimensions in pixels. If the last pair is missing, they will be calculated from the lines and columns using a 9x16 cell size.
$_[HEAP]{child} = POE::Wheel::Run->new(
# ... among other things ...
Winsize => [ 25, 80, 720, 400 ],
);Winsize is only valid for conduits that use pseudo-ttys: "pty" and "pty-pipe". Other conduits don't simulate terminals, so they don't have window sizes.
Winsize defaults to the parent process' window size, assuming the parent process has a terminal to query.
CloseOnCall, when true, turns on close-on-exec emulation for subprocesses that don't actually call exec(). These would be instances when the child is running a block of code rather than executing an external program. For example:
$_[HEAP]{child} = POE::Wheel::Run->new(
# ... among other things ...
CloseOnCall => 1,
Program => \&some_function,
);CloseOnCall is off (0) by default.
CloseOnCall works by closing all file descriptors greater than $^F in the child process before calling the application's code. For more details, please the discussion of $^F in perlvar.
StdioDriver specifies a single POE::Driver object to be used for both STDIN and STDOUT. It's equivalent to setting "StdinDriver" and "StdoutDriver" to the same POE::Driver object.
POE::Wheel::Run will create and use a POE::Driver::SysRW driver of one isn't specified. This is by far the most common use case, so it's the default.
StdinDriver sets the POE::Driver used to write to the child process' STDIN IPC conduit. It is almost never needed. Omitting it will allow POE::Wheel::Run to use an internally created POE::Driver::SysRW object.
StdoutDriver sets the POE::Driver object that will be used to read from the child process' STDOUT conduit. It's almost never needed. If omitted, POE::Wheel::Run will internally create and use a POE::Driver::SysRW object.
StderrDriver sets the driver that will be used to read from the child process' STDERR conduit. As with "StdoutDriver", it's almost always preferable to let POE::Wheel::Run instantiate its own driver.
CloseEvent contains the name of an event that the wheel will emit when the child process closes its last open output handle. This is a consistent notification that the child is done sending output. Please note that it does not signal when the child process has exited. Programs should use sig_child() to detect that.
While it is impossible for ErrorEvent or StdoutEvent to happen after CloseEvent, there is no such guarantee for CHLD, which may happen before or after CloseEvent.
In addition to the usual POE parameters, each CloseEvent comes with one of its own:
ARG0 contains the wheel's unique ID. This can be used to keep several child processes separate when they're managed by the same session.
A sample close event handler:
sub close_state {
my ($heap, $wheel_id) = @_[HEAP, ARG0];
my $child = delete $heap->{child}->{$wheel_id};
print "Child ", $child->PID, " has finished.\n";
}ErrorEvent contains the name of an event to emit if something fails. It is optional; if omitted, the wheel will not notify its session if any errors occur. However, POE::Wheel::Run->new() will still throw an exception if it fails.
ARG0 contains the name of the operation that failed. It may be 'read', 'write', 'fork', 'exec' or the name of some other function or task. The actual values aren't yet defined. They will probably not correspond so neatly to Perl builtin function names.
ARG1 and ARG2 hold numeric and string values for $!, respectively. "$!" will eq "" for read error 0 (child process closed the file handle).
ARG3 contains the wheel's unique ID.
ARG4 contains the name of the child filehandle that has the error. It may be "STDIN", "STDOUT", or "STDERR". The sense of ARG0 will be the opposite of what you might normally expect for these handles. For example, POE::Wheel::Run will report a "read" error on "STDOUT" because it tried to read data from the child's STDOUT handle.
A sample error event handler:
sub error_state {
my ($operation, $errnum, $errstr, $wheel_id) = @_[ARG0..ARG3];
$errstr = "remote end closed" if $operation eq "read" and !$errnum;
warn "Wheel $wheel_id generated $operation error $errnum: $errstr\n";
}Note that unless you deactivate the signal pipe, you might also see EIO (5) error during read operations.
StdinEvent contains the name of an event that Wheel::Run emits whenever everything queued by its put() method has been flushed to the child's STDIN handle. It is the equivalent to POE::Wheel::ReadWrite's FlushedEvent.
StdinEvent comes with only one additional parameter: ARG0 contains the unique ID for the wheel that sent the event.
StdoutEvent contains the name of an event that Wheel::Run emits whenever the child process writes something to its STDOUT filehandle. In other words, whatever the child prints to STDOUT, the parent receives a StdoutEvent---provided that the child prints something compatible with the parent's StdoutFilter.
StdoutEvent comes with two parameters. ARG0 contains the information that the child wrote to STDOUT. ARG1 holds the unique ID of the wheel that read the output.
sub stdout_state {
my ($heap, $input, $wheel_id) = @_[HEAP, ARG0, ARG1];
print "Child process in wheel $wheel_id wrote to STDOUT: $input\n";
}StderrEvent behaves exactly as StdoutEvent, except for data the child process writes to its STDERR filehandle.
StderrEvent comes with two parameters. ARG0 contains the information that the child wrote to STDERR. ARG1 holds the unique ID of the wheel that read the output.
sub stderr_state {
my ($heap, $input, $wheel_id) = @_[HEAP, ARG0, ARG1];
print "Child process in wheel $wheel_id wrote to STDERR: $input\n";
}This is a filehandle or filename to which standard output will be redirected. It is an error to use this option together with StdoutEvent. This is useful in case your program needs to have standard I/O, but do not actually care for its contents to be visible to the parent.
Just like RedirectStdout, but with standard error. It is an error to use this together with StderrEvent
This is a filehandle or filename which the child process will use as its standard input. It is an error to use this option with StdinEvent
This will redirect stderr and stdout to the same filehandle. This is equivalent to do doing something like
$ something > /path/to/output 2>&1in bourne shell.
While output filehandles will be closed if there are no events to be received on them, stdin is open by default - because lack of an event handler does not necessarily mean there is no desired input stream. This option explicitly disables the creation of an IPC stdin conduit.
StdioFilter, if used, must contain an instance of a POE::Filter subclass. This filter describes how the parent will format put() data for the child's STDIN, and how the parent will parse the child's STDOUT.
If STDERR will also be parsed, then a separate StderrFilter will also be needed.
StdioFilter defaults to a POE::Filter::Line instance, but only if both StdinFilter and StdoutFilter are not specified. If either StdinFilter or StdoutFilter is used, then StdioFilter is illegal.
StdinFilter may be used to specify a particular STDIN serializer that is different from the STDOUT parser. If specified, it conflicts with StdioFilter. StdinFilter's value, if specified, must be an instance of a POE::Filter subclass.
Without a StdinEvent, StdinFilter is illegal.
StdoutFilter may be used to specify a particular STDOUT parser that is different from the STDIN serializer. If specified, it conflicts with StdioFilter. StdoutFilter's value, if specified, must be an instance of a POE::Filter subclass.
Without a StdoutEvent, StdoutFilter is illegal.
StderrFilter may be used to specify a filter for a child process' STDERR output. If omitted, POE::Wheel::Run will create and use its own POE::Filter::Line instance, but only if a StderrEvent is specified.
Without a StderrEvent, StderrFilter is illegal.
Group contains a numeric group ID that the child process should run within. By default, the child process will run in the same group as the parent.
Group is not fully portable. It may not work on systems that have no concept of user groups. Also, the parent process may need to run with elevated privileges for the child to be able to change groups.
User contains a numeric user ID that should own the child process. By default, the child process will run as the same user as the parent.
User is not fully portable. It may not work on systems that have no concept of users. Also, the parent process may need to run with elevated privileges for the child to be able to change users.
When true, NoSetSid disables setsid() in the child process. By default, the child process calls setsid() is called so that it may execute in a separate UNIX session.
This option is deprecated. As IO::Pty uses setsid() this module has no control on whether it is invoked or not.
When true, NoSetPgrp disables setprgp() in the child process. By default, the child process calls setpgrp() to change its process group, if the OS supports that.
setsid() is used instead of setpgrp() if Conduit is pty or pty-pipe. See "NoSetSid".
Priority adjusts the child process' niceness or priority level, depending on which (if any) the underlying OS supports. Priority contains a numeric offset which will be added to the parent's priority to determine the child's.
The priority offset may be negative, which in UNIX represents a higher priority. However UNIX requires elevated privileges to increase a process' priority.
Program specifies the program to exec() or the block of code to run in the child process. Program's type is significant.
If Program holds a scalar, its value will be executed as exec($program). Shell metacharacters are significant, per exec(SCALAR) semantics.
If Program holds an array reference, it will executed as exec(@$program). As per exec(ARRAY), shell metacharacters will not be significant.
If Program holds a code reference, that code will be called in the child process. This mode allows POE::Wheel::Run to execute long-running internal code asynchronously, while the usual modes execute external programs. The child process will exit after that code is finished, in such a way as to avoid DESTROY and END block execution. See "Coderef Execution Side Effects" for more details.
perlfunc has more information about exec() and the different ways to call it.
Please avoid calling exit() explicitly when executing a subroutine. The child process inherits all objects from the parent, including ones that may perform side effects. POE::Wheel::Run takes special care to avoid object destructors and END blocks in the child process, but calling exit() will trigger them.
If specified, ProgramArgs should refer to a list of parameters for the program being run.
my @parameters = qw(foo bar baz); # will be passed to Program
ProgramArgs => \@parameters;event() allows programs to change the events that Wheel::Run emits when certain activities occurs. EVENT_TYPE may be one of the event parameters described in POE::Wheel::Run's constructor.
This example changes the events that $wheel emits for STDIN flushing and STDOUT activity:
$wheel->event(
StdinEvent => 'new-stdin-event',
StdoutEvent => 'new-stdout-event',
);Undefined EVENT_NAMEs disable events.
put() queues up a list of RECORDS that will be sent to the child process' STDIN filehandle. These records will first be serialized according to the wheel's StdinFilter. The serialized RECORDS will be flushed asynchronously once the current event handler returns.
get_stind_filter() returns the POE::Filter object currently being used to serialize put() records for the child's STDIN filehandle. The return object may be used according to its own interface.
get_stdout_filter() returns the POE::Filter object currently being used to parse what the child process writes to STDOUT.
get_stderr_filter() returns the POE::Filter object currently being used to parse what the child process writes to STDERR.
Set StdinFilter and StdoutFilter to the same new FILTER_OBJECT. Unparsed STDOUT data will be parsed later by the new FILTER_OBJECT. However, data already put() will remain serialized by the old filter.
Set StdinFilter to a new FILTER_OBJECT. Data already put() will remain serialized by the old filter.
Set StdoutFilter to a new FILTER_OBJECT. Unparsed STDOUT data will be parsed later by the new FILTER_OBJECT.
Set StderrFilter to a new FILTER_OBJECT. Unparsed STDERR data will be parsed later by the new FILTER_OBJECT.
Pause reading of STDOUT from the child. The child process may block if the STDOUT IPC conduit fills up. Reading may be resumed with resume_stdout().
Pause reading of STDERR from the child. The child process may block if the STDERR IPC conduit fills up. Reading may be resumed with resume_stderr().
Resume reading from the child's STDOUT filehandle. This is only meaningful if pause_stdout() has been called and remains in effect.
Resume reading from the child's STDERR filehandle. This is only meaningful if pause_stderr() has been called and remains in effect.
shutdown_stdin() closes the child process' STDIN and stops the wheel from reporting StdinEvent. It is extremely useful for running utilities that expect to receive EOF on STDIN before they respond.
ID() returns the wheel's unique ID. Every event generated by a POE::Wheel::Run object includes a wheel ID so that it can be matched to the wheel that emitted it. This lets a single session manage several wheels without becoming confused about which one generated what event.
ID() is not the same as PID().
PID() returns the process ID for the child represented by the POE::Wheel::Run object. It's often used as a parameter to sig_child().
PID() is not the same as ID().
POE::Wheel::Run's kill() method sends a SIGNAL to the child process the object represents. kill() is often used to force a reluctant program to terminate. SIGNAL is one of the operating signal names present in %SIG.
kill() returns the number of processes successfully signaled: 1 on success, or 0 on failure, since the POE::Wheel::Run object only affects at most a single process.
kill() sends SIGTERM if SIGNAL is undef or omitted.
get_driver_out_messages() returns the number of put() records remaining in whole or in part in POE::Wheel::Run's POE::Driver output queue. It is often used to tell whether the wheel has more input for the child process.
In most cases, StdinEvent may be used to trigger activity when all data has been sent to the child process.
get_driver_out_octets() returns the number of serialized octets remaining in POE::Wheel::Run's POE::Driver output queue. It is often used to tell whether the wheel has more input for the child process.
In the past POE::Wheel::Run did not support MSWin32 and users had to use custom work-arounds. Then Chris Williams ( BINGOS ) arrived and saved the day with his POE::Wheel::Run::Win32 module. After some testing, it was decided to merge the win32 code into POE::Wheel::Run. Everyone was happy!
However, after some investigation Apocalypse ( APOCAL ) found out that in some situations it still didn't behave properly. The root cause was that the win32 code path in POE::Wheel::Run didn't exit cleanly. This means DESTROY and END blocks got executed! After talking with more people, the solution was not pretty.
The problem is that there is no equivalent of POSIX::_exit() for MSWin32. Hopefully, in a future version of Perl this can be fixed! In the meantime, POE::Wheel::Run will use CORE::kill() to terminate the child. However, this comes with a caveat: you will leak around 1KB per exec. The code has been improved so the chance of this happening has been reduced.
As of now the most reliable way to trigger this is to exec an invalid binary. The definition of "invalid binary" depends on different things, but what it means is that Win32::Job->spawn() failed to run. This will force POE::Wheel::Run to use the workaround to exit the child. If this happens, a very big warning will be printed to the STDERR of the child and the parent process will receive it.
If you are a Perl MSWin32 hacker, PLEASE help us with this situation! Go read rt.cpan.org bug #56417 and talk with us/p5p to see where you can contribute.
Thanks again for your patience as we continue to improve POE::Wheel::Run on MSWin32!
Windows will often fail to report EOF on pipes when subprocesses are killed. The work-around is to catch the signal in the subprocess, and exit normally:
my $child = POE::Wheel::Run->new(
Program => sub {
$SIG{INT} = sub { exit };
...;
},
...,
);Be sure to kill() the subprocess using the same signal that it catches and exits upon. Remember, not all signals can be caught by user code.
$child->kill("INT");It's common to scrub a child process' environment, so that only required, secure values exist. This amounts to clearing the contents of %ENV and repopulating it.
Environment scrubbing is easy when the child process is running a subroutine, but it's not so easy---or at least not as intuitive---when executing external programs.
The way we do it is to run a small subroutine in the child process that performs the exec() call for us.
Program => \&exec_with_scrubbed_env,
sub exec_with_scrubbed_env {
delete @ENV{keys @ENV};
$ENV{PATH} = "/bin";
exec(@program_and_args);
}That deletes everything from the environment and sets a simple, secure PATH before executing a program.
The child process is created by fork(), which duplicates the parent process including a copy of POE::Kernel, all running Session instances, events in the queue, watchers, open filehandles, and so on.
When executing an external program, the UNIX exec() call immediately replaces the copy of the parent with a completely new program.
When executing internal coderefs, however, we must preserve the code and any memory it might reference. This leads to some potential side effects.
Objects that were created in the parent process are copied into the child. When the child exits normally, any DESTROY and END blocks are executed there. Later, when the parent exits, they may run again.
POE::Wheel::Run takes steps to avoid running DESTROY and END blocks in the child process. It uses POSIX::_exit() to bypass them. If that fails, it may even kill() itself.
If an application needs to exit explicitly, for example to return an error code to the parent process, then please use POSIX::_exit() rather than Perl's core exit().
This warning is displayed from POE::Kernel's DESTROY method. It's a side effect of calling exit() in a child process that was started before POE::Kernel->run() could be called. The child process receives a copy of POE::Kernel where run() wasn't called, even if it was called later in the parent process.
The most direct solution is to call POSIX::_exit() rather than exit(). This will bypass POE::Kernel's DESTROY, and the message it emits.
Calling POE::Kernel->run() in the child process effectively resumes the copy of the parent process. This is rarely (if ever) desired.
More commonly, an application wants to run an entirely new POE::Kernel instance in the child process. This is supported by first stop()ping the copied instance, starting one or more new sessions, and calling run() again. For example:
Program => sub {
# Wipe the existing POE::Kernel clean.
$poe_kernel->stop();
# Start a new session, or more.
POE::Session->create(
...
);
# Run the new sessions.
POE::Kernel->run();
}Strange things are bound to happen if the program does not call "stop" in POE::Kernel before "run" in POE::Kernel. However this is vaguely supported in case it's the right thing to do at the time.
POE::Wheel describes wheels in general.
The SEE ALSO section in POE contains a table of contents covering the entire POE distribution.
POE::Wheel::Run's constructor should emit proper events when it fails. Instead, it just dies, carps or croaks. This isn't necessarily bad; a program can trap the death in new() and move on.
Priority is a delta, not an absolute niceness value.
It might be nice to specify User by name rather than just UID.
It might be nice to specify Group by name rather than just GID.
POE::Pipe::OneWay and Two::Way don't require the rest of POE. They should be spun off into a separate distribution for everyone to enjoy.
If StdinFilter and StdoutFilter seem backwards, remember that it's the filters for the child process. StdinFilter is the one that dictates what the child receives on STDIN. StdoutFilter tells the parent how to parse the child's STDOUT.
Please see POE for more information about authors and contributors.