TalkingToSubprocess » History » Version 7

Chris Cannam, 2010-12-03 01:39 PM

1 1 Chris Cannam
h1. Communicating with a subprocess
2 1 Chris Cannam
3 1 Chris Cannam
h3. The problem
4 1 Chris Cannam
5 2 Chris Cannam
We want to run a subprocess (Hg) in such a way as to be able to talk to it interactively, pretending we are a terminal -- so we can deal with username/password interactions.  The subprocess has some possibly platform-dependent test for whether its input is a terminal; if not, it will not ask for credentials (failing instead).
6 2 Chris Cannam
7 2 Chris Cannam
In practice, it appears the test comes down to a call to Python's sys.stdio.isatty() function.  The documentation suggests you can override this test with e.g. --config ui.interactive=true on the command line; I'm not entirely sure.
8 2 Chris Cannam
9 2 Chris Cannam
We are using QProcess, which perhaps doesn't actually make matters any simpler to understand.
10 2 Chris Cannam
11 2 Chris Cannam
h3. Linux and OS/X
12 2 Chris Cannam
13 2 Chris Cannam
Two parts to the problem: how to give the subprocess a pseudoterminal, and how to make sure it doesn't get the controlling terminal of the parent process.  (isatty returns true if the file descriptor is a terminal or pseudoterminal.)
14 2 Chris Cannam
15 2 Chris Cannam
QProcess can set the subprocess stdin to an existing file by name.  So, we can call openpty() (nonstandard, nonrecommended BSD API) or carry out similar magic to allocate a pseudoterminal master/slave pair via /dev/ptmx or whatever, then pass the slave device name to QProcess::setStandardInputFile.
16 2 Chris Cannam
17 5 Chris Cannam
This seems to get confused if the parent process already has a controlling terminal.  We need to detach from our controlling terminal first.
18 2 Chris Cannam
19 2 Chris Cannam
A method that seems to be recommended in some quarters is to fork, then setsid, then fork -- but we can't easily do this at the point of running the subprocess, because QProcess is handling that fork for us.  (Would execvp not be easier?)
20 2 Chris Cannam
21 2 Chris Cannam
And we can't fork/setsid/fork at the start of our own process, because although this works fine on Linux, OS/X will reject it in subsequent Core API calls ("__THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__").
22 2 Chris Cannam
23 2 Chris Cannam
So instead at startup we explicitly detach from the terminal using ioctl with TIOCNOTTY on fd 0 if isatty(0) returns true, or on the result of opening /dev/tty otherwise.  That seems to work.
24 2 Chris Cannam
25 6 Chris Cannam
*Update:* Doesn't work as well as I thought.  It's OK for the password, but for the username it tries to start using readline which confuses us completely.
26 6 Chris Cannam
27 2 Chris Cannam
h3. Windows
28 2 Chris Cannam
29 3 Chris Cannam
isatty on Windows returns true if the fd "is associated with a character device (a terminal, console, printer, or serial port)".  (I'm not sure what it means by a terminal -- Windows doesn't seem to have them.)
30 3 Chris Cannam
31 3 Chris Cannam
It may be possible to override the isatty check with a config property as above -- but this doesn't solve things for us on Windows because Python's getpass uses putch and getch to write and read; these are low-level functions that interact directly with the console, not with stderr/stdin.
32 3 Chris Cannam
33 3 Chris Cannam
Options:
34 3 Chris Cannam
35 3 Chris Cannam
# Give the subprocess a console.  QProcess explicitly uses the CREATE_NO_WINDOW flag to CreateProcess which sets no console handle on the subprocess, so we would have to stop using QProcess.  A console in Windows is visible as a window -- there doesn't seem to be an equivalent to a pty -- not ideal.
36 3 Chris Cannam
# Send username/password to the subprocess on the command-line, perhaps using "MercurialKeyring":http://mercurial.selenic.com/wiki/KeyringExtension as well: @--config auth.default.username=plugh --config auth.default.password=plagh --config ui.interactive=false --config extensions.mercurial_keyring=@ (ugh, and not secure enough)
37 3 Chris Cannam
# Use the keyring extension and use the keyring directly beforehand, to set the password (this one, like the previous one, imply that we know in advance whether the password is needed -- perhaps we can check for the authorization required message when running noninteractively).  This means we need to use the same keyring protocol, obviously -- I believe the Hg extension uses "Win32 CryptAPI":http://msdn.microsoft.com/en-us/library/ms947420.aspx via "Python Keyring":http://pypi.python.org/pypi/keyring
38 4 Chris Cannam
# Provide our own Mercurial extension which overrides ui.getpass in a way that is compatible with our purposes (e.g. perhaps showing its own dialog) -- is there really no such thing out there? There must be
39 3 Chris Cannam
# Call the whole thing off and rewrite in Python using the Hg API like any sensible person would have done in the first place.
40 7 Chris Cannam
41 7 Chris Cannam
*Update:* We're taking -- or at least trying -- the approach of providing our own Mercurial extension (as @easyhg.py@), using PyQt for the dialog.  Tested on Linux anyway... We should probably also promote the use of @mercurial_keyring@.