Commit Graph

250 Commits

Author SHA1 Message Date
David Wilson 4d9d21c808 docs: fix typo 2018-03-19 21:58:28 +05:45
David Wilson 4a6d55ced6 docs: vastly simplify importer concurrency docs 2018-03-19 21:58:28 +05:45
David Wilson 984b39180e importer: Beginnings of howitworks section. 2018-03-19 21:58:28 +05:45
David Wilson 4d01dc3ba6 Initial pass at module preloading
* Don't implement the rules for when preloading occurs yet
* Don't attempt to streamily preload modules downstream while this
  context hasn't yet received the final module. There is quite
  significant latency buried in here, but for now it's a lot of work to
  fix.

This works well enough to handle at least the mitogen package, but it's
likely broken for anything bigger.
2018-03-19 21:58:28 +05:45
David Wilson 219718740b Move django test script into permanent test 2018-03-19 21:58:28 +05:45
David Wilson b580ada89f Tidy up and correct the new example 2018-03-19 21:58:28 +05:45
David Wilson eb2aef866b Add shell replacement example. 2018-03-19 21:58:28 +05:45
David Wilson b75e77b410 master: force set_block() in tty_create_child too.
For gevent, just as in 5f7633cd56
2018-03-19 21:58:28 +05:45
David Wilson ed71ae72f8 master: make mitogen minimally functional under gevent
It seems gevent automatically sets blocking behaviour on fds produced by
the socket module, which causes the Python process we fork to fail
horribly. So in the child, always reset the blocking flag.
2018-03-19 21:58:28 +05:45
David Wilson 641498e7f6 docs: Fix function name. 2018-03-19 21:58:28 +05:45
David Wilson ec1de4b4b9 tests: ensure strace is available in the docker image..
..and enable privileged=True since that's needed for ptrace()
2018-03-19 21:58:28 +05:45
David Wilson 247f50e08d master: add comment 2018-03-19 21:58:28 +05:45
David Wilson bbcfc585a8 master: add a comment to explain what's going on, and fix log msg.
Closes #70
2018-03-19 21:58:28 +05:45
David Wilson 3cbe9baf6d issue #70: remove redundant 'raise SystemExit'
Either execvp succeeds, in which case the process image is replaced, or
execvp throws an exception, in which case control flow exits the frame
anyway.
2018-03-19 21:58:28 +05:45
David Wilson b2909ee5f4 Replace nasty print statements in six_brokenpkg. 2018-03-19 21:58:28 +05:45
David Wilson 4f934465b3 Remove overall build timeout from .travis.yml. 2018-03-19 21:58:28 +05:45
David Wilson da8253bab5 Fix broken requires file syntax. 2018-03-19 21:58:28 +05:45
David Wilson 326886832e Add license text everywhere. 2018-03-19 21:58:28 +05:45
David Wilson f1f36cec35 Simplify the API, make Broker optional and auto-shutdown on main thread exit. 2018-03-19 21:58:28 +05:45
David Wilson f077b5b092 issue #56: fix up setup.py, bump version number. 2018-03-19 21:58:28 +05:45
Alex Willmer 7e01c5bed5 Import and use unittest2 without aliasing it 2018-03-19 21:58:28 +05:45
Alex Willmer e8e023ce59 Use specific TestCase assertions throughout tests
e.g. assert x == y -> self.assertEqual(x, y);
self.assertTrue(isinstance(x, y)) -> self.assertIsInstance(x, y)

These specific methods give more useful errors in the case of a test
failure.
2018-03-19 21:58:28 +05:45
Alex Willmer df9556d1d9 Replace custom assertContains() with unitest2 assertIn() 2018-03-19 21:58:27 +05:45
Alex Willmer 17da6e9998 Use unittest2 in all tests
This allows using features such as TestCase,assertIsInstance on Python
2.6 and earlier.
2018-03-19 21:58:27 +05:45
Alex Willmer d743823a5e Require unittest2 2018-03-19 21:58:27 +05:45
David Wilson bbcf1a0bd4 Fix confusing return statements, closes #67. 2018-03-19 21:58:27 +05:45
Alex Willmer 3831ac360f Replace all calls to file() with open()
Although these are synonyms in Python 2.x, when using MyPy to typecheck
code use of file() causes spurious errors.

This commit also serves as one small step to Python 3.x compatibility,
since 3.x removes the file() builtin.
2018-03-19 21:58:27 +05:45
Taihsiang Ho (tai271828) 4e4c9894cf Describe the installation and test process
We use test.sh to consolidate test cases in one file so the README
would not be outdated easier rather than the hard-coded test commands
in a code block.
2018-03-19 21:58:27 +05:45
R W H Bricheno df61f6ab4d Prevent MacOS exception on shutdown of closed socket. 2018-03-19 21:58:27 +05:45
R W H Bricheno 39f325af3e Fix getting the directory path on Mac OS X. 2018-03-19 21:58:27 +05:45
David Wilson c09dcd82a7 Fix Python 3 fix :/ closes #57 2018-03-19 21:58:27 +05:45
Alex Willmer 026669aa93 tests: Fix GetModuleViaPkgutilTest fetching .pyc
On my laptop (Ubuntu 17.10, Python 2.7.14 in a virtualenv),
`test_regular_mod` fails with

```
AssertionError: "\nimport sys\n\n\ndef say_hi():\n    print 'hi'\n" !=
'\x03\xf3\r\n\xbbW\xd5Yc\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00@\x00\x00\x00s\x19\x00\x00\x00d\x00\x00d\x01\x00l\x00\x00Z\x00\x00d\x02\x00\x84\x00\x00Z\x01\x00d\x01\x00S(\x03\x00\x00\x00i\xff\xff\xff\xffNc\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00C\x00\x00\x00s\t\x00\x00\x00d\x01\x00GHd\x00\x00S(\x02\x00\x00\x00Nt\x02\x00\x00\x00hi(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00sF\x00\x00\x00/home/alex/src/mitogen/tests/data/module_finder_testmod/regular_mod.pyt\x06\x00\x00\x00say_hi\x05\x00\x00\x00s\x02\x00\x00\x00\x00\x01(\x02\x00\x00\x00t\x03\x00\x00\x00sysR\x01\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00(\x00\x00\x00\x00sF\x00\x00\x00/home/alex/src/mitogen/tests/data/module_finder_testmod/regular_mod.pyt\x08\x00\x00\x00<module>\x02\x00\x00\x00s\x02\x00\x00\x00\x0c\x03'
```

`__file__` contains the path of the compiled `.pyc`, not the `.py`
source file.
2018-03-19 21:58:27 +05:45
Alex Willmer 77d57a7e38 fakessh: Remove colon causing SyntaxError
fixes #58
2018-03-19 21:58:27 +05:45
David Wilson 658806bde8 fakessh: interruptible wait compatible with <2.7; closes #55 2018-03-19 21:58:27 +05:45
David Wilson b3491b190c master: Lazy-format more logging arguments, minor 3.x compat increments 2018-03-19 21:58:27 +05:45
Alex Willmer 0e7cc55e33 first stage: Remove/dedent else block
Since the above if block ends in a call to os.execv() this block will
only ever run when the if condition was false. Hence putting it in an
else clause is unnecessary.
2018-03-19 21:58:27 +05:45
Alex Willmer d1c10f64a5 first stage: Use string concatenation 2018-03-19 21:58:27 +05:45
Alex Willmer b490cde604 first stage: eliminate whitespace 2018-03-19 21:58:27 +05:45
Alex Willmer c7c0266e99 first stage: single letter variables 2018-03-19 21:58:27 +05:45
David Wilson 8cabba6965 Add first_stage_test to Travis. 2018-03-19 21:58:27 +05:45
David Wilson 32cf2e1a97 Sort .travis.yml 2018-03-19 21:58:27 +05:45
David Wilson 65defaa354 tests: initial first_stage_test. 2018-03-19 21:58:27 +05:45
David Wilson 3d0e8c36d2 issue #49: 2.x/3.x compatible decode.
Much uglier command line, but it works across major Python releases.
2018-03-19 21:58:27 +05:45
David Wilson 038ab04908 docs: convert paragraph to footnote. 2018-03-19 21:58:27 +05:45
David Wilson e3d2c8d649 issue #49: update howitworks.rst for command line change 2018-03-19 21:58:27 +05:45
David Wilson db9bec8720 Implement Alex's awesome idea of zlibbing the first stage too! Closes #49 2018-03-19 21:58:27 +05:45
Alex Willmer 8dcfb1b7f7 testlib: Auto negotiate Docker API version
Ubuntu 17.04 provides Docker 1.12.6, which has API version 1.24.
`dev_requirements.txt` specifies the docker-py 2.5.1, which by default
requests API version 1.30.

Hence when the SSH unit tests try to run the container specified in
`DockerizedSshDaemon` an error occurs

```
APIError: 400 Client Error: Bad Request ("client is newer than server
(client API version: 1.30, server API version: 1.24)")
```
2018-03-19 21:58:27 +05:45
Alex Willmer 94992940dd ssh_test: Correct hostname in SShTest.test_stream_name()
I presume that `u1704` is a relic of a bygone container/host.
2018-03-19 21:58:27 +05:45
Alex Willmer dfc7b85504 testlib: Wait for sshd before running tests
On Ubuntu 17.10 something (probably Docker) appears to be accepting
connections, before sshd is fully ready. This results in a race
condition, and hence connection errors for the first few tests (2-3 on
my laptop).

testlib.wait_for_port() checks not only that the port can be connected
to, but also something resembling the sshd banner is sent.

Fixes #51
2018-03-19 21:58:27 +05:45
David Wilson f8a84616d7 tests: stop responder_test spamming the logs 2018-03-19 21:58:27 +05:45