a4127fb6d5
Add a tick event for addons and scripts, and use it for race-free reload
2016-07-14 19:54:15 +12:00
deffed2196
Script cleanup: editing in console, Python3 compatibility fixes
2016-07-14 19:54:15 +12:00
b94f5fd361
Convert examples and example tests for new-style scripts
...
Remove the test that just loads all the example scripts for now - it's a very
low-value test, and we need to think of something better.
2016-07-14 19:54:15 +12:00
a6821aad8e
Zap old scripts infrastructure, fix concurrency tests
2016-07-14 19:54:15 +12:00
a3a22fba33
First-order integration of scripts addon
2016-07-14 19:54:15 +12:00
1266255842
Re-add deletion of stickycookies ( #1355 )
...
This was originally added in #1324 (fixing issue #1096 )
but got lost during "the big refactor" (#1352 )
2016-07-14 00:37:27 -07:00
f84a23f11d
minor fixes
2016-07-13 22:01:16 -07:00
ec6fbe9eb6
make options explicit
2016-07-13 21:45:25 -07:00
b2c4f301cb
Stream to file -> addon
...
This commit also clarifies a confusion about the "outfile" attribute and its
use in testing in the mitmdump master.
2016-07-14 13:52:09 +12:00
703c05066e
Fix indeterminacy in sticky cookie tests
...
How has this not bitten us before?
2016-07-14 12:59:00 +12:00
00509d86a8
StickyCookies to addon
2016-07-14 12:13:14 +12:00
143bf0dfa9
AntiCache to addon
2016-07-14 11:24:06 +12:00
97b2e6d958
Add a .setter helper to Options
...
This returns a function that sets a named attribute, and is handy for
event-driven code like mitmproxy console.
2016-07-14 11:11:41 +12:00
a8a083a10d
Move StickyAuth into addon
...
This is the first addon that handles an OptionsError, so this commit also demos
how this works in console. Handling of command-line erorrs is on its way.
2016-07-14 10:08:41 +12:00
255e1eb00b
Add the addons mechanism
...
Demonstrate how it works and interacts with Options by adding our first new
builtin: anticomp.
2016-07-13 23:45:49 +12:00
7f8fd3cdff
Basic outline of addons
...
Add addons.py, integrate with our event mechanism, and change the Master API so
options is the first init argument.
2016-07-13 23:26:04 +12:00
a20f8e9620
More powerful Options scheme
...
This prepares us for the addon configuration mechanism and gives us a more
flexible way to handle options changes. This changeset should spell the end of
the current anti-pattern in our codebase where we duplicate data out of options
onto the master when mutability is needed. From now on, Options can be the one
source of thruth.
- Change notifications
- Rollback on error
2016-07-13 18:45:50 +12:00
3579c6dd43
http2: fix race-condition in tests
2016-07-12 19:55:28 +02:00
7c09255298
http2: more coverage
2016-07-10 20:07:43 +02:00
b611997619
http2: simplify test class
2016-07-10 18:53:00 +02:00
7eade1ef7c
Remove redundant tests
2016-07-10 01:08:02 +05:30
39f5108400
Test cookies.is_expired separately
2016-07-10 01:06:50 +05:30
6a746deff5
Add tests for deletion of stickycookies
2016-07-09 22:20:25 +05:30
83a1cc5a9a
Make escape_control_characters handle strings on Py2
2016-07-09 13:28:20 +05:30
c7cbbf5f0d
Set verbosity & flow_detail to 0
2016-07-09 12:32:41 +05:30
147f61fa62
py3++
2016-07-09 11:14:12 +05:30
15c10d49f6
Merge pull request #1327 from mitmproxy/remove-script-context
...
Remove script context
2016-07-08 20:11:12 -07:00
5d2b7c52f9
move script context to mitmproxy.ctx
2016-07-08 19:57:57 -07:00
f7639e077a
py3++
2016-07-08 15:58:12 -07:00
05ab9c1b37
Updated coding style and modified flow format compat test to use v0.11 and v0.10 dump files
2016-07-08 15:03:36 -07:00
7c67faa8da
remove script contexts
2016-07-07 23:50:55 -07:00
c048ae1d5b
remove context from all scripts
2016-07-07 20:14:52 -07:00
00dce24015
tests++
2016-07-07 18:02:59 -07:00
76473d44e0
py3++
2016-07-07 17:50:26 -07:00
f62e976e1e
py3++
2016-07-07 17:29:22 -07:00
d962bd4e83
py3++
2016-07-07 20:25:48 +02:00
c870d26bc7
py3++: test_protocol_http2
2016-07-07 16:39:25 +02:00
f259b9478f
disable harparser on py3
2016-07-07 01:21:15 -07:00
64ac2c73fe
Merge pull request #1318 from mhils/tnetstring3-unicode-type
...
[tests actually pass] Add unicode to tnetstring3
2016-07-07 09:48:22 +02:00
275cf7a6c2
Detect JSON API media type as a json content view
...
IANA assignment here:
https://www.iana.org/assignments/media-types/application/vnd.api+json
More about JSON API:
https://github.com/json-api/json-api
2016-07-06 21:10:35 -07:00
64a867973d
sni is now str, not bytes
2016-07-06 21:03:17 -07:00
8287ce7e6d
fix tests
2016-07-06 19:55:48 -07:00
d406bee988
tnetstring3: adapt to unicode support
2016-07-06 19:55:48 -07:00
684b4b5130
tnetstring: keys are str on py3. migrate flow.io_compat
2016-07-06 19:55:48 -07:00
9c873d63f4
py3++, multidict fixes
...
This commit improves Python 3 compatibility and fixes two multidict
issues:
1. Headers.items(multi=True) now decodes fields
2. MultiDict.clear(item) has been removed, as Python's
MutableMapping already defines .clear() with different
semantics. This is confusing for everyone who expects a
dict-like object. `.pop("attr", None)` is not fantastic,
but it's the Python way to do it.
2016-07-06 19:53:29 -07:00
444f0a4c39
py3++
2016-07-06 17:31:08 -07:00
e84f5fcc72
Merge pull request #1315 from mitmproxy/better-unicode-handling
...
remove clean_bin, clarify unicode handling
2016-07-05 16:02:37 -07:00
00a35aa895
Merge pull request #1297 from dufferzafar/py3-flow-export
...
Python 3 - test_flow_export
2016-07-05 15:17:18 -07:00
d51cf543bb
remove clean_bin, clarify unicode handling
2016-07-05 15:11:32 -07:00
5fecb8c843
http2: test fatal connection termination
2016-07-05 22:15:57 +02:00
8d2042ea8a
http2: test throttling at MAX_CONCURRENT_STREAMS
2016-07-05 22:15:57 +02:00
a6b3551934
raise ValueError if content-encoding is invalid
2016-07-04 13:58:09 -07:00
45aa2174e2
http2: Remove TestReadRequestConnect test
2016-07-03 21:14:01 +05:30
f623b3d99b
py3++: test_flow_export
2016-07-02 21:15:20 +05:30
393c1d74af
fix lint issues
2016-07-02 18:56:22 +05:30
2f8a1fd2cb
tests++
2016-07-02 03:03:42 -07:00
d9f797e7e6
make the linter happy
2016-07-02 02:11:00 -07:00
6032c4f235
message.content -> .raw_content, implement .text
...
This PR improves our handling of HTTP message body encodings:
- The unaltered message body is now accessible as `.raw_content`
- The "content-encoding"-decoded content (i.e. gzip removed) content
is not `.content`, as this is what we want in 99% of the cases.
- `.text` now provides the "content-encoding"-decoded and then
"content-type charset"-decoded message body.
- The decoded values for `.content` and `.text` are cached,
so that repeated access and `x.text = x.text` is cheap.
- The `decoded()` decorator is now deprecated, as we can now just use
`.content`. Similarly `HTTPMessage.get_decoded_content()` is
deprecated.
2016-07-02 01:51:47 -07:00
2c09e0416b
minor fixes
2016-07-01 17:49:18 -07:00
fa72462798
fix tcp message handling
2016-07-01 17:17:16 -07:00
536c7acd13
py3++
2016-07-01 14:10:48 -07:00
268e42e55e
py3++: test/mitmproxy/test_server
2016-07-01 10:23:04 -07:00
f7d4319c5a
minor fixes
2016-06-28 23:48:19 -07:00
3b53d3e393
fix mysterious tox issue
2016-06-28 19:26:08 -07:00
5d0de16f57
http2: improve error handling in tests
2016-06-22 22:09:57 +02:00
c0ba47094d
py3++: mitmproxy.protocol.http1
2016-06-22 13:37:41 +05:30
0a53550955
mitmproxy.tnetstring -> mitmproxy.contrib.tnetstring
2016-06-21 23:53:59 +05:30
10f4a36a5a
Imported tests for tnetstring
...
Copied from
a808f6b5ba/tests/test_format.py
2016-06-21 23:47:36 +05:30
44abb4caea
Merge pull request #1276 from dufferzafar/console-pathedit
...
mitmproxy.console tests - PathEdit
2016-06-20 20:36:31 -07:00
5b95803be0
py3++
2016-06-20 21:57:18 +02:00
efeade134a
py3++
2016-06-20 21:39:39 +02:00
6aeba9777a
Add dummy directory to completion
2016-06-20 01:24:27 +05:30
ba13fda10d
Add tests for PathEdit
2016-06-20 00:50:41 +05:30
7bde38dc4a
py3++
2016-06-19 12:00:37 +02:00
eb3ed87100
move custom HTTP/2 stack from netlib to pathod
2016-06-17 15:06:01 +02:00
2a0e850615
pathod tests: do not string_escape specs
2016-06-16 17:08:21 -07:00
6782b4ec00
Py3: Store certificate with a byte key
...
Had to debug this for hours!
2016-06-16 17:08:06 -07:00
ea5576238f
Py3: Use BytesIO in a pathoc test
2016-06-16 17:08:06 -07:00
f399a5cd29
Py3: Fix tests by using byte literals
2016-06-16 17:08:06 -07:00
52fa195204
Unexpected spaces around =
2016-06-15 09:49:39 +02:00
f83433e674
Py3: pathoc
2016-06-15 09:49:38 +02:00
fa40531a80
Py3: pathod
2016-06-15 09:49:38 +02:00
c2d0910e59
Make har_extractor.py output HAR 1.2 spec-compliant
...
HAR files were failing to load in harviewer (http://www.softwareishard.com/har/viewer/ ) due to:
- ISO 8601 dates for startedDateTime missing timezone (http://www.softwareishard.com/blog/har-12-spec/#pages )
-- Used UTC but could add detection of default system timezone if desired
- pages object missing pageTimings (http://www.softwareishard.com/blog/har-12-spec/#pageTimings )
-- Used {} because all child fields are optional
2016-06-14 12:47:10 -04:00
a63817f2b8
Merge pull request #1253 from mhils/fix-debug
...
Fix Debug Mode on Windows
2016-06-14 17:12:02 +12:00
93276d45be
Merge branch 'master' into netlibrace
2016-06-14 16:29:15 +12:00
4c292b0197
Fix Python3 tests
2016-06-14 16:13:50 +12:00
50a20fefee
Satisfy lint
2016-06-14 14:34:30 +12:00
9e63350a96
Use handlers in http2 test suite
2016-06-14 14:32:18 +12:00
e6fd98bb72
Start rolling out leak detection in netlib
2016-06-14 14:32:18 +12:00
63c6660895
update examples, tests, docs
2016-06-13 18:17:09 -07:00
3c9a70b761
fix debug mode on Windows
2016-06-13 18:06:44 -07:00
9bea616441
Be stricter about handling connetcts in the pathoc test suite
...
Attempt to clear a niggling Appveyor exception buggering up our CI.
2016-06-12 10:39:19 +12:00
7b86560ded
debug: On SIGUSR2, we dump tracebacks for all threads to screen
2016-06-11 23:07:42 +12:00
09edbd9492
Improve debugging of thread and other leaks
...
- Add basethread.BaseThread that all threads outside of test suites should use
- Add a signal handler to mitmproxy, mitmdump and mitmweb that dumps resource
information to screen when SIGUSR1 is received.
- Improve thread naming throughout to make thread dumps understandable
2016-06-11 19:52:24 +12:00
5b9f07c81c
debug.sysinfo: tests and coverage
2016-06-11 17:56:17 +12:00
2cf79b7912
Remove timestamps from pathoc output
...
Pathoc is an interactive tool, no need for a long leading timestamp. More
generally, make timestamps optional in the logging mechanism so we can
configure this with command-line flags or something down the track.
2016-06-10 10:47:28 +12:00
c421c41307
Remove odict
...
- Adds default implementations for _kconv and _reduce_values to MultiDict.
Without these, operations fail in really, really non-obvious ways.
- Replace the remaining few instances of ODict
Fixes #1159
2016-06-09 13:28:43 +12:00
b3b4156c2f
Py3: Fix test_language_http2 tests by using byte literals
2016-06-08 16:15:54 +05:30
6b03df2633
Py3: Use global next() instead of iterator method
2016-06-08 16:15:54 +05:30
0280af9522
Py3: Use BytesIO instead of StringIO
2016-06-08 16:15:54 +05:30
a5cb241c7c
If a message has been acked, all other processors are skipped
...
This applies the constraint, but does to clumsily. When we've unified modules
and processors it will be much nicer. We also make some exceptions for the
master processors that we may want to re-evaluate down the track.
2016-06-08 12:58:58 +12:00
b3bf754e53
Simplify script concurrency helpers
...
We now have take() to prevent double-replies.
2016-06-08 11:21:38 +12:00
a388ddfd78
A new interface for reply
...
Reply is now explicit - it's no longer a callable itself. Instead, we have:
reply.kill() - kill the flow
reply.ack() - ack, but don't send anything
reply.send(message) - send a response
This is part of an incremental move to detach reply from our flow objects,
and unify the script and handler interfaces.
2016-06-08 10:44:20 +12:00
982077ec31
Add reply.ack and reply.kill
2016-06-08 10:14:34 +12:00
db11fe0087
Merge branch 'master' into beautiful-json
2016-06-08 09:19:25 +12:00
c98c83190b
tests++
2016-06-07 01:03:08 -07:00
d254fe791a
Py3: Fix a test_language_writer test by using a byte literal
2016-06-07 00:01:41 -07:00
ba466dbb35
simplify pretty_json, improve test coverage
2016-06-06 23:08:58 -07:00
1ffc273c94
Utils cleanups
...
- Move more stuff that belongs in netlib.human
- Move some stuff to near the only use
- Zap mitmproxy.utils.timestamp(). I see the rationale, but we used it
interchangeably with time.time() throughout the project. Since time.time()
dominates in the codebase and timestamp() is such low utility, away it goes.
2016-06-07 17:12:52 +12:00
80292c0ac3
fix all pathod tests on Windows 🎉
2016-06-06 16:03:45 -07:00
2ee5e8ef0e
fix NestedMessage encoding
2016-06-06 13:39:38 -07:00
6447c8ae22
fix .freeze(), improve tests
2016-06-06 13:31:44 -07:00
f48073af56
Py3: Fix status_code and other tests by using byte literals
2016-06-06 11:52:45 -07:00
5a2932adc1
Py3: Fix corresponding base.Integer test in test_language_base
2016-06-06 11:52:45 -07:00
40056e1cd7
Py3: Fix useragent tests
2016-06-06 11:52:45 -07:00
f5a6b295f7
Py3: Fix websocket tests
2016-06-06 11:52:45 -07:00
b86ef9fd04
Py3: Use BytesIO in tutils.render
2016-06-06 11:52:45 -07:00
bafce6cfe9
Py3: Use byte string literals
2016-06-06 11:51:36 -07:00
9a19540f8b
Py3: Use BytesIO instead of StringIO for tests
2016-06-06 11:51:36 -07:00
12dd6f6707
Py3: Use global next() instead of iterator method
2016-06-06 11:51:36 -07:00
ced6fb4cf9
Py3: Fix tests by using byte literals
2016-06-06 11:48:58 -07:00
efb0f7133c
Py3: Use global next() instead of iterator method
2016-06-06 11:48:54 -07:00
210c376901
Fix sequencing in Pathoc websocket termination test
2016-06-06 18:17:22 +12:00
9458eaece7
Make more of the pathod test suite use synchronisation
2016-06-06 16:05:44 +12:00
08344ee38b
Merge pull request #1215 from redfast00/master
...
Now keeps header capitalization
2016-06-06 08:45:35 +12:00
c31b9c461d
Merge pull request #1211 from cortesi/pathod
...
WIP: Radical webectomy of pathod
2016-06-06 08:40:20 +12:00
25505733bc
fix multidict unittests
2016-06-05 21:20:37 +02:00
435bfeca0b
Use BytesIO instead of StringIO
2016-06-06 00:36:15 +05:30
7b6da98719
Py3: Use byte literals for comparison
2016-06-06 00:07:44 +05:30
be306c8439
Py3: Use global next() function instead of iterator method
2016-06-06 00:07:44 +05:30
375d7c9741
Remove last vestiges of noapi and noweb from pathod
...
Also hide HTTP2 freeze bug by making explain configurable in the tests
2016-06-05 13:24:46 +12:00
48da24ae7e
First-order removal of pathod API and app
2016-06-05 13:04:13 +12:00
c0c45c051a
Fix test that may fail due to binary header data
...
This is just inherently not a determinisitc test. We don't use the log HTTP
interface any more, so it can just go. A more radical "solution" is inbound
shortly.
Fixes #1207
2016-06-05 11:47:52 +12:00
6ffe8b376e
Py3: Write bytes to file
2016-06-04 18:28:50 +05:30
0d4437a404
Fix function name typo
2016-06-04 18:28:03 +05:30
bd7b275d44
Py3: Convert all data to bytes in pathod.language.writer test
2016-06-04 15:23:09 +05:30
614a3d7a54
Py3: Use global next() instead of iterator method
2016-06-04 15:23:09 +05:30
d5532319c1
Basic ConsoleMaster tests, based on mastertester
2016-06-04 19:07:27 +12:00
af63a3e064
console/__init__.py -> console/master.py
2016-06-04 16:06:44 +12:00
12a4f9ecf9
Very basic test truss for web.app
2016-06-04 15:30:28 +12:00
f7f9cab5dc
Rebase on master
2016-06-04 14:37:07 +12:00
e60860e65d
Make tcp.Client.connect return a context manager that closes the connection
2016-06-03 12:06:36 +12:00
c5076f5e01
Implement a service connection handler counter, use it in Pathod test suite
...
Lots of failures, but that's a good thing.
2016-06-03 11:47:07 +12:00
c6d15904f3
Merge remote-tracking branch 'upstream/master' into testsuite
2016-06-03 10:19:22 +12:00
29c52d98dc
Merge remote-tracking branch 'duffer/pathod-lang-generators'
2016-06-02 09:53:22 -07:00
5fd52970fd
Satisfy linting gods
2016-06-02 19:55:52 +12:00
c9dd574f48
Silence requests warnings
...
Once we convert the rest of the suite to pathoc, this can go.
2016-06-02 19:40:39 +12:00
d8c52964c7
Reimplement test retrievals on pathoc and ditch requests
...
Requests uses urllib3, which has a connection pool that's not easy to disable
(https://github.com/shazow/urllib3/issues/383 ).
2016-06-02 19:36:02 +12:00
b510616c69
Py3: Return bytes from FileGenerator
...
and use __getitem__ instead of __getslice__
2016-06-02 13:00:44 +05:30
a04d7fd166
Py3: Return bytes from RandomGenerator
...
and use __getitem__ rather than __getslice__
2016-06-02 13:00:02 +05:30
Aldo Cortesi