Commit Graph

407 Commits

Author SHA1 Message Date
Aldo Cortesi 9d0e3c8d61 Doc and help adjustments. 2011-08-03 19:15:01 +12:00
Aldo Cortesi 028d5bacc5 Make "C" clear eventlog when it has focus. 2011-08-03 17:41:13 +12:00
Aldo Cortesi e337682d8e Enable "|" command to run a oneshot script on a single flow. 2011-08-03 17:35:18 +12:00
Aldo Cortesi cfc6e8777e Add script set/unset shortcut for mitmproxy. 2011-08-03 17:14:11 +12:00
Aldo Cortesi e3196dac4d Move commands around to make space for "s" script shortcut.
New commands are:

"w" - save all flows
"W" - save this flow
"S" - server replay
2011-08-03 16:52:41 +12:00
Aldo Cortesi 179cf75862 Add script hooks, enable new engine for mitmdump. 2011-08-03 16:36:20 +12:00
Aldo Cortesi f7e4e89b12 Move the event notification mechanism into flow.py 2011-08-03 13:33:18 +12:00
Aldo Cortesi 12d2b1f926 Rip out old script interface, start replacing with new stubs.
Scripts are broken for now.
2011-08-03 13:20:36 +12:00
Aldo Cortesi 62088a6661 Start stubbing out a much more powerful script architecture. 2011-08-03 11:06:29 +12:00
Aldo Cortesi a817db5bd6 Refresh current connection when toggling autodecode.
Also fix the unit tests I forgot to commit...
2011-08-02 20:47:53 +12:00
Aldo Cortesi 8cc0469ee7 Tweak encoding behaviour
- Don't fail to identity encoding when an unknown encoding is specified.
- Don't constrain encodings. I want to try to modify traffic as little as
possible by default.
- When decoding, delete content-encoding header rather than set it to "identity"
- Refuse to decode/encode when there is an existing but unknown
content-encoding header.
2011-08-02 20:42:46 +12:00
Aldo Cortesi bb6ec29b18 Fix encoding import crash. 2011-08-02 16:55:54 +12:00
Aldo Cortesi 1ff6a767d0 Unit test++ 2011-08-02 16:52:47 +12:00
Aldo Cortesi 357502fe03 General cleanup.
Cut out unused variables and code, generally shut up pychecker as much as is
reasonable.
2011-08-02 16:14:33 +12:00
Aldo Cortesi 17835b9b78 Fix a rare undefined variable crash in proxy.py. 2011-08-02 15:43:35 +12:00
Aldo Cortesi a1456742a8 Make ConnectionList key bindings work even if there are no entries. 2011-08-02 15:35:54 +12:00
Aldo Cortesi f3742f29da We no longer need to track clientconnections. 2011-08-02 14:56:09 +12:00
Aldo Cortesi f3f8462ddc Make the mitmproxy eventlog display useful information. 2011-08-02 14:17:15 +12:00
Aldo Cortesi 73a7d893e3 Give a visual indication that the eventlog has focus. 2011-08-02 11:02:23 +12:00
Aldo Cortesi 759f5d71a6 Initial key bindings and event handlers for event log. 2011-08-02 10:48:29 +12:00
Aldo Cortesi af92153974 Start stubbing out a UI for the eventlog in mtimproxy. 2011-08-02 09:17:54 +12:00
Aldo Cortesi 9b398c03ab Exit with error if mitmproxy can't load a file specified on cmdline. 2011-08-01 13:27:46 +12:00
Aldo Cortesi 675b3133b4 Improve performance of loading flows from a file hugely.
Fell into the "expensive __eq__ method" trap. Oh, Python, you little scamp.
2011-08-01 11:26:09 +12:00
Aldo Cortesi 43f1c72511 Refactor the way we calculate views of the flow list.
The naive approach we used before recalculated the view on every access, and
consequently had serious performance problems.
2011-08-01 11:17:01 +12:00
Aldo Cortesi ddb5748a76 Add decoding/encoding for requests. 2011-08-01 10:43:01 +12:00
Aldo Cortesi c89c4361c3 Merge remote-tracking branch 'alts/encoding' 2011-07-28 11:19:07 +12:00
Stephen Altamirano 78049abac1 Changes replace logic to function in both Python 2.6.x and 2.7.x
Tests now only assume Python 2.6.x rather than requiring 2.7.x. This does not preclude the use of flags as a kwarg in replace
2011-07-26 22:47:08 -07:00
Stephen Altamirano c1eaa9f74c Adds encode and decode methods to Response objects 2011-07-26 22:03:41 -07:00
Aldo Cortesi e6288e2d07 Fix crash when sticky cookies are read from file.
Cookielib expects strings, not unicode.
2011-07-24 16:08:27 +12:00
Aldo Cortesi 0f4ae61e7d Fix a crash in mitmdump event display. 2011-07-23 16:59:48 +12:00
Aldo Cortesi 6cd32bf96f Unbreak mitmproxy. Oops. 2011-07-23 13:39:17 +12:00
Aldo Cortesi 3648c7953a Extend eventlog information.
Also, squash an SSL-related bug revealed by the extended logging.
2011-07-23 13:37:06 +12:00
Aldo Cortesi 4043829cf2 Add an eventlog option to mitmdump
This shows client connections, disconnections and requests (before a complete
flow is assembled). We need to add an analogous display to mitmproxy.
2011-07-23 12:57:54 +12:00
Aldo Cortesi 689f5f0d1f Don't turn off output if -v flag is passed more than twice. 2011-07-23 11:50:30 +12:00
Aldo Cortesi 47e1695512 Also replace strings path for requests. 2011-07-22 20:52:13 +12:00
Aldo Cortesi 6ce8b49e05 Make script pipe globally available. 2011-07-22 19:09:32 +12:00
Aldo Cortesi 1b961fc4ad Add utility functions to search and replace strings in flows
This is a common task in pentesting scenarios. This commit adds the following
functions:

utils.Headers.replace
proxy.Request.replace
proxy.Response.replace
flow.Flow.replace
2011-07-22 17:48:42 +12:00
Stephen Altamirano 9c24401b18 Removes last_encoding attribute from Response. Prompts for encoding on identity responses 2011-07-21 22:09:48 -07:00
Stephen Altamirano 74d8b18408 Removes should_autodecode attribute from Response. Adds commandline option 'd' to toggle autodecode, adds togglable option 'd' to do the same 2011-07-21 20:22:13 -07:00
Aldo Cortesi 5936a48e59 Drop cert expiry time to avoid a bug in some OpenSSL versions. 2011-07-22 11:11:45 +12:00
Stephen Altamirano aa7f8ac90b Switches hotkeys. En/decode is now bound to 'z', kill connection now 'X' 2011-07-20 00:14:24 -07:00
Stephen Altamirano ebfa9b2a5d Fixes issue #10 regarding broken json printing 2011-07-19 10:51:49 -07:00
Stephen Altamirano 25b0631190 Switches hotkey to unused 'g', adds help message 2011-07-18 22:04:23 -07:00
Stephen Altamirano 1c5434d72c Adds ability to toggle between encodings in the response view 2011-07-18 21:52:40 -07:00
Stephen Altamirano ecd4645988 Adds encode counterparts to decode functions 2011-07-17 20:16:47 -07:00
Aldo Cortesi b0849387b7 Add explicit notice when data has been auto-decoded. 2011-07-18 14:18:47 +12:00
Aldo Cortesi 669ce8ee7c Correctly detect urlencoded data
This broke when we introduced case preservation for headers.
2011-07-17 15:31:58 +12:00
Aldo Cortesi 6df4be93e3 Fix error in anticomp commandline specification. 2011-07-17 14:36:38 +12:00
Aldo Cortesi f756d3bec1 Make help display for options nicer. 2011-07-17 11:14:18 +12:00
Aldo Cortesi 1559ded009 Expose the anticompression flag as an option through the "o" key in mitmproxy. 2011-07-17 10:34:43 +12:00
Aldo Cortesi ce41046786 Refine encoding support
- Push decoding down into the LRU cache
- Cope gracefully with corrupted data and incorrect encoding headers
2011-07-17 10:25:25 +12:00
alts 6dc0f105cc Adds support for content encoding, namely gip and deflate 2011-07-16 02:47:06 -07:00
Aldo Cortesi 94ae720a22 Add a pretty-printing mode for urlencoded form data. 2011-07-15 16:46:54 +12:00
Aldo Cortesi 76b4c6ba82 Introduce an anti-compression command-line argument.
This is on by default, which means we avoid compressed content unless the -z
flag is specified.
2011-07-15 15:24:56 +12:00
Aldo Cortesi 1a963b91bb Don't sort headers when displaying them in mitmproxy. 2011-07-14 17:20:32 +12:00
Aldo Cortesi 7e21ac0eb8 Refine path completion somewhat.
Make it match the behaviour of vim and mutt more closely
2011-07-14 16:54:04 +12:00
Aldo Cortesi 1c9e7b982a Rewrite Headers object to preserve order and case. 2011-07-14 16:01:54 +12:00
Aldo Cortesi b6e1bf63c3 Merge branch 'master' of github.com:cortesi/mitmproxy 2011-07-11 16:14:32 +12:00
Felix Wolfsteller 1a5b157c8f Specify certificate creation waiting time argument to fix type error in proxy.py . 2011-07-09 08:07:22 +00:00
Aldo Cortesi 65fbb7bd0d Bail out if no command was specified after | shortcut. 2011-07-08 21:41:00 +12:00
Aldo Cortesi 8e176c2086 Cast some data read from dump files to str, to prevent unicode promotion.
This fixes a bug that caused a traceback when de-serialized requests were
replayed. Also adds unit tests for the problem.
2011-07-01 14:20:42 +12:00
Aldo Cortesi 2a90ea69fd Show view mode in statusbar.
Also make "m" keyboard shortcut available globally.
2011-06-30 14:49:11 +12:00
Aldo Cortesi 37c8d3425d Fix edit prompt display, return code editing. 2011-06-30 14:30:48 +12:00
Aldo Cortesi 18d4c3a9e9 JSON pretty-printing.
Also rename the display modes ("pretty" instead of "indent"), and expand the
built-in documentation.
2011-06-30 13:27:27 +12:00
Aldo Cortesi 46ec8f52e7 Prep for 0.5 release
- Update CHANGELOG and CONTRIBUTORS
- Bump version
- Include Apple Gamecenter highscore setting tutorial in docs
2011-06-27 16:38:00 +12:00
Aldo Cortesi 0a642f2441 Make the certificate wait time configurable.
Since OpenSSL doesn't let us set certificate start times in the past, the
client and proxy machine time must be synchronized, or the client might reject
the certificate. We can bodgy over small discrepancies by waiting a few seconds
after a new certificate is generated (i.e. the first time an SSL domain is contacted).

Make this a configurable option, and turn it off by default.
2011-06-27 16:10:17 +12:00
Aldo Cortesi f004326855 Try not to hang when user views large request & response bodies
Two different strategies here:

    - Use a simple heuristic to detect if we're looking at XML data when indent
    mode is used. On non-XML data we can hang even on small documents.

    - Only view partial data for large bodies. At the moment the cutoff is
    100k. I might finetune this later.
2011-06-27 15:59:17 +12:00
Aldo Cortesi 2ae7808ca9 Don't redraw the screen more often than necessary. 2011-06-27 14:01:08 +12:00
Aldo Cortesi b04d074341 Repair a problem that sometimes caused SSL connections to peg the CPU. 2011-06-23 17:00:55 +12:00
Aldo Cortesi 0d9e0eac9a Don't backup flows before replay.
This lets us revert to the original request, even after replaying an edit.
2011-06-23 14:47:34 +12:00
Yuangxuan Wang e56793f01e Fix urwid version parsing error when it's something like 0.9.10-pre 2011-06-20 16:18:55 +08:00
Aldo Cortesi 7d7803a4d9 Add a hideous kludge to fix not-yet-valid certificates.
- The OpenSSL x509 has no way to explicitly set the notBefore value on
certificates.

- If two systems have the same configured time, it's possible to return a
certificate before the validity start time has arrived.

- We "solve" this by waiting for one second when a certificate is first
generated before returning the cert. The alternative is to rewrite pretty much
all of our certificate generation, a thought too horrible to contemplate.
2011-06-11 15:16:16 +12:00
Aldo Cortesi 07110bbbf1 Anticache and refresh_server_playback options are applied before flows are loaded.
You can now use mitmdump to preview how these options work, by running mitmdump
against a set of saved flows, and viewing the output.
2011-05-15 12:23:34 +12:00
Aldo Cortesi e285b17e3f Add -r option to mitmdump and mitmproxy.
This option reads a set of flows from a file. I've also regularized the
mitmdump and mitmproxy command-line signatures by removing mitmproxy's old way
of specifying flow loads through naked arguments.
2011-05-15 11:54:12 +12:00
Aldo Cortesi 613e9a298e Add a new flow loading mechanism.
We now simulate the normal connection flow when we load flows. That means
that we can run scripts, hooks, sticky cookies, etc.
2011-05-15 11:22:35 +12:00
Aldo Cortesi 6175d92583 Minor code cleanup - no need to recreate the master queue. 2011-05-14 12:12:03 +12:00
Aldo Cortesi f89581be1b Add a -n option which tells the tools not to bind a proxy.
This is useful when you just want to inspect or process dumps.
2011-05-14 10:44:25 +12:00
Aldo Cortesi d917cfd916 Refactoring. 2011-03-29 10:57:50 +13:00
Aldo Cortesi c3105153a5 Add some debugging output to help troubleshoot a performance problem. 2011-03-27 13:10:06 +13:00
Aldo Cortesi c0bd1a39e4 unit test coverage ++ 2011-03-20 18:52:16 +13:00
Aldo Cortesi c726519e73 Add a stickyauth option.
This allows us to replay an HTTP Authorization header, in the same way as we
replay cookies using stickycookies. This lets us conveniently get at HTTP Basic
Auth protected resources through the proxy, but is not enough to do the same
for HTTP Digest auth. We'll put that on the todo list.
2011-03-20 17:31:54 +13:00
Aldo Cortesi 4f877cde6a Reverse order of flows in mitmproxy.
It matches user expectations much better to have new flows appended to the bottom.
2011-03-20 09:31:39 +13:00
Aldo Cortesi e22fd74d06 Revamp key generation.
We now create three different files in the .mitmproxy directory when a dummy CA
is made:

mitmproxy-ca.pem - the CA, including private key

mitmproxy-ca-cert.p12 - A pkcs12 version of the certificate, for distribution to Windows.

mitmproxy-ca-cert.pem - A PEM version of the certificate, for distribution to everyone else.
2011-03-18 16:45:31 +13:00
Aldo Cortesi 3fbf343985 Tweak CA and cert setup to be nice to Windows.
For some reason Satan's Operating System doesn't join up the certification path
if the key identifiers are set to hash. This took a few hours of trial and
error to figure out.
2011-03-18 14:48:43 +13:00
Aldo Cortesi 907536503c Enable request and response script commandline arguments for mitmproxy. 2011-03-18 10:43:43 +13:00
Aldo Cortesi fc9e0dcacb Maintain focus in mitmproxy when flows are loaded from file. 2011-03-18 10:33:32 +13:00
Aldo Cortesi 0e62dd479b Do id-based comparison rather than value-based comparison to establish flow focus.
This fixes a bug where focus would jump unpredictably between identical flows.
2011-03-18 10:21:59 +13:00
Aldo Cortesi 894d3cc62d Use path_prompt rather than plain prompt where needed in mitmproxy. 2011-03-18 10:14:57 +13:00
Aldo Cortesi 89b0ac5195 Server replay shortcut in mitmproxy can now be used to interrupt a current replay. 2011-03-18 09:43:47 +13:00
Aldo Cortesi f97c144869 Client playback shortcut in mitmproxy can now be used to interrupt a current client playback. 2011-03-18 09:38:51 +13:00
Aldo Cortesi 4893e5e5a4 We have to pass -CAcreateserial after all. 2011-03-18 09:24:04 +13:00
Aldo Cortesi e983253ecc Docs, minor cert tweaks. 2011-03-18 09:04:49 +13:00
Aldo Cortesi 394bd1d6b0 Interrupt interception when deleting an intercepting flow.
Prompting the user for this is annoying.
2011-03-15 17:53:29 +13:00
Aldo Cortesi e3ad3790de Fix minor styling and glitches. 2011-03-15 17:37:09 +13:00
Aldo Cortesi 6d5c32ad4b Fix hang on shutdown. 2011-03-15 17:21:35 +13:00
Aldo Cortesi fb28e71f0b Do pre-processing of requests before replay.
This enables scripts, anticache, server playback and sticky cookies for
request replays.
2011-03-15 15:11:03 +13:00
Aldo Cortesi fe1e2f16ff Improve responsiveness of request and response viewing.
- Computing the view of a large body is expensive, so we introduce an LRU cache
to hold the latest 20 results.

- Use ListView more correctly, passing it individual urwid.Text snippets,
rather than a single large one. This hugely improves render time.
2011-03-15 13:05:33 +13:00
Aldo Cortesi bb8adfb731 Use .stop() and .start() to restart curses around external spawns. 2011-03-14 16:09:06 +13:00
Aldo Cortesi 974cb03631 Merge branch 'master' of github.com:cortesi/mitmproxy 2011-03-14 15:56:52 +13:00
Aldo Cortesi 9b961a8236 Use raw_display instead of curses_display. 2011-03-14 15:56:25 +13:00
Aldo Cortesi c17cf5c6a0 Fix traceback when viewing hex mode. 2011-03-14 14:34:55 +13:00
Aldo Cortesi 897bd5c2b8 We no longer use pytz. 2011-03-14 13:47:51 +13:00
Aldo Cortesi b4fab58c61 Indicate request and response replay status separately. 2011-03-13 21:53:26 +13:00
Aldo Cortesi 6cb40d03f4 Fix a number of small UI infelicities.
- Make request/response view switching behave sensibly
- Avoid having an empty string appear in connection view text. This makes urwid
misbehave.
- Make it clear that intercept and cookies specifications are filters.
2011-03-13 21:16:42 +13:00
Aldo Cortesi 038a6eb1a7 I've now added all the features I want for 0.4. Bump the version number. 2011-03-13 20:42:55 +13:00
Aldo Cortesi 1e47167ac6 Fix a traceback sometimes triggered viewing a connection with no response. 2011-03-13 20:03:13 +13:00
Aldo Cortesi 8d0152f2ff Add server replay to mitmproxy. 2011-03-13 17:11:59 +13:00
Aldo Cortesi 324d7c3955 Add client plaback to mitmproxy. 2011-03-13 16:50:11 +13:00
Aldo Cortesi 49c1532af7 Add anticache support for mitmproxy
Also stub out an option toggle command for anticache, killextra and norefresh
options.
2011-03-13 16:21:41 +13:00
Aldo Cortesi ef27e2fb29 Rejigger status bar somewhat. Add indicators for various bits of state.
In particular, we now display limits, sticky cookie expressions and intercepts
in the bar.
2011-03-13 15:55:47 +13:00
Aldo Cortesi 18d25ec1a1 Fix stickycookie setting, indented view traceback. 2011-03-13 11:43:55 +13:00
Aldo Cortesi 0042687f0d Remove beep functionality from mitmproxy.
This acts funny (sometimes works, sometimes doesn't) and isn't really useful.
2011-03-13 11:26:30 +13:00
Aldo Cortesi c901cbbada mitmproxy prompted input now display previously set value.
E.g. if you set a limit, then re-enter the limit prompt, you start with the
currently set value.
2011-03-13 11:24:49 +13:00
Aldo Cortesi 4ce393cc41 Unit tests for cmdline.py 2011-03-12 15:14:25 +13:00
Aldo Cortesi 96b10ca486 console.py 100% test coverage 2011-03-12 15:08:06 +13:00
Aldo Cortesi fbbc43a5b2 netstring.py 100% unit test coverage. 2011-03-12 14:39:41 +13:00
Aldo Cortesi 2f457e041e Unify mitmproxy and mitmdump commandline
- Extract common options into cmdline.py
- Change mitmproxy keybindings to fit command line

Some cmdline options and keybindings aren't in operation yet - just stubs
where functionality will be added in the next few commits.
2011-03-12 14:30:12 +13:00
Aldo Cortesi 40e9067b83 Handle invalid bind address specifications gracefully. 2011-03-12 13:47:37 +13:00
Aldo Cortesi 9f16a84a9e Make mitmdump handle invalid serialized data gracefully. 2011-03-11 15:16:31 +13:00
Aldo Cortesi 7d85db0da3 Display Errors and killed connections in mitmdump. 2011-03-11 13:06:51 +13:00
Aldo Cortesi daa9653ebe Add --norefresh to stop refreshing server playback to mitmdump.
Also, make cookie parsing for refreshing more error-tolerant.
2011-03-11 11:56:10 +13:00
Aldo Cortesi e99b1d1949 Stub out refresh for server-side replay. 2011-03-09 20:05:30 +13:00
Aldo Cortesi 03f1345385 Add an --anticache option to mitmdump.
This removes all headers that might cause a server to return 304-not-modified.

For now, all the new features are going into mitmdump - everything will be
ported over to mitmproxy once I have the feature set locked down.
2011-03-09 13:15:31 +13:00
Aldo Cortesi 592812467d Move version constants to version.py, add version to serialized flows.
For now, we don't do anything with the serialized version info, but this should
give us some future-proofing for serialization format changes.
2011-03-09 12:36:14 +13:00
Aldo Cortesi 897c4bfc52 Think harder about timestamps. Just save seconds since the epoch as a float. 2011-03-09 12:18:08 +13:00
Aldo Cortesi 765871bd11 Store timestamps on flow components as a UTC time tuple.
Format is:

(tm_year,tm_mon,tm_mday,tm_hour,tm_min, tm_sec,tm_wday,tm_yday,tm_isdst)
2011-03-07 13:46:02 +13:00
Aldo Cortesi 7330f5fccf Make mitmdump server playback also exit by default.
Like client playback, the --keepserving option makes mitmdump keep serving.
2011-03-06 17:08:56 +13:00
Aldo Cortesi ec00b5a66e Make mitmdump exit after client replay is complete by default.
Add an option --keepserving to make it keep serving after replay.
2011-03-06 16:54:49 +13:00
Aldo Cortesi e794cbc0d8 Add flow_count and active_flow_count methods to flow.State.
Use these in unit tests where sensible.
2011-03-06 16:11:45 +13:00
Aldo Cortesi 42ffded626 Bump unit tests, rearrange mitmdump command-line options slightly. 2011-03-06 16:02:28 +13:00
Aldo Cortesi 4f02480482 Allow client replays to be replayed against server replays.
Useful for testing.
2011-03-06 14:48:16 +13:00
Aldo Cortesi bad77f3470 Add client replay options to mitmdump. 2011-03-06 11:21:31 +13:00
Aldo Cortesi 5da4099ddf Massage namespace to make room for client replay.
Mostly replay -> server_replay
2011-03-05 13:03:26 +13:00
Aldo Cortesi 96471fde1d Basic client playback state structure. 2011-03-04 13:08:43 +13:00
Aldo Cortesi 6921b9ff2a Add an indicator that sticky cookies have been applied in mitmdump. 2011-02-25 21:23:44 +13:00
Aldo Cortesi 8cade9fbbf Move stringification funcs from proxy to dump. 2011-02-25 21:11:44 +13:00
Aldo Cortesi 3792b0084e Add -c and -C options to mitmdump to control sticky cookies.
It's dumb that this needs to be two different options, but optparse doesn't
support optional arguments. It would be much nicer to just have "-c" for "all",
and "-c filter" for a specified filter.
2011-02-25 17:32:03 +13:00
Aldo Cortesi 57f96c5fe0 Minor housekeeping - unused code, nocover directives.
95% test coverage.
2011-02-24 15:44:08 +13:00
Aldo Cortesi 79039eb5d2 More mature sticky cookie primitive. Use it in console.py. 2011-02-24 15:26:34 +13:00
Aldo Cortesi 57947b328e Start abstracting out sticky cookie state. 2011-02-24 10:33:39 +13:00
Aldo Cortesi 3c1db00ebb Remove first iteration playback/record.
Bonus: unit test coverage goes from 70% to 94% with one commit. ;)
2011-02-23 12:40:30 +13:00
Aldo Cortesi 39207ffdd2 Add a way for users to specify header significance in server replay.
Also add the --rheader command-line option to mitmdump to let the user specify
an arbitrary number of significant headers. The default is to treat no headers
as significant.
2011-02-23 10:54:51 +13:00
Aldo Cortesi c80214ba55 Minor housekeeping: bump version, clean todo. 2011-02-21 12:27:01 +13:00
Aldo Cortesi 7bc913c40d Move script hooks into the flow primitives.
This lets handle scripts in corner cases like playback situations more easily.
2011-02-21 11:40:49 +13:00
Aldo Cortesi fe99871df8 Add --kill option to mitmdump
If this option is passed all requests that are not part of a replayed
conversation are killed. If the option is not passed, such requests are passed
through to the server as usual.
2011-02-21 11:08:35 +13:00
Aldo Cortesi c3e3897071 Fix a subtle Unicode problem in Response.assemble
If msg is Unicode, the proto string is automatically promoted to Unicode.  If
the proto string is promoted to Unicode, then the FMT interpolation is also
done in Unicode. If this happens, then binary data in content will cause an
exception.
2011-02-21 10:11:50 +13:00
Aldo Cortesi fd4dd8cb6b First pass of playback function for mitmdump. 2011-02-21 09:54:39 +13:00
Aldo Cortesi deb79a9c5a Add a simple server playback state object.
We use a loose hash to match incoming requests with recorded flows. At the
moment, this hash is over the host, port, scheme, method, path and content of
the request. Note that headers are not included here - if we do want to include
headers, we would have to do some work to normalize them to remove variations
between user agents, header order, etc. etc.
2011-02-21 08:47:19 +13:00
Aldo Cortesi aa16194518 Clean up and strip down netstrings module. 2011-02-20 14:03:32 +13:00
Aldo Cortesi 7ddba22f51 Certificates are now generated in a temporary per-session directory.
This means that certificates don't accumulate in the conf directory, users
don't have to clear certificates if the CA is regenerated, and the user can
specify a custom CA without invalid certificates being loaded inadvertently.
2011-02-20 13:29:41 +13:00
Aldo Cortesi c2ae8285f4 Revamp SSL configuration.
- Move option parsing utiliities to proxy.py

- Don't have a global config object. Pass it as an argument to ProxyServer.

- Simplify certificate generation logic.
2011-02-20 12:53:42 +13:00
Aldo Cortesi 4fc807cedd Clean up certificate generation.
- Use templates for config files. We can re-introduce customization of the
certificate attributes when we need them.

- Split CA and cert generation into separate functions.

- Generation methods provide an error return when generation fails.

- When the user explicitly specifies a certificate, we don't generate it, but
fail if it doesn't exist.
2011-02-20 12:17:10 +13:00
Aldo Cortesi 9c5c3c2b1a Implement state loading that doesn't change object identity.
We need this to let us load state from copied Flows returned from scripts.
2011-02-20 09:36:13 +13:00
Aldo Cortesi 58fc0041fa Stub out doc structure, add screenshots for configuring certs in Firefox. 2011-02-19 19:43:44 +13:00
Aldo Cortesi 1549ec8079 Bump test coverage to 100% for flow.py 2011-02-19 17:21:08 +13:00
Aldo Cortesi d9c6be3d13 Fix serialization when a Request has no associated client connection. 2011-02-19 17:03:44 +13:00
Aldo Cortesi 5da27a9905 Refactor Flow primitives to remove HTTP1.0 assumption.
This is a big patch removing the assumption that there's one connection per
Request/Response pair. It touches pretty much every part of mitmproxy, so
expect glitches until everything is ironed out.
2011-02-19 17:00:24 +13:00
Aldo Cortesi cd4eea3934 First pass of script hooks for mitmdump.
Also stub out docs, improve mitmdump error handling.
2011-02-18 12:40:45 +13:00
Aldo Cortesi 7769e5a898 Whitespace in mitmdump output. 2011-02-17 15:56:54 +13:00
Aldo Cortesi afa9104005 Refine verbose mitmdump output. 2011-02-17 15:10:50 +13:00
Aldo Cortesi cbe6982c6d Improve mitmdump text output. 2011-02-17 14:26:50 +13:00
Aldo Cortesi ca4df062bf Whitepsace and extraneous code. 2011-02-17 11:45:12 +13:00
Aldo Cortesi f028dc7e1b Add filtering to mitmdump. 2011-02-17 10:44:08 +13:00
Aldo Cortesi f0f1fb4b55 Add file writing to mitmdump. 2011-02-17 10:18:38 +13:00
Henrik Nordstrom 7758385ac1 Extract common SSL certificate option processing 2011-02-16 15:50:44 +01:00
Aldo Cortesi d9374ff97b Extract common SSL certificate options into a group.
Use this only in mitmdump and mitmproxy for now.
2011-02-16 23:09:42 +13:00
Aldo Cortesi f5511350eb Tweaks and cleanups for mitmdump. 2011-02-16 23:03:46 +13:00
Aldo Cortesi 8a9f20b78c Don't leave dangling browser connections for pipelined requests. 2011-02-16 22:37:04 +13:00
Aldo Cortesi 0dd1aa7cc7 Initial port of mitmdump to Flows. 2011-02-16 22:10:24 +13:00
Aldo Cortesi 692556cf20 Fix minor display issues in console app. 2011-02-16 21:03:55 +13:00
Aldo Cortesi 3f0b84bb49 Un-break request replay. 2011-02-16 19:37:40 +13:00
Aldo Cortesi f009770d4c Fix a bug in HTTP 1.1 pipelining that caused Requests to be over-written.
We use the ClientConnection object to tie requests, responses and errors
together. This is an HTTP 1.0 assumption, but we can fix it by just making
copies of the connection object when we handle multiple requests.
2011-02-16 19:22:19 +13:00
Aldo Cortesi 66349c9783 FlowMaster bugfixes and unit tests. 2011-02-16 16:43:35 +13:00
Aldo Cortesi 27f0765cdd Extract flow-specific Master operations into FlowMaster. 2011-02-16 16:03:22 +13:00
Aldo Cortesi 33dac700fa Switch over to new serialization format.
Remove BSON from contrib.
2011-02-16 15:10:00 +13:00
Aldo Cortesi 0c6f846861 First draft of the new serialization mechanism. 2011-02-16 14:33:04 +13:00
Aldo Cortesi 5692c7359c Import Will McGugan's netstring module.
Module is in the Public Domain.

I expect to modify and extend this module, so I've imported into main library
rather than contrib. Code has been reformatted to suite our code standard,
tests have been extrated into /tests directory.
2011-02-16 11:21:06 +13:00
Henrik Nordstrom 89627a702a drop unused protocol parameter from Response 2011-02-15 22:24:38 +01:00
Aldo Cortesi 759496e24e Repair unit test suite. 2011-02-11 11:06:30 +13:00
Henrik Nordstrom 51cdd7b95d Bump version 2011-02-10 02:59:51 +01:00
Henrik Nordstrom 32adee8743 Implement a dummy CA 2011-02-10 02:59:51 +01:00
Henrik Nordstrom 4ffaadd435 Allow specifying the accepted ciphersuites 2011-02-10 02:59:51 +01:00
Henrik Nordstrom 061cea89da Import cache store control into console controller 2011-02-10 02:59:51 +01:00
Henrik Nordstrom d11dd742d8 Simple record & playback functionality 2011-02-10 02:59:51 +01:00
Henrik Nordstrom 4bae297fbb Basic HTTP/1.1 Support
Adds support for chunked transfer encoding, and a couple other minor
protocol corrections.

Improve HTTP support

- Support intercepted requests with Host header
- Support HEAD requests proper
- Support any HTTP method including extensions, not just a couple known ones

Support expect: 100-continue and 100 Continue messages

Persistent client connections

Generalize ServerConnection a bit in preparation for keep-alive support

Correct HTTP status codes on errors forwarding the request
2011-02-10 02:59:51 +01:00
Henrik Nordstrom 0613321aef Optimize CONNECT responses, sent in a single packet 2011-02-10 02:59:51 +01:00
Henrik Nordstrom 4cf3392e50 Sort header names for a predictable result 2011-02-10 02:59:51 +01:00
Henrik Nordstrom bd01126b2b Move try_del to utils 2011-02-10 02:59:51 +01:00
Henrik Nordstrom 2d8cfbafc3 Automatically update response message when editing response code 2011-02-10 02:59:51 +01:00
Henrik Nordstrom 98b1acc51d Add response creation to edit function on intercepted requests 2011-02-10 02:59:51 +01:00
Henrik Nordstrom fcc39e1aaf Terminate workers when main thread terminates 2011-02-10 02:59:51 +01:00
Aldo Cortesi eb15ef7b80 Merge remote branch 'hno/master' 2011-02-06 18:44:35 +13:00
Aldo Cortesi a4eaafab5a Further fine-tuning for pretty_xmlish. 2011-02-06 16:56:13 +13:00
Aldo Cortesi 7156d1a73a Rip out BeautifulSoup, and use a custom XML-ish prettyprinter. 2011-02-06 14:17:30 +13:00
Aldo Cortesi 44dc3a052e Add option to tell mitmproxy which interfaces to bind to. 2011-02-06 10:52:54 +13:00
Aldo Cortesi d0c2d9480c Also serialize ClientConnection and flow backups. 2011-02-06 10:28:43 +13:00
Henrik Nordstrom b35cbc0760 Fix kill() again, got broken by the change to use None 2011-02-03 23:39:28 +01:00
Henrik Nordstrom 5bf6482bd0 Merge remote-tracking branch 'origin/master' 2011-02-03 23:30:03 +01:00
Henrik Nordstrom 2f813fa748 Messages may need to stay in the controller for considerable
amount of time. Don't use a timeout waiting for the controller response.
Example where this happens is intercepted flows.
2011-02-03 23:28:24 +01:00
Aldo Cortesi 2ad4c5adf3 Get rid of ReplayConnection - we now have only one ClientConnection class. 2011-02-04 10:58:08 +13:00