Commit Graph

2700 Commits

Author SHA1 Message Date
Joran van Apeldoorn cba66953a3
Magisk module onboarding for Android (#5547)
* Added magisk module generation

* Fixed typo

* changelog

* Fixed mypy bug

* Changed action based on ubuntu 18.04 due to https://bit.ly/3QOw87Z

* Workflow pinned to ubuntu 20.04

* Moved magisk code to utils and gen on download

* Styling

* Removed magisk from git repo

* Added tests

* Fixed dead line

* Update CHANGELOG.md

* Hardcoded hash

Co-authored-by: Joran van Apeldoorn <joran@bitsoffreedom.nl>
Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2022-08-23 14:52:11 +00:00
Maximilian Hils 40ac31da02 remove `GetSocket`, simplify transparent mode destination resolution 2022-08-21 23:52:39 +02:00
Maximilian Hils bc370276a1 flask: fix tests
Flask 2.2 has compatibility issues with pytest-asyncio (https://github.com/pallets/flask/issues/4773),
but it turns out we don't even need our tests to be async here. So we just use sync tests to fix the CI issues.
2022-08-17 18:56:04 +02:00
Maximilian Hils f4dc2f2cfd
Replace blinker with custom implementation (#5528)
* replace blinker with custom implementation

The major benefit here is type checking, followed by proper support for async receivers.

* fix compatibility with Python 3.9

* fix nits

* try harder to force gc

* try harderer

* coverage++

* coverage++

* nits
2022-08-16 12:21:55 +00:00
Manuel Meitinger e11626395e
Fix running hook issue and signal proxyserver updates. (#5512)
* add startup hook

* remove StartupHook

* add instance state reports

* make server stop log entries clearer

* fix default param issue

* add SyncSignal and AsyncSignal

* fix signal issues

* simpler update handling

* fix coverage

* minor change to better reflect server update state

* fix reuse tests on linux/wsl

* exterminate `mitmproxy.exceptions.TypeError`

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-08-14 00:11:01 +02:00
Manuel Meitinger fbcffee87f
Include server information in bind/listen errors. (#5495)
* include host and port in listen errors

* common UDP server and reuse message

* update tests

* fix reuse test for master
2022-08-02 19:00:34 +02:00
Miguel Guarniz 05c8ce150e use valid port number in test
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-07-29 19:02:06 +02:00
Miguel Guarniz 5f93227685 update tests after merging DTLS into TLS
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-07-29 19:02:06 +02:00
Miguel Guarniz 544f60c1e6 merge DTLS layer into TLS layer
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-07-29 19:02:06 +02:00
Miguel Guarniz 0860cb1dda remove conflicts and fix lint errors
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-07-29 19:02:06 +02:00
Miguel Guarniz 9cb16c8f6d remove duplicate test
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-07-29 19:02:05 +02:00
Miguel Guarniz 8a175b3754 update multi-server mode tests
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-07-29 19:02:05 +02:00
Miguel Guarniz afaf6fc939 fix coverage failures
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-07-29 19:02:05 +02:00
Miguel Guarniz 70f150203d fix lint errors
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-07-29 19:02:05 +02:00
Miguel Guarniz 5d1c622a65 add UDP layer tests
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-07-29 19:02:05 +02:00
Miguel Guarniz de20be8f62 add tests for Kaitai DTLS ClientHello decoder
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-07-29 19:02:05 +02:00
Miguel Guarniz 709916ef91 add dtls parser tests
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-07-29 19:02:05 +02:00
Miguel Guarniz 7da17c6bd8 fix tests
Signed-off-by: Miguel Guarniz <mi9uel9@gmail.com>
2022-07-29 19:02:05 +02:00
Maximilian Hils 5cee595117
Merge pull request #5481 from mitmproxy/citest
simplify ci build script, add MSIX installer and Microsoft store.
2022-07-28 16:03:11 +02:00
Manuel Meitinger cd4a74fae7
Add support for raw UDP. (#5414) 2022-07-27 02:20:30 +02:00
Maximilian Hils 633130e208
add `mode_specs` docs, remove `@classmethod @property` usage (#5485)
`@classmethod @property` is deprecated in Python 3.11.
2022-07-25 14:16:12 +00:00
Maximilian Hils 3872d33111 simplify ci build script, add MSIX installer and Microsoft store. 2022-07-25 01:22:57 +02:00
Maximilian Hils e2157ecdff fix individual_coverage on Windows 2022-07-21 15:24:11 +02:00
Maximilian Hils 83e543c3e6 add multi proxy mode
This commit makes it possible for mitmproxy to spawn multiple
TCP/UDP proxy servers at the same time, see
https://github.com/mitmproxy/mitmproxy/discussions/5288
2022-07-21 15:24:11 +02:00
Maximilian Hils 3a2b02b420 don't error on connection errors when streaming, fix #5383 2022-06-28 15:53:39 +02:00
Maximilian Hils 87d3d1cb88 pyinstaller: move hooks into package 2022-06-28 14:18:40 +02:00
Denis 82f18f6afb
fix huge (>65kb) http2 responses corrupted (#5430)
* [#5428] fix huge (>65kb) http2 responses corrupted

* [#5428] add tests for huge (>65kb) http2 responses

* add changelog for #5428

* style fix

Co-authored-by: dhabensky <dhabensky@yandex-team.ru>
2022-06-28 11:52:55 +02:00
Maximilian Hils 8c151c66a4
add `asyncio_utils.set_current_task_debug_info` (#5392) 2022-06-03 15:20:25 +02:00
Jixun Wu f02d28ecae
Fix a bug where an excess empty chunk has been sent for chunked HEAD request (#5374)
* fix: don't generate empty chunk for head request (#5372)

* docs: update changlog

* chore: assert for optional self.request.
2022-05-26 13:08:34 +02:00
Manuel Meitinger 8fce7c7fa3
Support specifying the local address for outgoing connections. (#5366)
* allow sockname to specify local-addr

* set local_addr via command line

* minor fix for reconfig

* minor rewording

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2022-05-23 14:54:25 +00:00
rinsuki 400eb3a285
Fix handling of multiple Cookie headers when proxying HTTP/2 to HTTP/1 (#5342)
* Fix handling of multiple Cookie headers when proxying HTTP/2 to HTTP/1

fix #5337

* Add Tests about converts multiple Cookie headers to HTTP/1 from HTTP/2

* Adjust order

* minor code style nits

* Update CHANGELOG.md

* Update _http1.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2022-05-12 19:04:36 +00:00
Maximilian Hils e9176deae9 tlsconfig: add IP address verification test 2022-05-11 16:44:02 +02:00
Maximilian Hils 7afabfe3f7 share `SSL.Context` for all server connections
This should cause not functional differences, but increase performance.
2022-05-11 16:16:16 +02:00
Maximilian Hils b3abf5dc9f
don't crash for HTTP/2 informational responses (#5332) 2022-05-08 14:21:00 +02:00
Robert Xiao c296132d61
Properly encode hostname for upstream requests. (#5325)
* Tests for the fix to #5316

* Encode host part of authority using idna in upstream proxy.

Fixes #5316.
2022-05-06 13:02:49 +02:00
Robert Xiao c562865880
Fix CommandError when loading a new contentview (#5324)
* Console tests: don't mess with the mouse

* Fix CommandError when adding a custom view after startup.

Previously, hot-reloading a script with a custom view after at least one flow
was viewed could throw "CommandError: Not viewing a flow" due to show() being
called while the flow view window was not visible.

Additionally, this patch also refreshes the FlowView when a custom view is
removed.

Fixes #5319.

* Test for the fix to #5319.

* Update CHANGELOG to fix #5319
2022-05-06 07:06:17 +02:00
Maximilian Hils 46f01ad958
fix compatibility with OpenSSL 3 (#5311) 2022-05-01 20:24:32 +00:00
Maximilian Hils 1ef2a350ae formally deprecate `HTTPFlow.mode`
This was non-functioning already anyways, time to deprecate it properly.
2022-04-28 19:36:34 +02:00
Maximilian Hils b3587b52b2 make it black! 2022-04-28 19:11:10 +02:00
Maximilian Hils fdde9ba3b3 use Python 3.9+ typing 2022-04-26 13:53:23 +02:00
Maximilian Hils b31ad66225 `autoflake --in-place **/*.py` 2022-04-26 13:52:46 +02:00
Maximilian Hils e83ec8390a `pyupgrade --py39-plus **/*.py` 2022-04-26 13:51:11 +02:00
Maximilian Hils 2959750124
Merge pull request #5292 from mitmproxy/memoryleak
Fix Memory Leaks
2022-04-25 12:52:43 +02:00
Maximilian Hils e88d5764c3 make `asyncio_utils.create_task` infallible 2022-04-25 11:28:50 +02:00
Maximilian Hils c8d8e303a8 lint 2022-04-25 11:28:50 +02:00
Maximilian Hils 035b3bf37d drop HTTP streams that are completed, fix #4456 2022-04-25 11:28:50 +02:00
Maximilian Hils 56eea20f63 tutils: add BytesMatching placeholder 2022-04-25 11:28:50 +02:00
Maximilian Hils 0600e8b4bc debug signals: stay alive by default 2022-04-25 11:28:50 +02:00
Maximilian Hils 330243f2ab dns: improve web display, coverage++ 2022-04-24 23:09:11 +02:00
Maximilian Hils f6554924ee fix nits 2022-04-24 22:14:10 +02:00
Maximilian Hils f253a65a46 console: test that flow detail views don't crash 2022-04-24 19:06:43 +02:00
Manuel Meitinger ebaacfd850 [dns] additional intercept test 2022-04-24 15:25:07 +02:00
Manuel Meitinger bda488d871 [dns] flowfilter url test 2022-04-24 15:09:06 +02:00
Manuel Meitinger 4229c7c85c [dns] offline tests for proxyserver 2022-04-24 14:56:55 +02:00
Manuel Meitinger dd61b21ce3 [dns] offline dns_resolve tests at 100% coverage 2022-04-24 14:28:11 +02:00
Manuel Meitinger 0553b73d36 [dns] 100% coverage 2022-04-23 23:31:41 +02:00
Manuel Meitinger 99bcfb7f55 [dns] move resolve code into addon 2022-04-23 22:45:51 +02:00
Manuel Meitinger c821d02e09 [dns] update layer tests and bugfix 2022-04-23 21:57:22 +02:00
Maximilian Hils 51b640b600 skip internet tests, fix nits 2022-04-23 19:03:25 +02:00
Maximilian Hils 9c33870ac8 move DNS resolution into own addon, `simple` -> `regular` 2022-04-23 18:49:16 +02:00
Maximilian Hils 0407d9697b proxy modes: use mitmproxy terminology
We consistently use "reverse proxy" and not "forward proxy",
so we should do the same for DNS. Also removed "custom" mode.
If users want such things, the correct approach is to specify a reverse
proxy and set `.response` in the request hook (similar to how it works for HTTP)
2022-04-23 17:05:30 +02:00
Maximilian Hils 2bc43fc10d self-connect: don't resolve
resolving domains here would leak DNS requests if the user aborts,
we don't want that. It's very much a heuristic only anyways.
2022-04-23 16:50:55 +02:00
Maximilian Hils 74f0965a0b `Connection.protocol` -> `Connection.transport_protocol`
This is less ambiguous. Also use a string literal to simplify things.
2022-04-23 16:42:11 +02:00
Maximilian Hils c17e3a16ec dumper: improve dns query display 2022-04-23 16:12:28 +02:00
Maximilian Hils 5c9ab2e774 `str` -> `to_str`
`str` is a bit of a footgun because it overrides the builtin `str()` if imported directly.
2022-04-23 13:34:04 +02:00
Manuel Meitinger 976b1a5f27
Merge branch 'mitmproxy:main' into dns 2022-04-22 22:18:14 +02:00
EndUser509 35703b0b6b
HTTP/2 keepalive (#5144)
* Example addon for saving streamed data including a small bug fix to make it work.

* Revert "Example addon for saving streamed data including a small bug fix to make it work."

This reverts commit 02ab78def9.

* Add https_ping_threshold option to enable keep-alive for HTTP/2 server connections by sending PING frames if the conection is idle longer than the threshold.

* Fixed test

* Fix test

* Adding pragma

* Moved timer logic to _http2.py

* Small code improvement

* Update mitmproxy/options.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>

* Update mitmproxy/options.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>

* Update mitmproxy/proxy/commands.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>

* Update mitmproxy/proxy/commands.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>

* Update mitmproxy/proxy/layers/http/_http2.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>

* Sending Wakup back to right client

* Update mitmproxy/proxy/server.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>

* Update mitmproxy/proxy/server.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>

* Update mitmproxy/proxy/server.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>

* Incorporated suggested changes
Fixed almost all tests

* make `Wakeup` a `CommandCompleted` event.

This allows us to use it with `reply()` in tests,
which makes sure that the correct instance is reused.
 # Please enter the commit message for your changes. Lines starting

* nits

`typing.Set` for Python 3.8 compatibility and a few minor stylistic changes.

* nits nits

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-04-22 13:59:55 +02:00
Manuel Meitinger 5b69087a19 [dns] coverage response codes 2022-04-11 01:42:46 +02:00
Manuel Meitinger 9a55989b25 [dns] refactored enums 2022-04-11 01:38:19 +02:00
Manuel Meitinger 16e04d1c49 [dns] use random test ports 2022-04-10 23:05:29 +02:00
Manuel Meitinger f2c17a7198 [dns] alternate DNS test port 2022-04-10 18:09:49 +02:00
Manuel Meitinger 383dd67a61 [dns] set DNS test port > 1024 2022-04-10 17:57:39 +02:00
Manuel Meitinger ec1f736171 [dns] proxyserver tests 2022-04-10 17:49:19 +02:00
Manuel Meitinger 563f041ba6 [dns] generalize self-connect test 2022-04-10 11:16:13 +02:00
Manuel Meitinger c43dbdc79b [dns] minor fixes 2022-04-09 04:29:11 +02:00
Manuel Meitinger 7acc1b2590 Merge branch 'main' of https://github.com/mitmproxy/mitmproxy into dns 2022-04-09 04:03:23 +02:00
Manuel Meitinger ef3f9e492e [dns] rewrite of udp, merge dnsserver>proxyserver 2022-04-09 03:58:23 +02:00
Maximilian Hils 02d2b6d310
warn users if sse flows are received without streaming, refs #4469 (#5249) 2022-04-06 10:51:58 +00:00
kjy00302 0454f63e98
Fix text truncation for full-width characters (#4278)
* Fix text truncation for full-width characters

* Add test for TruncatedText
2022-04-06 11:17:55 +02:00
Maximilian Hils 6d67a405a9
add `Flow.timestamp_created`, which retains list order on replay (#5227) 2022-04-06 10:34:17 +02:00
Manuel Meitinger ea6f9727da [dns] use snake_case in web flows 2022-04-04 05:29:05 +02:00
Manuel Meitinger b5c115c7a1 [dns] fix simple mode and add tests 2022-04-04 04:30:55 +02:00
Manuel Meitinger 16e7662c87 [dns] domain_names tests 2022-04-04 02:37:33 +02:00
Manuel Meitinger 0a68ff7736 [dns] flowfilter tests 2022-03-31 03:55:16 +02:00
Manuel Meitinger 30cd490c79 [dns] udp reader tests 2022-03-31 01:45:37 +02:00
Manuel Meitinger 9adf715124 [dns] server config test 2022-03-31 00:44:30 +02:00
Manuel Meitinger 2612053415 [dns] layer test full coverage 2022-03-31 00:20:27 +02:00
Manuel Meitinger 993c3e4ee2 [dns] layer test 2022-03-30 23:43:57 +02:00
Manuel Meitinger 4fff28e2a5
Merge branch 'mitmproxy:main' into dns 2022-03-30 20:57:43 +02:00
Maximilian Hils 0fa7c46368
fix cut addon to work with binary content, fix #3965 (#5230) 2022-03-30 12:24:18 +00:00
Manuel Meitinger f1d937bf4b Merge branch 'main' of https://github.com/mitmproxy/mitmproxy into dns 2022-03-29 22:43:08 +02:00
Manuel Meitinger 86adb1a5a0 [dns] temporarily exclude dns.py and more tests 2022-03-29 17:00:13 +02:00
Manuel Meitinger c504bc6eb1 [dns] eventsequence test 2022-03-29 16:25:34 +02:00
Manuel Meitinger fc0bdc143c [dns] view test 2022-03-29 16:16:45 +02:00
Manuel Meitinger 525ac67605 [dns] save test 2022-03-29 16:07:39 +02:00
Maximilian Hils 1bc265ace7
Merge pull request #5226 from mhils/cleanup
`pyupgrade --py38-plus`
2022-03-29 15:36:32 +02:00
Maximilian Hils 9d1e3107e8 `pyupgrade --keep-runtime-typing --py38-plus` 2022-03-29 15:29:37 +02:00
Manuel Meitinger 1ad264781b
Merge branch 'mitmproxy:main' into dns 2022-03-29 15:27:36 +02:00
Maximilian Hils b8bed1d770
errorcheck: print message to stderr for console UI (#5225) 2022-03-29 15:26:46 +02:00
Maximilian Hils f0da667516
tlsconfig: make sure to use the correct connection, fix #5109 (#5224) 2022-03-29 08:27:53 +00:00
Manuel Meitinger 08a9a1be5c [dns] more tests 2022-03-28 17:29:47 +02:00
Manuel Meitinger 4c2365ebeb [dns] first tests 2022-03-27 00:11:48 +01:00
Paulo Romeira e534086053
Add flatpak support to the browser addon (#5200) 2022-03-23 13:56:20 +01:00
Maximilian Hils b06fb6d157 security: reject whitespace in HTTP/1 header names
This commit fixes GHSA-gcx2-gvj7-pxv3 by making mitmproxy
reject header names that contain whitespace characters by default.
A new `validate_inbound_headers` option is provided to turn this behavior
off at the expense of allowing HTTP smuggling vulnerabilities.
2022-03-19 17:20:32 +01:00
Maximilian Hils 9243ba4e25 fix vt code detection on Windows 2022-03-19 17:20:32 +01:00
Alexander Prinzhorn e8ae38c8b6
Fixed encoding guessing: only search for meta tags in HTML bodies (#4566) 2022-03-17 14:25:26 +00:00
Maximilian Hils a9283befad
improve self-connect error message (#5192) 2022-03-17 14:24:42 +00:00
EndUser509 3a5550a09c
Rotate stream files (#5097)
* Example addon for saving streamed data including a small bug fix to make it work.

* Revert "Example addon for saving streamed data including a small bug fix to make it work."

This reverts commit 02ab78def9.

* Add support for rotating stream files every hour or day

* Added tests

* Modified to change the stream file every time the formating string changes as time moves on.

* Update to more compact version

* simplify save addon logic

* make mypy happy

* fix compatibility with Python 3.8

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-17 10:41:05 +00:00
Maximilian Hils ecd4790cbb
vendor `click.style` instead of depending on click (#5188)
click introduces subdependencies like colorama, this approach is much simpler.
2022-03-16 17:02:53 +00:00
Maximilian Hils e2f42ddb30
exit for all tools on startup error, fix #4544 (#5187) 2022-03-16 13:23:32 +00:00
Maximilian Hils 148429c0b3
lowercase user-added HTTP/2 headers, fix #4746 (#5186) 2022-03-16 10:59:30 +01:00
Maximilian Hils 6f0587734e
Refactor how we process `--set` options (#5067)
* refactor how we process `--set` options

* minor improvements based on @marwinxxii's review

* fix nits

* update changelog
2022-03-16 07:33:30 +00:00
Alexander Prinzhorn 8e1adbc5df
Add WebSocketMessage.injected flag (#5105)
* Add WebSocketMessage.injected flag

* add flow format migration

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-16 07:10:23 +00:00
Vinayak Khandelwal 3d5f6da048
ISSUE_5068 (#5161)
* changes for custom port number

* indent correction

* test coverage

* coverage correction

* simplify LDAP auth

* make mypy hapy

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-15 20:39:38 +00:00
Maximilian Hils e22f8db526 add asyncio exception handler test 2022-03-15 19:59:51 +01:00
Maximilian Hils 290ec68aca remove `pytest.mark.asyncio` 2022-03-15 19:50:06 +01:00
Maximilian Hils a7d473c1c1 fix compatibility with Python <3.10 2022-03-15 19:50:06 +01:00
Maximilian Hils 5fc20e3e8c tests++ 2022-03-15 18:43:56 +01:00
Maximilian Hils b491528981 exclude time-sensitive test from CI 2022-03-04 11:38:28 +01:00
Maximilian Hils 372a632161 reintroduce `Flow.live`
We previously relied on the state of `Flow.reply` to check if a flow can be killed,
but this doesn't work anymore with `Flow.reply` being removed. Instead, we now
reintroduce the `Flow.live` attribute, which signals if we are on a live connection.
Killing still is not ideal (see comment in `Flow.killable`), but this paves the way.
2022-02-04 17:30:58 +01:00
Maximilian Hils 4f5930ec5d simplify concurrency tests 2022-02-04 17:30:20 +01:00
Robert Xiao 794c18bec0 Entirely remove the reply system.
The major, breaking change is that it is no longer possible to "take" a reply in
order to block the effect of a later addon hook.

This is patch 4/4 of the reply-ectomy.
2022-02-04 17:30:20 +01:00
Robert Xiao ede269fce4 Flow.intercept: use an Event instead of the reply system
This is patch 3/4 of the reply-ectomy.
2022-02-04 17:30:20 +01:00
Robert Xiao fd43ca19c4 Flow.kill: don't depend on reply status.
In principle, a flow is killable as long as the connection handler is still
checking the error status of the flow.

This is patch 2/4 of the reply-ectomy.
2022-02-04 17:30:20 +01:00
Robert Xiao 4448550746 Rewrite @concurrent to use async instead of the reply mechanism.
This should improve behaviour, since calls to @concurrent will now be serialized
relative to other hooks on the same flow (but will still run in parallel with
hooks on different flows). Unlike a plain async hook, @concurrent allows blocking
sync APIs to run concurrently (e.g. requests).

This is patch 1/4 of the reply-ectomy.
2022-02-04 17:30:20 +01:00
Robert Xiao caf49300c2 Use async for tctx.cycle/tctx.invoke. 2022-02-04 17:30:20 +01:00
Robert Xiao e186ccb3ba fix test.helper_tools.dumperview script.
Previously, it was throwing the following error:

  File mitmproxy/addons/dumper.py, line 133, in _echo_request_line
    elif flow.client_conn.peername:
AttributeError: 'bool' object has no attribute 'peername'
2022-02-04 17:30:20 +01:00
Robert Xiao ee4999e8e4 Rename new async helper functions.
async_trigger -> trigger_event
invoke_addon -> invoke_addon_sync (API breakage)
async_invoke_addon -> invoke_addon
2022-02-04 17:30:20 +01:00
Robert Xiao 687ab4b589 Tests for async hook support 2022-02-04 17:30:20 +01:00
Maximilian Hils 8c86fd06db add changelog linkifier util 2022-02-04 16:00:44 +01:00
Maximilian Hils 53f60c88b1
fix a crash when refreshing headers with a negative unix timestamp, fix #5054 (#5078) 2022-01-18 22:32:38 +00:00
Maximilian Hils d47fd3e9c3
ClientHello: add `.raw_bytes()`, refs #4877 (#5075) 2022-01-18 11:34:09 +00:00
Maximilian Hils ef8c88da1f
setup websocket connection before finishing HTTP response, fix #5017 (#5020) 2021-12-27 15:58:24 +00:00
Maximilian Hils 3fbf3cf8ee tlsconfig: don't overwrite existing TLS context, refs #5019 2021-12-27 12:22:08 +01:00
Maximilian Hils c74806feac
fix tests 2021-12-20 21:06:14 +01:00
Maximilian Hils 6997129bc0
make sure that `running()` is only invoked once on startup. (#4964)
fix #3584
2021-11-27 13:11:23 +00:00
Maximilian Hils 2dd845ed95 catch malformed cert warning in tests
we may need to catch this properly in `get_cert` at some point,
let's see if this ever turns out to be an issue.
2021-11-22 12:55:17 +01:00
Maximilian Hils 3cb87f5a2f split `tls_handshake` hook into client/server and success/fail variants 2021-11-22 10:23:21 +01:00
Maximilian Hils a72f61ef57 Merge remote-tracking branch 'origin/main' into ignore-after-clienthello 2021-11-22 09:54:08 +01:00
shindexro 9a469806eb
quote argument of view.flows.resolve (#4910)
* Fix #4902

* Update type signature

* Switch to None check

* Fix spacing

* Quote view.flows.resolve argument

* Switch to call_strings
2021-11-19 13:04:20 +01:00
shindexro 888ce66f90
Correct flow-detail documentation (#4909)
* Fix #4902

* Update type signature

* Switch to None check

* Fix spacing

* Skip URL shortening when flow-detail is 0

* Sync docs with implementation

* Update URL shortening test
2021-11-18 08:26:05 +01:00
shindexro 39fa242e25
fix crash when invoking `replay.server.count` from console (#4905)
fix #4902
2021-11-17 16:53:43 +00:00
Peter Hoffmann 260fc68211
Fix #4876 Don't do CONNECT on plaintext HTTP replays via upstream (#4882)
* Replays via upstream also need to comply with upstream handling

* Adjusted test for HTTP upstream replay which should NOT do a CONNECT

* Added Changelog

* Test for replay https pver upstream with CONNECT

* Proxy requests use full URL with host & port

* Finally remove some prints

* lint!

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-10-31 20:23:04 +00:00
mame82 3a884ceb8f
gRPC packed repeated encoding (#4872)
* Replaced Kaitai protobuf parser with custom implementation, to prepare packed data handling

* Clamp varint size for bool conversion to 64bit, to satify tests

* moved WireParser into ProtoParser

* preserve work state

* Full packed support

* noc changes to example addon

* Adjusted test regex for ValueError

* Do not try to unpack fields twice
2021-10-21 13:20:13 +02:00
Maximilian Hils 79f464bc78
ignore superfluous newlines after HTTP CONNECT, fix #4870 (#4871) 2021-10-19 11:25:22 +00:00
mame82 8b6345e4a7
gRPC view: Adjusted parsing strategy for deprecated protobuf fields (#4858)
* Adjusted parsing strategy for deprecated protobuf fields

* Added test comment

* adjusted comment to reflect correct field IDs
2021-10-13 13:16:49 +02:00
mame82 480052f58b
Grpc contentview (#4851)
* Partial gRPC contentview prototype, not linted, no tests, not as add-on

* Linted (flake8)

* Save dev state

* Rewrote of protobuf parser, use decoding strategy, reduced rendered data. Parser uses  generators

* minor cleanup

* fix: preferred encoding was provided as function instead of value

* flake8: line length

* Backlinked message tree objects, temporary debug out

* Partial implementation of gRPC definitions. Save state to fix a cras (data invalidate in edit mode)

* hack: deal with missing exception handling for generator based content views

* gRPC/Protoparser descriptions (with test code)

* replaced manual gzip decoding with mitmproxy.net.encoding.decode

* Refactored typing imports

* Reafctoring

* distinguish request vs response definitions, separate view config from parser config

* Code cleaning, moved customized protobuf definitions to example addon

* final cleanup

* changelog

* Stubs for tests

* Fixed render_riority of addon example

* Started adding tests

* Work on tests

* mypy

* Added pseudo encoder to tests, to cover special decodings

* Example addon test added

* finalized tests, no 100 percent coverage possible, see comments un uncovered code

* minor adjustments

* fixup tests

* Typos

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2021-10-12 13:32:56 +02:00
Maximilian Hils aad92c9d5a
Merge pull request #4847 from mhils/flowfilter
Flowfilter Improvements
2021-10-09 18:38:56 +02:00
Maximilian Hils e3640801a5 lint! 2021-10-09 18:34:29 +02:00
Maximilian Hils 25bdf2f9d8 web: update filters 2021-10-08 18:39:22 +02:00