From 7e3380e6283b97d53fe852fe55c0474648f2320b Mon Sep 17 00:00:00 2001 From: Jurrie Overgoor <1213142+Jurrie@users.noreply.github.com> Date: Mon, 13 Feb 2023 17:07:24 +0100 Subject: [PATCH] Add a section on using Magisk and Magisk modules in the Android howto (#5924) * Add a section on using Magisk and Magisk modules in the Android howto This allows you to use Google Play builds with mitmproxy. * [autofix.ci] apply automated fixes * Explain how to get the Magisk module from mitmweb --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> --- CHANGELOG.md | 2 + ...howto-install-system-trusted-ca-android.md | 78 +++++++++++++++++-- 2 files changed, 73 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 39d55eab7..9f330a4d1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,8 @@ ([#5908](https://github.com/mitmproxy/mitmproxy/pull/5908), @sujaldev) * Fix editing of multipart HTTP requests from the CLI. ([#5148](https://github.com/mitmproxy/mitmproxy/issues/5148), @mhils) +* Added documentation on using Magisk module for intercepting traffic in Android production builds. + ([#5924](https://github.com/mitmproxy/mitmproxy/pull/5924), @Jurrie) ### Breaking Changes diff --git a/docs/src/content/howto-install-system-trusted-ca-android.md b/docs/src/content/howto-install-system-trusted-ca-android.md index b3bf5576e..f18bc6202 100644 --- a/docs/src/content/howto-install-system-trusted-ca-android.md +++ b/docs/src/content/howto-install-system-trusted-ca-android.md @@ -16,7 +16,7 @@ Please note, that apps can decide to ignore the system certificate store and mai - [Android Studio/Android Sdk](https://developer.android.com/studio) is installed (tested with Version 4.1.3 for Linux 64-bit) - An Android Virtual Device (AVD) was created. Setup documentation available [here](https://developer.android.com/studio/run/managing-avds) - - The AVD must not run a production build (these will prevent you from using `adb root`) + - AVD production builds (those labeled with "Google Play") will prevent you from using `adb root`. You need to use [the Magisk method]({{< ref "#instructions-when-using-magisk" >}}) if you need Google Play installed. - The proxy settings of the AVD are configured to use mitmproxy. Documentation [here](https://developer.android.com/studio/run/emulator-networking#proxy) - Emulator and adb executables from Android Sdk have been added to $PATH variable @@ -45,10 +45,70 @@ By default, the mitmproxy CA certificate is located in this file: `~/.mitmproxy/ ## 3. Insert certificate into system certificate store -Now we have to place our CA certificate inside the system certificate store located at `/system/etc/security/cacerts/` in the Android filesystem. By default, the `/system` partition is mounted as read-only. The following steps describe how to gain write permissions on the `/system` partition and how to copy the certificate created in the previous step. +Now we have to place our CA certificate inside the system certificate store located at `/system/etc/security/cacerts/` in the Android filesystem. By default, the `/system` partition is mounted as read-only. The following steps describe how to gain write permissions on the `/system` partition and how to copy the certificate created in [the previous step]({{< ref "#2-rename-certificate" >}}). -### Instructions for API LEVEL > 28 - Starting from API LEVEL 29 (Android 10), it seems to be impossible to mount the "/" partition as read-write. Google provided a [workaround for this issue](https://android.googlesource.com/platform/system/core/+/master/fs_mgr/README.overlayfs.md) using OverlayFS. Unfortunately, at the time of writing this (11. April 2021), the instructions in this workaround will result in your emulator getting stuck in a [boot loop](https://issuetracker.google.com/issues/144891973). Some smart guy on Stackoverflow [found a way](https://stackoverflow.com/questions/60867956/android-emulator-sdk-10-api-29-wont-start-after-remount-and-reboot) to get the `/system` directory writable anyway. +### Instructions when using Magisk +If you want to use a production build (labeled "Google Play"; it's those builds that have Google Play installed) you can use Magisk to obtain root in your AVD. +[Magisk](https://github.com/topjohnwu/Magisk) allows root on your Android device or emulator. + +See the [instructions here](https://github.com/shakalaca/MagiskOnEmulator) for installing Magisk on your AVD. +The instructions have been tested with API level 30, but are reportedly working with API levels 22 up to and including 30 and 'S' (except API level 28). +Note: the instructions say to start your AVD. Do not supply an `-http-proxy` directive to mitmproxy at this point. + +When you are done with that, your emulator will allow root. You can check this by running a terminal emulator and typing `su`. +Magisk should ask you if you want to grant root to the program. After granting this, typing `whoami` would display `root`. + +However, after you have installed Magisk, you can no longer start your emulator with `-writable-system`. It will cause a boot loop. (Start your AVD with `-show-kernel` to see the error.) +But you can install your mitmproxy certificate by putting it in a Magisk module, and installing that module. +Magisk will take care of copying your certificate to `/system/etc/security/cacerts/` during boot. + +#### Downloading the Magisk module from mitmweb +If you run mitmweb, you can get simply download the Magisk module instead of handcrafting it. +Stop your AVD, and start it again with `-http-proxy 127.0.0.1:8080` (or whatever IP and port combination you are running mitmweb's proxy on). + +Then, *inside* the AVD, start a browser and navigate to `http://mitm.it/cert/magisk`. +You will be prompted to download `mitmproxy-magisk-module.zip`, which is the Magisk module you need. Store that file somewhere (like in 'Downloads'). + +Then open up Magisk, click on `Modules` and install your module. + +Reboot your AVD. + +#### Creating the Magisk module containing your certificate +If you do not run mitmweb, you'll need to create a Magisk module yourself. +See [here](https://topjohnwu.github.io/Magisk/guides.html#magisk-modules) for in-depth information on Magisk modules, but basically it boils down to this: + +Create the following directories: +- `mitmproxycert` (this will be the root of your module) +- `mitmproxycert/com/google/android` +- `mitmproxycert/system/etc/security/cacerts` + +Place your renamed certificate from [step 2]({{< ref "#2-rename-certificate" >}}) inside `mitmproxycert/system/etc/security/cacerts` and `chmod 664` it. + +Save the content of [https://github.com/topjohnwu/Magisk/blob/master/scripts/module_installer.sh](https://github.com/topjohnwu/Magisk/blob/master/scripts/module_installer.sh) as a local file `update-binary` and place it inside `mitmproxycert/com/google/android`. + +Create a file named `updater-script` containing only the string `#MAGISK` and place it inside `mitmproxycert/com/google/android`. + +Create a file named `module.prop` and place it inside `mitmproxycert`. The file should contain something like: + +``` +id=mitmproxycert +name=MITM proxy certificate +version=1 +versionCode=1 +author=mitmproxycert +description=My shiny MITM proxy certificate to reveal all secrets and obtain world domination! +``` + +Zip the module using something like `cd ./mitmproxycert ; zip -r ./../mitmproxycert.zip ./` and push it to your running AVD using `adb push ./../mitmproxycert.zip /storage/emulated/0/Download/`. + +The go to your AVD, open up Magisk, click on `Modules` and install your module (you'll find it in the Downloads folder). + +Reboot your AVD. + +### Instructions for API LEVEL > 28 using `-writable-system` +By default, the `/system` partition is mounted as read-only. The following steps describe how to gain write permissions on the `/system` partition and how to copy the certificate created in chapter 2. + +Starting from API LEVEL 29 (Android 10), it seems to be impossible to mount the "/" partition as read-write. Google provided a [workaround for this issue](https://android.googlesource.com/platform/system/core/+/master/fs_mgr/README.overlayfs.md) using OverlayFS. Unfortunately, at the time of writing this (11. April 2021), the instructions in this workaround will result in your emulator getting stuck in a [boot loop](https://issuetracker.google.com/issues/144891973). Some smart guy on Stackoverflow [found a way](https://stackoverflow.com/questions/60867956/android-emulator-sdk-10-api-29-wont-start-after-remount-and-reboot) to get the `/system` directory writable anyway. **Keep in mind:** You always have to start the emulator using the `-writable-system` option if you want to use your certificate. Otherwise Android will load a "clean" system image. @@ -62,11 +122,11 @@ Tested on emulators running API LEVEL 29 and 30 - reboot device: `adb reboot` - restart adb as root: `adb root` - perform remount of partitions as read-write: `adb remount`. (If adb tells you that you need to reboot, reboot again `adb reboot` and run `adb remount` again.) - - push your renamed certificate from step 2: `adb push /system/etc/security/cacerts` + - push your renamed certificate from [step 2]({{< ref "#2-rename-certificate" >}}): `adb push /system/etc/security/cacerts` - set certificate permissions: `adb shell chmod 664 /system/etc/security/cacerts/` - reboot device: `adb reboot` -### Instructions for API LEVEL <= 28 +### Instructions for API LEVEL <= 28 using `-writable-system` Tested on emulators running API LEVEL 26, 27 and 28 @@ -76,6 +136,10 @@ Tested on emulators running API LEVEL 26, 27 and 28 - Start the desired AVD: `emulator -avd -writable-system` (add `-show-kernel` flag for kernel logs) - restart adb as root: `adb root` - perform remount of partitions as read-write: `adb remount`. (If adb tells you that you need to reboot, reboot again `adb reboot` and run `adb remount` again.) - - push your renamed certificate from step 2: `adb push /system/etc/security/cacerts` + - push your renamed certificate from [step 2]({{< ref "#2-rename-certificate" >}}): `adb push /system/etc/security/cacerts` - set certificate permissions: `adb shell chmod 664 /system/etc/security/cacerts/` - reboot device: `adb reboot` + +### Testing that your certificate is loaded from the system certificate store + +In your AVD, go to Settings → Security → Advanced → Encryption & credentials → Trusted credentials. Find your certificate (default name is `mitmproxy`) in the list.