From b22c7ba327c17a77f8c40c31c09be4588651589a Mon Sep 17 00:00:00 2001 From: Michel Oosterhof Date: Sun, 20 Jan 2019 14:39:37 +0400 Subject: [PATCH] Doclink (#990) * move docs to docs dir and link back --- CHANGELOG.rst | 110 +----------------- CONTRIBUTING.rst | 54 +-------- INSTALL.rst | 254 +----------------------------------------- LICENSE.rst | 30 +---- README.rst | 110 +----------------- docs/CHANGELOG.rst | 109 ++++++++++++++++++ docs/CONTRIBUTING.rst | 53 +++++++++ docs/INSTALL.rst | 253 +++++++++++++++++++++++++++++++++++++++++ docs/LICENSE.rst | 29 +++++ docs/README.rst | 109 ++++++++++++++++++ 10 files changed, 558 insertions(+), 553 deletions(-) mode change 100644 => 120000 CHANGELOG.rst mode change 100644 => 120000 CONTRIBUTING.rst mode change 100644 => 120000 INSTALL.rst mode change 100644 => 120000 LICENSE.rst mode change 100644 => 120000 README.rst create mode 100644 docs/CHANGELOG.rst create mode 100644 docs/CONTRIBUTING.rst create mode 100644 docs/INSTALL.rst create mode 100644 docs/LICENSE.rst create mode 100644 docs/README.rst diff --git a/CHANGELOG.rst b/CHANGELOG.rst deleted file mode 100644 index 125d9b86..00000000 --- a/CHANGELOG.rst +++ /dev/null @@ -1,109 +0,0 @@ - -Release 1.5.2 -============= - -* 2018-11-19 Fix tftp exception and tftp test -* 2018-11-14 Remove `dblog` mechanism and `splunk` legacy output plugin. -* 2018-11-01 Add Python3 support for Splunk output plugin -* 2018-10-23 Improved free command -* 2018-10-20 Improved uname command -* 2018-10-16 Save VT results to JSON log - -Release 1.5.1 -============= - -* 2018-10-13 Fixes VT uploads, tab completion on Python3, Hassh support, setuptools functional. userdb migration -* 2018-09-07 NOTE! data/userdb.txt has moved to etc/userdb.txt and a default config is no longer provided! -* 2018-08-25 Downloads and TTY logs have moved to the var/ directory -* 2018-08-11 SSH keys now stored in var/lib/cowrie -* 2018-07-21 source code has move to the src/ directory. Delete old directories twisted/cowrie with compiled code -* 2018-06-29 txtcmds have been moved to share/cowrie/txtcmds -* 2018-06-28 filesystem config entry has changed. please verify if you have custom entry or pickle file -* 2018-06-23 fingerprint log message now holds KEX attributes and a unique fingerprint for the client -* 2018-04-27 Output plugins now require the mandatory config entry 'enabled'. -* 2018-02-06 cowrie.log now uses same rotation mechanism as cowrie.json. One file per day, rather than the default 1MB per file. -* 2017-12-13 Default umask for logs is now 0007. This means group members can access. -* 2017-10-24 Can store uploaded and downloaded artifacts to S3 -* 2017-09-23 First proxy implementation for exec commands only -* 2017-07-03 Cuckoo v2 integration -* 2017-05-16 now combines config files: cowrie.cfg.dist and cowrie.cfg in this order -* 2017-05-09 start.sh and stop.sh have been replace by bin/cowrie start|stop -* 2017-04-27 New syntax "listen_endpoints" for configuring listening IP addresses/portnumbers -* 2017-03-15 SSH Forwarding/SFTP/keys/version config have been moved to [ssh]. Change your config file! -* 2017-02-12 Implemented toggle for SSH forwarding -* 2016-08-22 Merged Telnet support by @obilodeau! -* 2016-08-20 Update your libraries! 'configparser' now required: "pip install configparser" -* 2016-05-06 Load pickle once at startup for improved speed -* 2016-04-28 files in utils/ have been moved to bin/ -* 2016-01-19 Support openssh style delayed compression -* 2016-01-13 Correct '.' support and +s and +t bits in ls -* 2016-01-13 Full username/group in SFTP ls -* 2016-01-05 Basic VirusTotal support has been added -* 2016-01-04 No longer crash when client tries ecdsa -* 2015-12-28 Interact port (default 5123) only listens on loopback interface now (127.0.0.1) -* 2015-12-24 Redirect to file (>) now works for most commands and is logged in dl/ directory -* 2015-12-06 UID information is now retrieved from honeyfs/etc/passwd. If you added additional users - you will need to add these to the passwd file as well -* 2015-12-04 New 'free' command with '-h' and '-m' options -* 2015-12-03 New 'env' command that prints environment variables -* 2015-02-02 Now use honeyfs/etc/passwd and group to get uid/gid info -* 2015-11-29 Size limit now enforced for SFTP uploads -* 2015-11-25 New 'sudo' command added -* 2015-11-19 Queued input during commands is now sent to shell to be executed - when command is finished -* 2015-11-18 Added SANS DShield output (Thanks @UnrealAkama) -* 2015-11-17 Added ElasticSearch output (Thanks @UnrealAkama) -* 2015-11-17 Standard input is now saved with SHA256 checksum. Duplicate data is not saved -* 2015-11-12 New 'busybox' command added (Thanks @mak) -* 2015-09-26 keyboard-interactive is back as authentication method, after - Twisted removed support initially -* 2015-07-30 Local syslog output module -* 2015-06-15 Cowrie now has a '-c' startup switch to specify the configuration file -* 2015-06-15 Removed exec_enabled option. This feature is now always enabled -* 2015-06-03 Cowrie now uses twisted plugins and has gained the '-p' commandline option -* 2015-06-01 Cowrie no longer search for config files in /etc and /etc/cowrie -* 2015-04-12 JSON output is now default via 'output' plugin mechanism. Rotates daily -* 2015-04-10 Fix for downloading files via SFTP -* 2015-03-31 Small tweaks on session close, closing session does not close ssh transport -* 2015-03-18 Merged 'AuthRandom' login class by Honigbij -* 2015-02-25 Internals for dblog/ modules changed completely. - Now accepts structured logging arguments, and uses eventids instead of regex parsing -* 2015-02-20 Removed screen clear/reset on logout -* 2015-02-19 Configuration directives have changed! ssh_addr has become listen_addr and ssh_port has become listen_port. The old keywords are still accepted for backwards compatibility - -* default behaviour is changed to disable the exit jail -* sftp support -* exec support -* stdin is saved as a file in dl/ when using exec commands - to support commands like 'cat >file; ./file' -* allow wget download over non-80 port -* simple JSON logging added -* accept log and deny publickey authentication -* add uname -r, -m flags -* add working sleep command -* enabled ssh diffie-hellman-group-exchange-sha1 algorithm -* add 'bash -c' support (no effect option) -* enable support for && multiple commands -* create uuid to uniquely identify each session -* log and deny direct-tcpip attempts -* add "chattr" command -* support emacs keybindings (c-a, c-b, c-f, c-p, c-n, c-e) -* add "sync" command -* accept, log and deny public key authentication -* add "uname -r" support -* logstash and kibana config files added, based on JSON log -* fix for honeypot detection (pre-auth differences with openssh) -* added verbose logging of client requested key exchange parameters (for client fingerprinting) -* fixes for behavior with non-existent files (cd /test, cat /test/nonexistent, etc) -* fix for ability to ping/ssh non-existent IP address -* always send ssh exit-status 0 on exec and shell -* ls output is now alphabetically sorted -* banner_file is deprecated. honeyfs/etc/issue.net is default -* add 'dir' alias for 'ls' -* add 'help' bash builtin -* add 'users' aliased to 'whoami' -* add 'killall' and 'killall5' aliased to nop -* add 'poweroff' 'halt' and 'reboot' aliases for shutdown -* add environment passing to commands -* added 'which', 'netstat' and 'gcc' from kippo-extra -* logging framework allows for keyword use diff --git a/CHANGELOG.rst b/CHANGELOG.rst new file mode 120000 index 00000000..ea2d0f01 --- /dev/null +++ b/CHANGELOG.rst @@ -0,0 +1 @@ +docs/CHANGELOG.rst \ No newline at end of file diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst deleted file mode 100644 index b60a4b6b..00000000 --- a/CONTRIBUTING.rst +++ /dev/null @@ -1,53 +0,0 @@ -Contributing Guidelines -####################### - -Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional -documentation, we greatly value feedback and contributions from our community. - -Please read through this document before submitting any issues or pull requests to ensure we have all the necessary -information to effectively respond to your bug report or contribution. - - -Reporting Bugs/Feature Requests -############################### -We welcome you to use the GitHub issue tracker to report bugs or suggest features. - -When filing an issue, please check `existing open `_, or `recently closed `_, issues to make sure somebody else hasn't already -reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: - -* A reproducible test case or series of steps -* The version of our code being used -* Any modifications you've made relevant to the bug -* Anything unusual about your environment or deployment - - -Contributing via Pull Requests -############################## -Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: - -1. You are working against the latest source on the *master* branch. -2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. -3. You open an issue to discuss any significant work - we would hate for your time to be wasted. - -To send us a pull request, please: - -1. Fork the repository. -2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. -3. Ensure local tests pass. -4. Commit to your fork using clear commit messages. -5. Send us a pull request, answering any default questions in the pull request interface. -6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. - -GitHub provides additional document on `forking a repository `_ and -`creating a pull request `_. - - -Finding contributions to work on -################################ -Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any `help wanted' `_ issues is a great place to start. - - -Licensing -######### -See the `LICENSE `_ file for our project's licensing. We will ask you confirm the licensing of your contribution. - diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst new file mode 120000 index 00000000..60a7e7fe --- /dev/null +++ b/CONTRIBUTING.rst @@ -0,0 +1 @@ +docs/CONTRIBUTING.rst \ No newline at end of file diff --git a/INSTALL.rst b/INSTALL.rst deleted file mode 100644 index cfb0d493..00000000 --- a/INSTALL.rst +++ /dev/null @@ -1,253 +0,0 @@ - -Installing Cowrie in seven steps. -################################# - -* [Step 1: Install dependencies](#step-1-install-dependencies) -* [Step 2: Create a user account](#step-2-create-a-user-account) -* [Step 3: Checkout the code](#step-3-checkout-the-code) -* [Step 4: Setup Virtual Environment](#step-4-setup-virtual-environment) -* [Step 5: Install configuration file](#step-5-install-configuration-file) -* [Step 6: Generate a DSA key (OPTIONAL)](#step-6-generate-a-dsa-key) -* [Step 7: Starting Cowrie](#step-7-turning-on-cowrie) -* [Step 8: Port redirection (OPTIONAL)](#step-8-port-redirection-optional) -* [Running within supervisord (OPTIONAL)](#running-using-supervisord) -* [Configure Additional Output Plugins (OPTIONAL)](#configure-additional-output-plugins-optional) -* [Troubleshooting](#troubleshooting) - -Step 1: Install dependencies -**************************** - -First we install system-wide support for Python virtual environments and other dependencies. -Actual Python packages are installed later. - -On Debian based systems (last verified on Debian 9, 2017-07-25): -For a Python3 based environment:: - - $ sudo apt-get install git python-virtualenv libssl-dev libffi-dev build-essential libpython3-dev python3-minimal authbind - -Or for Python2:: - - $ sudo apt-get install git python-virtualenv libssl-dev libffi-dev build-essential libpython-dev python2.7-minimal authbind - -Step 2: Create a user account -***************************** - -It's strongly recommended to run with a dedicated non-root user id:: - - $ sudo adduser --disabled-password cowrie - Adding user 'cowrie' ... - Adding new group 'cowrie' (1002) ... - Adding new user 'cowrie' (1002) with group 'cowrie' ... - Changing the user information for cowrie - Enter the new value, or press ENTER for the default - Full Name []: - Room Number []: - Work Phone []: - Home Phone []: - Other []: - Is the information correct? [Y/n] - - $ sudo su - cowrie - -Step 3: Checkout the code -***************************** - -Check out the code:: - - $ git clone http://github.com/cowrie/cowrie - Cloning into 'cowrie'... - remote: Counting objects: 2965, done. - remote: Compressing objects: 100% (1025/1025), done. - remote: Total 2965 (delta 1908), reused 2962 (delta 1905), pack-reused 0 - Receiving objects: 100% (2965/2965), 3.41 MiB | 2.57 MiB/s, done. - Resolving deltas: 100% (1908/1908), done. - Checking connectivity... done. - - $ cd cowrie - -## Step 4: Setup Virtual Environment -************************************ - -Next you need to create your virtual environment:: - - $ pwd - /home/cowrie/cowrie - $ virtualenv --python=python3 cowrie-env - New python executable in ./cowrie/cowrie-env/bin/python - Installing setuptools, pip, wheel...done. - -Alternatively, create a Python2 virtual environment:: - - $ virtualenv --python=python2 cowrie-env - New python executable in ./cowrie/cowrie-env/bin/python - Installing setuptools, pip, wheel...done. - -Activate the virtual environment and install packages:: - - - $ source cowrie-env/bin/activate - (cowrie-env) $ pip install --upgrade pip - (cowrie-env) $ pip install --upgrade -r requirements.txt - -Step 5: Install configuration file -********************************** - -The configuration for Cowrie is stored in cowrie.cfg.dist and -cowrie.cfg. Both files are read on startup, where entries from -cowrie.cfg take precedence. The .dist file can be overwritten by -upgrades, cowrie.cfg will not be touched. To run with a standard -configuration, there is no need to change anything. To enable telnet, -for example, create cowrie.cfg and input only the following:: - - [telnet] - enabled = true - -Step 6: Starting Cowrie -*********************** - -Start Cowrie with the cowrie command. You can add the cowrie/bin -directory to your path if desired. An existing virtual environment -is preserved if activated, otherwise Cowrie will attempt to load -the environment called "cowrie-env":: - - - $ bin/cowrie start - Activating virtualenv "cowrie-env" - Starting cowrie with extra arguments [] ... - -Step 7: Listening on port 22 (OPTIONAL) -*************************************** - -There are three methods to make Cowrie accessible on the default SSH port (22): `iptables`, `authbind` and `setcap`. - -Iptables -======== - -Port redirection commands are system-wide and need to be executed as root. -A firewall redirect can make your existing SSH server unreachable, remember to move the existing -server to a different port number first. - -The following firewall rule will forward incoming traffic on port 22 to port 2222 on Linux:: - - $ sudo iptables -t nat -A PREROUTING -p tcp --dport 22 -j REDIRECT --to-port 2222 - -Or for telnet:: - - $ sudo iptables -t nat -A PREROUTING -p tcp --dport 23 -j REDIRECT --to-port 2223 - -Note that you should test this rule only from another host; it doesn't apply to loopback connections. - -On MacOS run:: - - $ echo "rdr pass inet proto tcp from any to any port 22 -> 127.0.0.1 port 2222" | sudo pfctl -ef - - -Authbind -======== - -Alternatively you can run authbind to listen as non-root on port 22 directly:: - - $ sudo apt-get install authbind - $ sudo touch /etc/authbind/byport/22 - $ sudo chown cowrie:cowrie /etc/authbind/byport/22 - $ sudo chmod 770 /etc/authbind/byport/22 - -Edit bin/cowrie and modify the AUTHBIND_ENABLED setting - -Change the listening port to 22 in cowrie.cfg:: - - [ssh] - listen_endpoints = tcp:22:interface=0.0.0.0 - -Or for telnet:: - - $ apt-get install authbind - $ sudo touch /etc/authbind/byport/23 - $ sudo chown cowrie:cowrie /etc/authbind/byport/23 - $ sudo chmod 770 /etc/authbind/byport/23 - -Change the listening port to 23 in cowrie.cfg:: - - [telnet] - listen_endpoints = tcp:2223:interface=0.0.0.0 - -Setcap -====== - -Or use setcap to give permissions to Python to listen on ports<1024:: - - $ setcap cap_net_bind_service=+ep /usr/bin/python2.7 - -And change the listening ports in `cowrie.cfg` as above. - - -Running using Supervisord (OPTIONAL) -************************************ - -On Debian, put the below in /etc/supervisor/conf.d/cowrie.conf:: - - [program:cowrie] - command=/home/cowrie/cowrie/bin/cowrie start - directory=/home/cowrie/cowrie/ - user=cowrie - autorestart=true - redirect_stderr=true - -Update the bin/cowrie script, change:: - - DAEMONIZE="" - -to:: - - DAEMONIZE="-n" - -Configure Additional Output Plugins (OPTIONAL) -********************************************** - -Cowrie automatically outputs event data to text and JSON log files -in `var/log/cowrie`. Additional output plugins can be configured to -record the data other ways. Supported output plugins include: - -* Cuckoo -* ELK (Elastic) Stack -* Graylog -* Kippo-Graph -* Splunk -* SQL (MySQL, SQLite3, RethinkDB) - -See ~/cowrie/docs/[Output Plugin]/README.rst for details. - - -Troubleshooting -############### - -If you see `twistd: Unknown command: cowrie` there are two - possibilities. If there's a Python stack trace, it probably means - there's a missing or broken dependency. If there's no stack trace, - double check that your PYTHONPATH is set to the source code directory. - -Default file permissions - -To make Cowrie logfiles public readable, change the ``--umask 0077`` option in start.sh into ``--umask 0022`` - -Updating Cowrie -################# - -Updating is an easy process. First stop your honeypot. Then fetch updates from GitHub, and upgrade your Python dependencies:: - - bin/cowrie stop - git pull - pip install --upgrade -r requirements.txt - -If you use output plugins like SQL, Splunk, or ELK, remember to also upgrade your dependencies for these too:: - - pip install --upgrade -r requirements-output.txt - -And finally, start Cowrie back up after finishing all updates:: - - bin/cowrie start - -Modifying Cowrie -################ - -The pre-login banner can be set by creating the file `honeyfs/etc/issue.net`. -The post-login banner can be customized by editing `honeyfs/etc/motd`. diff --git a/INSTALL.rst b/INSTALL.rst new file mode 120000 index 00000000..46ed5404 --- /dev/null +++ b/INSTALL.rst @@ -0,0 +1 @@ +docs/INSTALL.rst \ No newline at end of file diff --git a/LICENSE.rst b/LICENSE.rst deleted file mode 100644 index e67e184d..00000000 --- a/LICENSE.rst +++ /dev/null @@ -1,29 +0,0 @@ -LICENSE -####### - -Copyright (c) 2009 Upi Tamminen -All rights reserved. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions -are met: -1. Redistributions of source code must retain the above copyright - notice, this list of conditions and the following disclaimer. -2. Redistributions in binary form must reproduce the above copyright - notice, this list of conditions and the following disclaimer in the - documentation and/or other materials provided with the distribution. -3. The names of the author(s) may not be used to endorse or promote - products derived from this software without specific prior written - permission. - -THIS SOFTWARE IS PROVIDED BY THE AUTHORS ''AS IS'' AND ANY EXPRESS OR -IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES -OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. -IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY DIRECT, INDIRECT, -INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, -BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; -LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED -AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY -OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF -SUCH DAMAGE. diff --git a/LICENSE.rst b/LICENSE.rst new file mode 120000 index 00000000..206d0943 --- /dev/null +++ b/LICENSE.rst @@ -0,0 +1 @@ +docs/LICENSE.rst \ No newline at end of file diff --git a/README.rst b/README.rst deleted file mode 100644 index 0578f15b..00000000 --- a/README.rst +++ /dev/null @@ -1,109 +0,0 @@ -Cowrie -###### - -|travis|_ -|codecov|_ - -Welcome to the Cowrie GitHub repository -***************************************** - -This is the official repository for the Cowrie SSH and Telnet -Honeypot effort. - -What is Cowrie -***************************************** - -Cowrie is a medium interaction SSH and Telnet honeypot designed to -log brute force attacks and the shell interaction performed by the -attacker. - -`Cowrie `_ is developed by Michel Oosterhof. - -Slack -***************************************** - -You can join the Cowrie community at the following `Slack workspace `_. - -Features -***************************************** - -Some interesting features: - -* Fake filesystem with the ability to add/remove files. A full fake filesystem resembling a Debian 5.0 installation is included -* Possibility of adding fake file contents so the attacker can `cat` files such as `/etc/passwd`. Only minimal file contents are included -* Session logs are stored in an `UML Compatible `_ format for easy replay with original timings with the `bin/playlog` utility. -* Cowrie saves files downloaded with wget/curl or uploaded with SFTP and scp for later inspection log - -Additional functionality over standard kippo: - -* SFTP and SCP support for file upload -* Support for SSH exec commands -* Logging of direct-tcp connection attempts (ssh proxying) -* Forward SMTP connections to SMTP Honeypot (e.g. `mailoney `_) -* Logging in JSON format for easy processing in log management solutions -* Many, many additional commands - -Docker -***************************************** - -Docker versions are available. - -* To get started quickly and give Cowrie a try, run:: - - docker run -p 2222:2222 cowrie/cowrie - ssh -p 2222 root@localhost - -* On Docker Hub: https://hub.docker.com/r/cowrie/cowrie - -* Or get the Dockerfile directly at https://github.com/cowrie/docker-cowrie - -Requirements -***************************************** - -Software required: - -* Python 2.7+, (Limited Python 3 support available for SSH only) -* python-virtualenv - -For Python dependencies, see requirements.txt - -Files of interest: -***************************************** - -* `cowrie.cfg` - Cowrie's configuration file. Default values can be found in `etc/cowrie.cfg.dist` -* `share/cowrie/fs.pickle` - fake filesystem -* `etc/userdb.txt` - credentials allowed or disallowed to access the honeypot -* `honeyfs/` - file contents for the fake filesystem - feel free to copy a real system here or use `bin/fsctl` -* `honeyfs/etc/issue.net` - pre-login banner -* `honeyfs/etc/motd` - post-login banner -* `var/log/cowrie/cowrie.json` - transaction output in JSON format -* `var/log/cowrie/cowrie.log` - log/debug output -* `var/lib/cowrie/tty/` - session logs, replayable with the `bin/playlog` utility. -* `var/lib/cowrie/downloads/` - files transferred from the attacker to the honeypot are stored here -* `share/cowrie/txtcmds/` - file contents for simple fake commands -* `bin/createfs` - used to create the fake filesystem -* `bin/playlog` - utility to replay session logs - -I have some questions! -***************************************** - -Please visit the `Slack workspace `_ and join the #questions channel. - -Contributors -*************** - -Many people have contributed to Cowrie over the years. Special thanks to: - -* Upi Tamminen (desaster) for all his work developing Kippo on which Cowrie was based -* Dave Germiquet (davegermiquet) for TFTP support, unit tests, new process handling -* Olivier Bilodeau (obilodeau) for Telnet support -* Ivan Korolev (fe7ch) for many improvements over the years. -* Florian Pelgrim (craneworks) for his work on code cleanup and Docker. -* And many many others. - - -.. |travis| image:: https://travis-ci.org/cowrie/cowrie.svg?branch=master -.. _travis: https://travis-ci.org/cowrie/cowrie - -.. |codecov| image:: https://codecov.io/gh/cowrie/cowrie/branch/master/graph/badge.svg -.. _codecov: https://codecov.io/gh/cowrie/cowrie diff --git a/README.rst b/README.rst new file mode 120000 index 00000000..cffceba7 --- /dev/null +++ b/README.rst @@ -0,0 +1 @@ +docs/README.rst \ No newline at end of file diff --git a/docs/CHANGELOG.rst b/docs/CHANGELOG.rst new file mode 100644 index 00000000..125d9b86 --- /dev/null +++ b/docs/CHANGELOG.rst @@ -0,0 +1,109 @@ + +Release 1.5.2 +============= + +* 2018-11-19 Fix tftp exception and tftp test +* 2018-11-14 Remove `dblog` mechanism and `splunk` legacy output plugin. +* 2018-11-01 Add Python3 support for Splunk output plugin +* 2018-10-23 Improved free command +* 2018-10-20 Improved uname command +* 2018-10-16 Save VT results to JSON log + +Release 1.5.1 +============= + +* 2018-10-13 Fixes VT uploads, tab completion on Python3, Hassh support, setuptools functional. userdb migration +* 2018-09-07 NOTE! data/userdb.txt has moved to etc/userdb.txt and a default config is no longer provided! +* 2018-08-25 Downloads and TTY logs have moved to the var/ directory +* 2018-08-11 SSH keys now stored in var/lib/cowrie +* 2018-07-21 source code has move to the src/ directory. Delete old directories twisted/cowrie with compiled code +* 2018-06-29 txtcmds have been moved to share/cowrie/txtcmds +* 2018-06-28 filesystem config entry has changed. please verify if you have custom entry or pickle file +* 2018-06-23 fingerprint log message now holds KEX attributes and a unique fingerprint for the client +* 2018-04-27 Output plugins now require the mandatory config entry 'enabled'. +* 2018-02-06 cowrie.log now uses same rotation mechanism as cowrie.json. One file per day, rather than the default 1MB per file. +* 2017-12-13 Default umask for logs is now 0007. This means group members can access. +* 2017-10-24 Can store uploaded and downloaded artifacts to S3 +* 2017-09-23 First proxy implementation for exec commands only +* 2017-07-03 Cuckoo v2 integration +* 2017-05-16 now combines config files: cowrie.cfg.dist and cowrie.cfg in this order +* 2017-05-09 start.sh and stop.sh have been replace by bin/cowrie start|stop +* 2017-04-27 New syntax "listen_endpoints" for configuring listening IP addresses/portnumbers +* 2017-03-15 SSH Forwarding/SFTP/keys/version config have been moved to [ssh]. Change your config file! +* 2017-02-12 Implemented toggle for SSH forwarding +* 2016-08-22 Merged Telnet support by @obilodeau! +* 2016-08-20 Update your libraries! 'configparser' now required: "pip install configparser" +* 2016-05-06 Load pickle once at startup for improved speed +* 2016-04-28 files in utils/ have been moved to bin/ +* 2016-01-19 Support openssh style delayed compression +* 2016-01-13 Correct '.' support and +s and +t bits in ls +* 2016-01-13 Full username/group in SFTP ls +* 2016-01-05 Basic VirusTotal support has been added +* 2016-01-04 No longer crash when client tries ecdsa +* 2015-12-28 Interact port (default 5123) only listens on loopback interface now (127.0.0.1) +* 2015-12-24 Redirect to file (>) now works for most commands and is logged in dl/ directory +* 2015-12-06 UID information is now retrieved from honeyfs/etc/passwd. If you added additional users + you will need to add these to the passwd file as well +* 2015-12-04 New 'free' command with '-h' and '-m' options +* 2015-12-03 New 'env' command that prints environment variables +* 2015-02-02 Now use honeyfs/etc/passwd and group to get uid/gid info +* 2015-11-29 Size limit now enforced for SFTP uploads +* 2015-11-25 New 'sudo' command added +* 2015-11-19 Queued input during commands is now sent to shell to be executed + when command is finished +* 2015-11-18 Added SANS DShield output (Thanks @UnrealAkama) +* 2015-11-17 Added ElasticSearch output (Thanks @UnrealAkama) +* 2015-11-17 Standard input is now saved with SHA256 checksum. Duplicate data is not saved +* 2015-11-12 New 'busybox' command added (Thanks @mak) +* 2015-09-26 keyboard-interactive is back as authentication method, after + Twisted removed support initially +* 2015-07-30 Local syslog output module +* 2015-06-15 Cowrie now has a '-c' startup switch to specify the configuration file +* 2015-06-15 Removed exec_enabled option. This feature is now always enabled +* 2015-06-03 Cowrie now uses twisted plugins and has gained the '-p' commandline option +* 2015-06-01 Cowrie no longer search for config files in /etc and /etc/cowrie +* 2015-04-12 JSON output is now default via 'output' plugin mechanism. Rotates daily +* 2015-04-10 Fix for downloading files via SFTP +* 2015-03-31 Small tweaks on session close, closing session does not close ssh transport +* 2015-03-18 Merged 'AuthRandom' login class by Honigbij +* 2015-02-25 Internals for dblog/ modules changed completely. + Now accepts structured logging arguments, and uses eventids instead of regex parsing +* 2015-02-20 Removed screen clear/reset on logout +* 2015-02-19 Configuration directives have changed! ssh_addr has become listen_addr and ssh_port has become listen_port. The old keywords are still accepted for backwards compatibility + +* default behaviour is changed to disable the exit jail +* sftp support +* exec support +* stdin is saved as a file in dl/ when using exec commands + to support commands like 'cat >file; ./file' +* allow wget download over non-80 port +* simple JSON logging added +* accept log and deny publickey authentication +* add uname -r, -m flags +* add working sleep command +* enabled ssh diffie-hellman-group-exchange-sha1 algorithm +* add 'bash -c' support (no effect option) +* enable support for && multiple commands +* create uuid to uniquely identify each session +* log and deny direct-tcpip attempts +* add "chattr" command +* support emacs keybindings (c-a, c-b, c-f, c-p, c-n, c-e) +* add "sync" command +* accept, log and deny public key authentication +* add "uname -r" support +* logstash and kibana config files added, based on JSON log +* fix for honeypot detection (pre-auth differences with openssh) +* added verbose logging of client requested key exchange parameters (for client fingerprinting) +* fixes for behavior with non-existent files (cd /test, cat /test/nonexistent, etc) +* fix for ability to ping/ssh non-existent IP address +* always send ssh exit-status 0 on exec and shell +* ls output is now alphabetically sorted +* banner_file is deprecated. honeyfs/etc/issue.net is default +* add 'dir' alias for 'ls' +* add 'help' bash builtin +* add 'users' aliased to 'whoami' +* add 'killall' and 'killall5' aliased to nop +* add 'poweroff' 'halt' and 'reboot' aliases for shutdown +* add environment passing to commands +* added 'which', 'netstat' and 'gcc' from kippo-extra +* logging framework allows for keyword use diff --git a/docs/CONTRIBUTING.rst b/docs/CONTRIBUTING.rst new file mode 100644 index 00000000..b60a4b6b --- /dev/null +++ b/docs/CONTRIBUTING.rst @@ -0,0 +1,53 @@ +Contributing Guidelines +####################### + +Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional +documentation, we greatly value feedback and contributions from our community. + +Please read through this document before submitting any issues or pull requests to ensure we have all the necessary +information to effectively respond to your bug report or contribution. + + +Reporting Bugs/Feature Requests +############################### +We welcome you to use the GitHub issue tracker to report bugs or suggest features. + +When filing an issue, please check `existing open `_, or `recently closed `_, issues to make sure somebody else hasn't already +reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: + +* A reproducible test case or series of steps +* The version of our code being used +* Any modifications you've made relevant to the bug +* Anything unusual about your environment or deployment + + +Contributing via Pull Requests +############################## +Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: + +1. You are working against the latest source on the *master* branch. +2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. +3. You open an issue to discuss any significant work - we would hate for your time to be wasted. + +To send us a pull request, please: + +1. Fork the repository. +2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. +3. Ensure local tests pass. +4. Commit to your fork using clear commit messages. +5. Send us a pull request, answering any default questions in the pull request interface. +6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. + +GitHub provides additional document on `forking a repository `_ and +`creating a pull request `_. + + +Finding contributions to work on +################################ +Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any `help wanted' `_ issues is a great place to start. + + +Licensing +######### +See the `LICENSE `_ file for our project's licensing. We will ask you confirm the licensing of your contribution. + diff --git a/docs/INSTALL.rst b/docs/INSTALL.rst new file mode 100644 index 00000000..cfb0d493 --- /dev/null +++ b/docs/INSTALL.rst @@ -0,0 +1,253 @@ + +Installing Cowrie in seven steps. +################################# + +* [Step 1: Install dependencies](#step-1-install-dependencies) +* [Step 2: Create a user account](#step-2-create-a-user-account) +* [Step 3: Checkout the code](#step-3-checkout-the-code) +* [Step 4: Setup Virtual Environment](#step-4-setup-virtual-environment) +* [Step 5: Install configuration file](#step-5-install-configuration-file) +* [Step 6: Generate a DSA key (OPTIONAL)](#step-6-generate-a-dsa-key) +* [Step 7: Starting Cowrie](#step-7-turning-on-cowrie) +* [Step 8: Port redirection (OPTIONAL)](#step-8-port-redirection-optional) +* [Running within supervisord (OPTIONAL)](#running-using-supervisord) +* [Configure Additional Output Plugins (OPTIONAL)](#configure-additional-output-plugins-optional) +* [Troubleshooting](#troubleshooting) + +Step 1: Install dependencies +**************************** + +First we install system-wide support for Python virtual environments and other dependencies. +Actual Python packages are installed later. + +On Debian based systems (last verified on Debian 9, 2017-07-25): +For a Python3 based environment:: + + $ sudo apt-get install git python-virtualenv libssl-dev libffi-dev build-essential libpython3-dev python3-minimal authbind + +Or for Python2:: + + $ sudo apt-get install git python-virtualenv libssl-dev libffi-dev build-essential libpython-dev python2.7-minimal authbind + +Step 2: Create a user account +***************************** + +It's strongly recommended to run with a dedicated non-root user id:: + + $ sudo adduser --disabled-password cowrie + Adding user 'cowrie' ... + Adding new group 'cowrie' (1002) ... + Adding new user 'cowrie' (1002) with group 'cowrie' ... + Changing the user information for cowrie + Enter the new value, or press ENTER for the default + Full Name []: + Room Number []: + Work Phone []: + Home Phone []: + Other []: + Is the information correct? [Y/n] + + $ sudo su - cowrie + +Step 3: Checkout the code +***************************** + +Check out the code:: + + $ git clone http://github.com/cowrie/cowrie + Cloning into 'cowrie'... + remote: Counting objects: 2965, done. + remote: Compressing objects: 100% (1025/1025), done. + remote: Total 2965 (delta 1908), reused 2962 (delta 1905), pack-reused 0 + Receiving objects: 100% (2965/2965), 3.41 MiB | 2.57 MiB/s, done. + Resolving deltas: 100% (1908/1908), done. + Checking connectivity... done. + + $ cd cowrie + +## Step 4: Setup Virtual Environment +************************************ + +Next you need to create your virtual environment:: + + $ pwd + /home/cowrie/cowrie + $ virtualenv --python=python3 cowrie-env + New python executable in ./cowrie/cowrie-env/bin/python + Installing setuptools, pip, wheel...done. + +Alternatively, create a Python2 virtual environment:: + + $ virtualenv --python=python2 cowrie-env + New python executable in ./cowrie/cowrie-env/bin/python + Installing setuptools, pip, wheel...done. + +Activate the virtual environment and install packages:: + + + $ source cowrie-env/bin/activate + (cowrie-env) $ pip install --upgrade pip + (cowrie-env) $ pip install --upgrade -r requirements.txt + +Step 5: Install configuration file +********************************** + +The configuration for Cowrie is stored in cowrie.cfg.dist and +cowrie.cfg. Both files are read on startup, where entries from +cowrie.cfg take precedence. The .dist file can be overwritten by +upgrades, cowrie.cfg will not be touched. To run with a standard +configuration, there is no need to change anything. To enable telnet, +for example, create cowrie.cfg and input only the following:: + + [telnet] + enabled = true + +Step 6: Starting Cowrie +*********************** + +Start Cowrie with the cowrie command. You can add the cowrie/bin +directory to your path if desired. An existing virtual environment +is preserved if activated, otherwise Cowrie will attempt to load +the environment called "cowrie-env":: + + + $ bin/cowrie start + Activating virtualenv "cowrie-env" + Starting cowrie with extra arguments [] ... + +Step 7: Listening on port 22 (OPTIONAL) +*************************************** + +There are three methods to make Cowrie accessible on the default SSH port (22): `iptables`, `authbind` and `setcap`. + +Iptables +======== + +Port redirection commands are system-wide and need to be executed as root. +A firewall redirect can make your existing SSH server unreachable, remember to move the existing +server to a different port number first. + +The following firewall rule will forward incoming traffic on port 22 to port 2222 on Linux:: + + $ sudo iptables -t nat -A PREROUTING -p tcp --dport 22 -j REDIRECT --to-port 2222 + +Or for telnet:: + + $ sudo iptables -t nat -A PREROUTING -p tcp --dport 23 -j REDIRECT --to-port 2223 + +Note that you should test this rule only from another host; it doesn't apply to loopback connections. + +On MacOS run:: + + $ echo "rdr pass inet proto tcp from any to any port 22 -> 127.0.0.1 port 2222" | sudo pfctl -ef - + +Authbind +======== + +Alternatively you can run authbind to listen as non-root on port 22 directly:: + + $ sudo apt-get install authbind + $ sudo touch /etc/authbind/byport/22 + $ sudo chown cowrie:cowrie /etc/authbind/byport/22 + $ sudo chmod 770 /etc/authbind/byport/22 + +Edit bin/cowrie and modify the AUTHBIND_ENABLED setting + +Change the listening port to 22 in cowrie.cfg:: + + [ssh] + listen_endpoints = tcp:22:interface=0.0.0.0 + +Or for telnet:: + + $ apt-get install authbind + $ sudo touch /etc/authbind/byport/23 + $ sudo chown cowrie:cowrie /etc/authbind/byport/23 + $ sudo chmod 770 /etc/authbind/byport/23 + +Change the listening port to 23 in cowrie.cfg:: + + [telnet] + listen_endpoints = tcp:2223:interface=0.0.0.0 + +Setcap +====== + +Or use setcap to give permissions to Python to listen on ports<1024:: + + $ setcap cap_net_bind_service=+ep /usr/bin/python2.7 + +And change the listening ports in `cowrie.cfg` as above. + + +Running using Supervisord (OPTIONAL) +************************************ + +On Debian, put the below in /etc/supervisor/conf.d/cowrie.conf:: + + [program:cowrie] + command=/home/cowrie/cowrie/bin/cowrie start + directory=/home/cowrie/cowrie/ + user=cowrie + autorestart=true + redirect_stderr=true + +Update the bin/cowrie script, change:: + + DAEMONIZE="" + +to:: + + DAEMONIZE="-n" + +Configure Additional Output Plugins (OPTIONAL) +********************************************** + +Cowrie automatically outputs event data to text and JSON log files +in `var/log/cowrie`. Additional output plugins can be configured to +record the data other ways. Supported output plugins include: + +* Cuckoo +* ELK (Elastic) Stack +* Graylog +* Kippo-Graph +* Splunk +* SQL (MySQL, SQLite3, RethinkDB) + +See ~/cowrie/docs/[Output Plugin]/README.rst for details. + + +Troubleshooting +############### + +If you see `twistd: Unknown command: cowrie` there are two + possibilities. If there's a Python stack trace, it probably means + there's a missing or broken dependency. If there's no stack trace, + double check that your PYTHONPATH is set to the source code directory. + +Default file permissions + +To make Cowrie logfiles public readable, change the ``--umask 0077`` option in start.sh into ``--umask 0022`` + +Updating Cowrie +################# + +Updating is an easy process. First stop your honeypot. Then fetch updates from GitHub, and upgrade your Python dependencies:: + + bin/cowrie stop + git pull + pip install --upgrade -r requirements.txt + +If you use output plugins like SQL, Splunk, or ELK, remember to also upgrade your dependencies for these too:: + + pip install --upgrade -r requirements-output.txt + +And finally, start Cowrie back up after finishing all updates:: + + bin/cowrie start + +Modifying Cowrie +################ + +The pre-login banner can be set by creating the file `honeyfs/etc/issue.net`. +The post-login banner can be customized by editing `honeyfs/etc/motd`. diff --git a/docs/LICENSE.rst b/docs/LICENSE.rst new file mode 100644 index 00000000..e67e184d --- /dev/null +++ b/docs/LICENSE.rst @@ -0,0 +1,29 @@ +LICENSE +####### + +Copyright (c) 2009 Upi Tamminen +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. +3. The names of the author(s) may not be used to endorse or promote + products derived from this software without specific prior written + permission. + +THIS SOFTWARE IS PROVIDED BY THE AUTHORS ''AS IS'' AND ANY EXPRESS OR +IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES +OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. +IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY DIRECT, INDIRECT, +INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, +BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED +AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. diff --git a/docs/README.rst b/docs/README.rst new file mode 100644 index 00000000..0578f15b --- /dev/null +++ b/docs/README.rst @@ -0,0 +1,109 @@ +Cowrie +###### + +|travis|_ +|codecov|_ + +Welcome to the Cowrie GitHub repository +***************************************** + +This is the official repository for the Cowrie SSH and Telnet +Honeypot effort. + +What is Cowrie +***************************************** + +Cowrie is a medium interaction SSH and Telnet honeypot designed to +log brute force attacks and the shell interaction performed by the +attacker. + +`Cowrie `_ is developed by Michel Oosterhof. + +Slack +***************************************** + +You can join the Cowrie community at the following `Slack workspace `_. + +Features +***************************************** + +Some interesting features: + +* Fake filesystem with the ability to add/remove files. A full fake filesystem resembling a Debian 5.0 installation is included +* Possibility of adding fake file contents so the attacker can `cat` files such as `/etc/passwd`. Only minimal file contents are included +* Session logs are stored in an `UML Compatible `_ format for easy replay with original timings with the `bin/playlog` utility. +* Cowrie saves files downloaded with wget/curl or uploaded with SFTP and scp for later inspection log + +Additional functionality over standard kippo: + +* SFTP and SCP support for file upload +* Support for SSH exec commands +* Logging of direct-tcp connection attempts (ssh proxying) +* Forward SMTP connections to SMTP Honeypot (e.g. `mailoney `_) +* Logging in JSON format for easy processing in log management solutions +* Many, many additional commands + +Docker +***************************************** + +Docker versions are available. + +* To get started quickly and give Cowrie a try, run:: + + docker run -p 2222:2222 cowrie/cowrie + ssh -p 2222 root@localhost + +* On Docker Hub: https://hub.docker.com/r/cowrie/cowrie + +* Or get the Dockerfile directly at https://github.com/cowrie/docker-cowrie + +Requirements +***************************************** + +Software required: + +* Python 2.7+, (Limited Python 3 support available for SSH only) +* python-virtualenv + +For Python dependencies, see requirements.txt + +Files of interest: +***************************************** + +* `cowrie.cfg` - Cowrie's configuration file. Default values can be found in `etc/cowrie.cfg.dist` +* `share/cowrie/fs.pickle` - fake filesystem +* `etc/userdb.txt` - credentials allowed or disallowed to access the honeypot +* `honeyfs/` - file contents for the fake filesystem - feel free to copy a real system here or use `bin/fsctl` +* `honeyfs/etc/issue.net` - pre-login banner +* `honeyfs/etc/motd` - post-login banner +* `var/log/cowrie/cowrie.json` - transaction output in JSON format +* `var/log/cowrie/cowrie.log` - log/debug output +* `var/lib/cowrie/tty/` - session logs, replayable with the `bin/playlog` utility. +* `var/lib/cowrie/downloads/` - files transferred from the attacker to the honeypot are stored here +* `share/cowrie/txtcmds/` - file contents for simple fake commands +* `bin/createfs` - used to create the fake filesystem +* `bin/playlog` - utility to replay session logs + +I have some questions! +***************************************** + +Please visit the `Slack workspace `_ and join the #questions channel. + +Contributors +*************** + +Many people have contributed to Cowrie over the years. Special thanks to: + +* Upi Tamminen (desaster) for all his work developing Kippo on which Cowrie was based +* Dave Germiquet (davegermiquet) for TFTP support, unit tests, new process handling +* Olivier Bilodeau (obilodeau) for Telnet support +* Ivan Korolev (fe7ch) for many improvements over the years. +* Florian Pelgrim (craneworks) for his work on code cleanup and Docker. +* And many many others. + + +.. |travis| image:: https://travis-ci.org/cowrie/cowrie.svg?branch=master +.. _travis: https://travis-ci.org/cowrie/cowrie + +.. |codecov| image:: https://codecov.io/gh/cowrie/cowrie/branch/master/graph/badge.svg +.. _codecov: https://codecov.io/gh/cowrie/cowrie