setting up a server

Would you—

• like to share your image collection between your computers, creating a simultaneous backup?
• like to share images amongst friends?
• like to publish regular manga or comic updates through a system that plugs natively into hydrus?
• like to tag all these images, possibly with others' help, according to a specific format that you determine?
• like to be a cool dude in cyberspace, respected by all the other console jockeys?

If so, you may wish to try running your own server. But first:

• I—the person who writes all the code and administrates the public tag repository—have no way of knowing where other servers or services are being run, nor who runs them, nor what is being stored on them. The servers never phone home. Feel free to check and run from source if you are paranoid.

  • Corollary: If you have a problem with something on some dude's server, please, do not come to me, as I can in no way help with your problem. If your ex-gf's nudes have leaked onto the internet, or you find something terribly offensive, or you just plain hate the free flow of information, I cannot help you at all. IT IS THE DAWN OF 20XX. WELCOME TO INTERNET
• This is not for noobs. If you are wholly unfamiliar with NAT (opening ports on firewalls) and setting up servers in general, this may be a step too far. If you get stuck, drag a nerd away from Xel'Naga Caverns to give you a hand.
• If you want your services to thrive in a community, you shall have to manage them. If you throw out a hundred uploader keys and go on a long vacation, do not expect your long-crafted thirteen-point 'Superman vs Goku ONLY!!!!!' ruleset to be upheld. Your repositories will clog with unrelated rubbish and your bandwidth will disappear. Banning worthless people and pruning worthless content is an important part of any online ecosystem.
• There are legal issues with running a file-hosting service. If you plan to do this on any kind of large scale, especially if you plan to sell access keys for cash-money, familiarise yourself with your legal rights and responsibilities. You may—without even meaning to—make yourself liable for others' sins.

still keen?

If this stuff be-fuzzles you, and you just want a simple server set up on the same computer you run the client, you can now go help->i don't know what I am doing->just set up the server on this computer, please and you _should_ be all set up with an admin service and tag/file repos automatically.

In discussing the hydrus network's server, I shall use two terms, server and service, to mean two distinct concepts:

• A server is an instantiation of the executable process, server.exe. It has a complicated and flexible database that can run many different services in parallel.
• A service sits on a port (e.g. 45871) and responds to certain http requests (e.g. /file or /update) that the hydrus client can plug into. A service might be a repository for a certain kind of data, the administration interface to manage what services run on a server, or anything else.

Setting up a hydrus server is easy compared to, say, Apache. There are no .conf files to mess about with, and everything is controlled through the client. When started, the server does nothing more visible than placing an icon in your system tray. Right click that, and you only get an option to exit.

The basic process for setting up a server is thus:

• Start the server.
• Set up your client with its address and initialise the administration interface
• Set the server's options and services.
• Make some access keys for your users.
• ???
• Profit

Let us look at these steps in more detail:

start the server

Since the server and client have so much common code, I have packaged them together. If you have the client, you have the server. To start it, you can hit the shortcut in your start menu or just go straight for server.exe/.pyw in the install directory. It will first try to take port 45870 or its administration interface, so make sure that is free. Open your firewall as appropriate.

set up the client

In the add, remove or edit services dialog, go to the servers admin tab, give your server admin interface a nickname and set the credentials to whatever-hostname:45870. Don't enter any access key. Ok those changes, and go to review services.

On the tab+page for your new server, hit the initialise button. If you have everything set right, the server should generate its first administrator account and return the access key, which the client will set for you. It will also try to save the key inside a text file.

YOU'LL WANT TO SAVE THAT FILE IN A SAFE PLACE

If you lose your admin access key, there is no way to get it back, and if you are not sqlite-proficient, you'll have to restart from the beginning by deleting your server's database files.

If the client can't connect to the server, it is either not running or you have a firewall/port-mapping problem. Google will help you with NAT. If you want a quick way to test the server's visibility, just put its host:port into your browser; if working, it should return some simple html identifying itself.

set up the server

You should notice a new menu, 'admin', in the client gui. This is where you control most server-wide stuff. admin->your server->options does exactly what you think.

admin->your server->manage services lets you add, edit, and delete the services your server runs. Every time you add one, you should also be added as that service's first administrator, with the same access key as for the admin interface.

Once you add a new service, the admin menu should give you another entry. Everything is fairly self-explanatory.

making access keys

Go admin->your service->create new accounts. You can then create x access keys of account type y and try to save them to a text file.

Go admin->manage account types to add, remove, or edit account types. Make sure everyone has at least downloader (get_data) permissions so they can stay synchronised.

You can create as many accounts of whatever kind you like, and distribute them the same. Depending on your usage scenario, you may want to have all uploaders, one uploader and many downloaders, or just a single administrator. There are many combinations.

It is your server, so do what you like.

On that note: python is simple enough that changes to the source are easy. If you want to auto-ban anyone who does not fit a certain quota, it is not that hard to find where the magic happens and alter it. Just do not test your changes on a live repository!

???

The most important part is to have fun! There are no losers on the INFORMATION SUPERHIGHWAY.

profit

I honestly hope you can get some benefit out of my code, whether just as a backup or as part of a far more complex system. Please mail me your comments as I am keen to make improvements.

btw, how to backup a repo's db

All of a server's files and options are stored in its accompanying .db file and respective subdirectories, which are created on first startup (just like with the client). You can backup and restore these files just by copying them about, but you have to be careful how you do it with a server; when it is running, it has a live connection to its database, and all sorts of things could be written or read at any time. If you just try to copy the .db somewhere and someone uploads a file, something might break. Instead, you have two options:

• Shut down the server, copy the .db files, then restart it. This is the only way, currently, to restore a db.
• Hit admin->your server->make a backup. This will lock the db server-side while it makes a copy right in the /db directory. The .db file will be tidied up and copied to .db.backup and the subdirectories will be copied to _backup as well. When the operation is complete, you can ftp/batch-copy/whatever these backup files wherever you like.

OMG EVERYTHING WENT WRONG

If you get to a point where you can no longer boot the repository, try running SQLite Studio and opening server.db. If the issue is simple—like manually changing the port number—you may be in luck. Send me an email if it is tricky.

If you somehow damage your computer and want to blame me, try taking some responsibility for your mistakes. Everything is breaking all the time. Make backups.