Rooba
/
hydrus
mirror of https://github.com/hydrusnetwork/hydrus.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Version 541

This commit is contained in:
Hydrus Network Developer 2023-08-30 11:25:24 -05:00
parent 785e37345f
commit e47a007a01
No known key found for this signature in database
GPG Key ID: 76249F053212133C
46 changed files with 3776 additions and 716 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

79
docs/changelog.md
View File

@ -7,6 +7,53 @@ title: Changelog
!!! note
This is the new changelog, only the most recent builds. For all versions, see the [old changelog](old_changelog.html).
## [Version 541](https://github.com/hydrusnetwork/hydrus/releases/tag/v541)
* misc
* fixed the gallery downloader and thread watcher loading with the 'clear highlight' button enabled despite there being nothing currently highlighted
* to fix the darkmode tooltips on the new Qt 6.5.2 on Windows (the text is stuck on a dark grey, which is unreadable in darkmodes), all the default darkmode styles now have an 'alternate-tooltip-colour' variant, which swaps out the tooltip background colour for the much brighter normal widget text colour
* rewrote the apng parser to work much faster on large files. someone encountered a 200MB giga apng that locked up the client for minutes. now it takes a second or two (unfortunately it looks like that huge apng breaks mpv, but there we go)
* the 'media' options page has two new checkboxes--'hide uninteresting import/modified times'--which allow you to turn off the media viewer behaivour where import and modified times similar to the 'added to my files xxx days ago' are hidden
* reworked the layout of the 'media' options page. everything is in sections now and re-ordered a bit
* the 'other file is a pixel-for-pixel duplicate png!' statements will now only show if the complement is a jpeg, gif, or webp. this statement isn't so appropriate for formats like PSD
* a variety of tricky tags like `:>=` are now searchable in normal autocomplete lookup. a test that determined whether to use a slower but more capable search was misfiring
* the client api key editing window has a new 'check all permissions' button
* fixed the updates I made last week to the missing-master-file-id recovery system. I made a stupid typo and didn't test it properly, fixed now. sorry for the trouble!
* thanks to a user, the help has a bunch of updated screenshots and fixed references to old concepts
* did a little more reformatting and cleanup of 'getting started with downloading' help document and added a short section on note import options
* cleaned up some of the syntax in our various batch files. fingers crossed, the setup_venv.bat script will absolutely retain the trailing space after its questions now, no matter what whitespace my IDE and github want to trim
### string joiner
* the parsing system has a new String Processor object--the 'String Joiner'. this is a simple concatenator that takes the list of strings and joins them together. it has two variables: what joining text to use, e.g. ', ', or '-', or empty string '' for simple concatenation; and an optional 'group size', which lets you join every two or three or n strings in 1-2-3, 1-2-3, 1-2-3 style patterns
### new file types
* thanks to a user; we now have support for QOI (a png-like lossless image type) and procreate (Apple image project file) files. the former has full support; the latter has thumbnails
* QOI needs Pillow 9.5 at least, so if you are on a super old 'running from source' version, try rebuilding your venv; or cope with you QOI-lessness
### client api
* thanks to a user, we now have `/add_tags/get_siblings_and_parents`, which, given a set of tags, shows their sibling and parent display rules for each service
* I wrote some help and unit tests for this
* client api version is now 51
### file storage (mostly boring)
* the file storage system is creaky and ugly to use. I have prepped some longer-term upgrades, mostly by writing new tools and cleaning and reworking existing code. I am nowhere near to done, but I'd like us to have four new features in the nearish future:
* - dynamic-length subfolders (where instead of a fixed set of 256 x00-xff folders, we can bump up to 4096 x000-xfff, and beyond, based on total number of files)
* - setting fixed space limits on particular database locations (e.g. 'no more than 200GB of files here') to complement the current weight system
* - permitting multiple valid locations for a particular subfolder prefix
* - slow per-file background migration between valid subfolders, rather than the giganto folder-atomic program-blocking 'move files now' button in database maintenance
* so, it is pretty boring so far, but I did the following:
* wrote a new class to handle a specific file storage subfolder and spammed it everywhere, replacing previous location and prefix juggling
* wrote some new tools to scan and check the coverage of multiple locations and dynamic-length subfolders
* rewrote the file location database initialisation, storage, testing, updating, and repair to support multiple valid locations
* updated the database to hold 'max num bytes' per file storage location
* the feature to migrate the SQLite database files and then restart is removed from the 'migrate database' dialog. it was always ultrajank in a place that really shouldn't be, and it was completely user-unfriendly. just move things manually, while the client is closed
* the old 'recover and merge surplus database locations into the correct position' side feature in 'move files now' is removed. it was always a little jank, was very rarely actually helpful, and had zero reporting. it will return in the new system as a better one-shot maintenance job
* touched up the migrated database help a little
## [Version 540](https://github.com/hydrusnetwork/hydrus/releases/tag/v540)
### misc
@ -368,35 +415,3 @@ title: Changelog
* the `/get_files/file` command now has a `download=true` parameter which converts the `Content-Disposition` from `inline` (show the file) to `attachment` (auto-download or open save-as dialog) (issue #1375)
* added help and a unit test for the above
* client api version is now 47
## [Version 531](https://github.com/hydrusnetwork/hydrus/releases/tag/v531)
### misc
* fixed editing favourite searches, which I accidentally broke last week with the collect-by updates
* when you right-click a tag and get the siblings/parents menus, the list of copyable siblings, parents, and children is now truncated to 10 items each per service. stuff like pokemon has hundreds of children and for a very long time has been spamming giganto 11-column menus that cover the entire screen
* same menu truncation for the open/copy URLs menu. if there's a file that has 600 URLs for interesting technical reasons, it won't nuke you any more (issue #1037)
* updated the default pixiv file page parser, which recently broke for users who were not logged in. they seem to hide original size behind the login now, so if you do a lot of pixiv work, get Hydrus Companion or figure out a cookies.txt solution and get yourself logged in
* the downloader progress panels have a couple of status text improvements: first, they will stop saying 'waiting for a work slot' when the actual error is something unusual such as the gallery search hitting the file limit. second, when there is an unusual status and the downloader is in the paused state, it can now properly differentiate between 'paused' and 'pausing'
* some invalid URL strings now raise the correct error in the downloader system, causing them to be properly filtered away instead of sticking around and being unhelpful
* if there is a connection error because of an SSL issue, the network job is now retried like any other connection error. I originally thought these were all non-retryable like cert validation errors, but it seems some of them are just write timeouts etc.. during the negotiation, so let's see how it goes
* I believe I have fixed an error when selecting a tag in a list when that list had been previously shift-selected and then cleared and repopulated
* manage siblings and parents should be better about focusing the correct text input after they boot and load
* in future, if a taglist tries to deselect something it no longer has, it'll do an emergency 'deselect all' to exorcise the ghosts fully
* reworded the text around 'reset potential duplicates' action in the duplicates page to be more clear on what it does
* I tinkered with some of the shutdown code hoping to catch an odd issue of the exit 'last session' not saving correctly, but I don't think I figured the issue out. if you have noticed you boot up and get a session that missed up to the last 15 minutes of changes before you last shut down, please let me know you your details
* added a link to `tagrank`, a new Client API project at https://github.com/matjojo/tagrank, to the Client API help. it shows you pairs of comparison images over and over and uses `trueskill` ranking algorithm to figure out which tags are your favourite
* added a link to 'Send to Hydrus', a Client API project at https://github.com/Wyrrrd/send-to-hydrus, to the Client API help. it sends URLs from an Android device to your client
### client api
* as part of a plan to migrate to service_key indexing everywhere and reduce file_metadata bloat, the client api has a new `services` structure, a service information Object where `service_key` is the key. this is now in the `/get_services` call and `/get_files/file_metadata`, under `services` under the root. the old type-based structure in `/get_services` and the in-file embedding of service info in `/get_files/file_metadata` are still in place, so nothing breaks today, but I am officially declaring them deprecated, to be deleted in 2024, and recommend all Client API devs move to the new system before the new year
* the new service object also includes info on the local rating services. I'd like to add ratings to file_metadata fairly soon
* if you don't want the services object in `/get_files/file_metadata`, there's a new `include_services_object` param you can set to false to hide it
* updated the unit tests and client api help to reflect all this. main new section: https://hydrusnetwork.github.io/hydrus/developer_api.html#services_object
* the client api version is now 46
### update woes
* I somewhat successfully pounded my head against an issue where the first tab (usually 'my tags') was disappearing in the _manage tags/siblings/parents_ dialogs for some users. this bug, for real, seems to be the combination of (Python 3.11 + PyQt6 6.5.x + two tabs + total tab text characters > ~12 + tab selection is set to 1 during init event). Change any of those things and it doesn't happen. This is so weird a problem to otherwise normal code that I won't pivot all my 50-odd instances of tab selection to handle it and instead have hacked an answer for the three tag dialogs and filename tagging. Sorry for the trouble if you got this! Let me know if you see any more
* in a similar-but-different thing, PySide6 6.5.1 has a bug related to certain Signal connections. don't use it with hydrus, it messes up all my menus! their dev notes suggest they are going to have a fix/revert for 6.5.1.1

48
docs/database_migration.md
View File

@ -14,7 +14,7 @@ A hydrus client consists of three components:
It doesn't really matter where you put this. An SSD will load it marginally quicker the first time, but you probably won't notice. If you run it without command-line parameters, it will try to write to its own directory (to create the initial database), so if you mean to run it like that, it should not be in a protected place like _Program Files_.
2. **the actual database**
2. **the actual SQLite database**
The client stores all its preferences and current state and knowledge _about_ files--like file size and resolution, tags, ratings, inbox status, and so on and so on--in a handful of SQLite database files, defaulting to _install_dir/db_. Depending on the size of your client, these might total 1MB in size or be as much as 10GB.
@ -29,7 +29,7 @@ A hydrus client consists of three components:
## these components can be put on different drives { id="different_drives" }
Although an initial install will keep these parts together, it is possible to, say, run the database on a fast drive but keep your media in cheap slow storage. This is an excellent arrangement that works for many users. And if you have a very large collection, you can even spread your files across multiple drives. It is not very technically difficult, but I do not recommend it for new users.
Although an initial install will keep these parts together, it is possible to, say, run the SQLite database on a fast drive but keep your media in cheap slow storage. This is an excellent arrangement that works for many users. And if you have a very large collection, you can even spread your files across multiple drives. It is not very technically difficult, but I do not recommend it for new users.
Backing such an arrangement up is obviously more complicated, and the internal client backup is not sophisticated enough to capture everything, so I recommend you figure out a broader solution with a third-party backup program like FreeFileSync.
@ -44,8 +44,6 @@ Go _database->migrate database_, giving you this dialog:
![](images/db_migration.png)
This is an image from my old laptop's client. At that time, I had moved the main database and its files out of the install directory but otherwise kept everything together. Your situation may be simpler or more complicated.
To move your files somewhere else, add the new location, empty/remove the old location, and then click 'move files now'.
**Portable** means that the path is beneath the main db dir and so is stored as a relative path. Portable paths will still function if the database changes location between boots (for instance, if you run the client from a USB drive and it mounts under a different location).
@ -54,13 +52,11 @@ To move your files somewhere else, add the new location, empty/remove the old lo
The operations on this dialog are simple and atomic--at no point is your db ever invalid. Once you have the locations and ideal usage set how you like, hit the 'move files now' button to actually shuffle your files around. It will take some time to finish, but you can pause and resume it later if the job is large or you want to undo or alter something.
If you decide to move your actual database, the program will have to shut down first. Before you boot up again, you will have to create a new program shortcut:
## informing the software that the SQLite database is not in the default location { id="launch_parameter" }
## informing the software that the database is not in the default location { id="launch_parameter" }
A straight call to the hydrus_client executable will look for a SQLite database in _install_dir/db_. If one is not found, it will create one. If you move your database and then try to run the client again, it will try to create a new empty database in that old location!
A straight call to the hydrus_client executable will look for a database in _install_dir/db_. If one is not found, it will create one. So, if you move your database and then try to run the client again, it will try to create a new empty database in the previous location!
So, pass it a -d or --db_dir command line argument, like so:
To tell it about the new database location, pass it a `-d` or `--db_dir` command line argument, like so:
* `hydrus_client -d="D:\media\my_hydrus_database"`
* _--or--_
@ -70,11 +66,11 @@ So, pass it a -d or --db_dir command line argument, like so:
And it will instead use the given path. If no database is found, it will similarly create a new empty one at that location. You can use any path that is valid in your system, but I would not advise using network locations and so on, as the database works best with some clever device locking calls these interfaces may not provide.
Rather than typing the path out in a terminal every time you want to launch your external database, create a new shortcut with the argument in. Something like this, which is from my main development computer and tests that a fresh default install will run an existing database ok:
Rather than typing the path out in a terminal every time you want to launch your external database, create a new shortcut with the argument in. Something like this:
![](images/db_migration_shortcut.png)
Note that an install with an 'external' database no longer needs access to write to its own path, so you can store it anywhere you like, including protected read-only locations (e.g. in 'Program Files'). If you do move it, just double-check your shortcuts are still good and you are done.
Note that an install with an 'external' database no longer needs access to write to its own path, so you can store it anywhere you like, including protected read-only locations (e.g. in 'Program Files'). Just double-check your shortcuts are good.
## finally { id="finally" }
@ -90,22 +86,18 @@ As an example, let's say you started using the hydrus client on your HDD, and no
Specifically:
* Update your backup if you maintain one.
* Create an empty folder on your HDD that is outside of your current install folder. Call it 'hydrus_files' or similar.
* Create two empty folders on your SSD with names like 'hydrus\_db' and 'hydrus\_thumbnails'.
* Set the 'thumbnail location override' to 'hydrus_thumbnails'. You should get that new location in the list, currently empty but prepared to take all your thumbs.
* Hit 'move files now' to actually move the thumbnails. Since this involves moving a lot of individual files from a high-latency source, it will take a long time to finish. The hydrus client may hang periodically as it works, but you can just leave it to work on its own--it will get there in the end. You can also watch it do its disk work under Task Manager.
* Now hit 'add location' and select your new 'hydrus\_files'. 'hydrus\_files' should be added and willing to take 50% of the files.
* Select the old location (probably 'install\_dir/db/client\_files') and hit 'decrease weight' until it has weight 0 and you are prompted to remove it completely. 'hydrus_files' should now be willing to take all the files from the old location.
* Hit 'move files now' again to make this happen. This should be fast since it is just moving a bunch of folders across the same partition.
* With everything now 'non-portable' and hence decoupled from the db, you can now easily migrate the install and db to 'hydrus_db' simply by shutting the client down and moving the install folder in a file explorer.
* Update your shortcut to the new hydrus_client.exe location and try to boot.
* Update your backup scheme to match your new locations.
* Enjoy a much faster client.
1. Update your backup if you maintain one.
* Create an empty folder on your HDD that is outside of your current install folder. Call it 'hydrus_files' or similar.
* Create two empty folders on your SSD with names like 'hydrus\_db' and 'hydrus\_thumbnails'.
* Set the 'thumbnail location override' to 'hydrus_thumbnails'. You should get that new location in the list, currently empty but prepared to take all your thumbs.
* Hit 'move files now' to actually move the thumbnails. Since this involves moving a lot of individual files from a high-latency source, it will take a long time to finish. The hydrus client may hang periodically as it works, but you can just leave it to work on its own--it will get there in the end. You can also watch it do its disk work under Task Manager.
* Now hit 'add location' and select your new 'hydrus\_files'. 'hydrus\_files' should be added and willing to take 50% of the files.
* Select the old location (probably 'install\_dir/db/client\_files') and hit 'decrease weight' until it has weight 0 and you are prompted to remove it completely. 'hydrus_files' should now be willing to take all the files from the old location.
* Hit 'move files now' again to make this happen. This should be fast since it is just moving a bunch of folders across the same partition.
* With everything now 'non-portable' and hence decoupled from the db, you can now easily migrate the install and db to 'hydrus_db' simply by shutting the client down and moving the install folder in a file explorer.
* Update your shortcut to the new hydrus_client.exe location and try to boot.
* Update your backup scheme to match your new locations.
* Enjoy a much faster client.
You should now have _something_ like this:
@ -113,4 +105,4 @@ You should now have _something_ like this:
## p.s. running multiple clients { id="multiple_clients" }
Since you now know how to tell the software about an external database, you can, if you like, run multiple clients from the same install (and if you previously had multiple install folders, now you can now just use the one). Just make multiple shortcuts to the same hydrus_client executable but with different database directories. They can run at the same time. You'll save yourself a little memory and update-hassle. I do this on my laptop client to run a regular client for my media and a separate 'admin' client to do PTR petitions and so on.
Since you now know how to tell the software about an external database, you can, if you like, run multiple clients from the same install (and if you previously had multiple install folders, now you can now just use the one). Just make multiple shortcuts to the same hydrus_client executable but with different database directories. They can run at the same time. You'll save yourself a little memory and update-hassle.

102
docs/developer_api.md
View File

@ -852,9 +852,9 @@ _Ask the client about how it will see certain tags._
Restricted access:
: YES. Add Tags permission needed.
Required Headers: n/a
Arguments (in percent-encoded JSON):
:
* `tags`: (a list of the tags you want cleaned)
@ -876,12 +876,108 @@ Response:
Mostly, hydrus simply trims excess whitespace, but the other examples are rare issues you might run into. 'system' is an invalid namespace, tags cannot be prefixed with hyphens, and any tag starting with ':' is secretly dealt with internally as "\[no namespace\]:\[colon-prefixed-subtag\]". Again, you probably won't run into these, but if you see a mismatch somewhere and want to figure it out, or just want to sort some numbered tags, you might like to try this.
### **GET `/add_tags/get_siblings_and_parents`** { id="add_tags_get_siblings_and_parents" }
_Ask the client about tags' sibling and parent relationships._
Restricted access:
: YES. Add Tags permission needed.
Required Headers: n/a
Arguments (in percent-encoded JSON):
:
* `tags`: (a list of the tags you want info on)
Example request:
: Given tags `#!json [ "blue eyes", "samus aran" ]`:
```
/add_tags/get_siblings_and_parents?tags=%5B%22blue%20eyes%22%2C%20%22samus%20aran%22%5D
```
Response:
: An Object showing all the display relationships for each tag on each service. Also [The Services Object](#services_object).
```json title="Example response"
{
"services" : "The Services Object"
"tags" : {
"blue eyes" : {
"6c6f63616c2074616773" : {
"ideal_tag" : "blue eyes",
"siblings" : [
"blue eyes",
"blue_eyes",
"blue eye",
"blue_eye"
],
"descendants" : [],
"ancestors" : []
},
"877bfcf81f56e7e3e4bc3f8d8669f92290c140ba0acfd6c7771c5e1dc7be62d7": {
"ideal_tag" : "blue eyes",
"siblings" : [
"blue eyes"
],
"descendants" : [],
"ancestors" : []
}
},
"samus aran" : {
"6c6f63616c2074616773" : {
"ideal_tag" : "character:samus aran",
"siblings" : [
"samus aran",
"samus_aran",
"character:samus aran"
],
"descendants" : [
"character:samus aran (zero suit)"
"cosplay:samus aran"
],
"ancestors" : [
"series:metroid",
"studio:nintendo"
]
},
"877bfcf81f56e7e3e4bc3f8d8669f92290c140ba0acfd6c7771c5e1dc7be62d7": {
"ideal_tag" : "samus aran",
"siblings" : [
"samus aran"
],
"descendants" : [
"zero suit samus",
"samus_aran_(cosplay)"
],
"ancestors" : []
}
}
}
}
```
This data is essentially how mappings in the `storage` `tag_display_type` become `display`.
The hex keys are the service keys, which you will have seen elsewhere, like [GET /get\_files/file\_metadata](#get_files_file_metadata). Note that there is no concept of 'all known tags' here. If a tag is in 'my tags', it follows the rules of 'my tags', and then all the services' display tags are merged into the 'all known tags' pool for user display.
**Also, the siblings and parents here are not just what is in _tags->manage tag siblings/parents_, they are the final computed combination of rules as set in _tags->manage where tag siblings and parents apply_.** The data given here is not guaranteed to be useful for editing siblings and parents on a particular service. That data, which is currently pair-based, will appear in a different API request in future.
- `ideal_tag` is how the tag appears in normal display to the user.
- `siblings` is every tag that will show as the `ideal_tag`, including the `ideal_tag` itself.
- `descendants` is every child (and recursive grandchild, great-grandchild...) that implies the `ideal_tag`.
- `ancestors` is every parent (and recursive grandparent, great-grandparent...) that our tag implies.
Every descendant and ancestor is an `ideal_tag` itself that may have its own siblings.
Most situations are simple, but remember that siblings and parents in hydrus can get complex. If you want to display this data, I recommend you plan to support simple service-specific workflows, and add hooks to recognise conflicts and other difficulty and, when that happens, abandon ship (send the user back to Hydrus proper). Also, if you show summaries of the data anywhere, make sure you add a 'and 22 more...' overflow mechanism to your menus, since if you hit up 'azur lane' or 'pokemon', you are going to get hundreds of children.
I generally warn you off computing sibling and parent mappings or counts yourself. The data from this request is best used for sibling and parent decorators on individual tags in a 'manage tags' presentation. The code that actually computes what siblings and parents look like in the 'display' context can be a pain at times, and I've already done it. Just run /search_tags or /file_metadata again after any changes you make and you'll get updated values.
### **GET `/add_tags/search_tags`** { id="add_tags_search_tags" }
_Search the client for tags._
Restricted access:
: YES. Search for Files permission needed.
: YES. Search for Files and Add Tags permission needed.
Required Headers: n/a

107
docs/getting_started_downloading.md
View File

@ -14,7 +14,9 @@ The downloader is highly parallelisable, and while the default [bandwidth rules]
It also takes a decent whack of CPU to import a file. You'll usually never notice this with just one hard drive import going, but if you have twenty different download queues all competing for database access and individual 0.1-second hits of heavy CPU work, you will discover your client starts to judder and lag. Keep it in mind, and you'll figure out what your computer is happy with. I also recommend you try to keep your total loaded files/urls to be under 20,000 to keep things snappy. Remember that you can pause your import queues, if you need to calm things down a bit.
## Downloader types
There are a number of different downloader types, each with its own purpose. This is a short summary of them:
There are a number of different downloader types, each with its own purpose:
**URL download**
: Intended for single posts or images. (Works with the [API](client_api.md))
@ -30,54 +32,15 @@ There are a number of different downloader types, each with its own purpose. Thi
**Simple downloader**
: Intended for simple one-off jobs like grabbing all linked images in a page.
# Import options
In previous versions these were split into completely different windows called `file import options` and `tag import options` so if you see those anywhere, this is what they're talking about and not some hidden menu anywhere.
## File import settings
File import settings has a number of options that deal with the files being downloaded and what should happen to them. There's a few more tickboxes if you turn on advanced mode.
![](images/file_import.png)
**pre-import checks**
: Pretty self-explanatory for the most part. If you want to redownload previously deleted files turning off `exclude previously deleted files` will have Hydrus ignore deletion status.
A few of the options have more information if you hover over them.
**import destinations**
: See [multiple file services](advanced_multiple_local_file_services.md), an advanced feature.
**post import actions**
: See the [files section on filtering](getting_started_files.md#inbox-and-archive) for the first option, the other two have information if you hover over them.
## Parsing
By default, hydrus now starts with a local tag service called 'downloader tags' and it will parse (get) all the tags from normal gallery sites and put them in this service. You don't have to do anything, you will get some decent tags. As you use the client, you will figure out which tags you like and where you want them. On the downloader page, click `import options`:
![](images/tag_import_options_default.png)
This is an important dialog, although you will not need to use it much. It governs which tags are parsed and where they go. To keep things easy to manage, a new downloader will refer to the 'default' tag import options for a website, but for now let's set some values just for this downloader:
![](images/tag_import_options_specific.png)
You can see that each tag service on your client has a separate section. If you add the PTR, that will get a new box too. A new client is set to _get all tags_ for 'downloader tags' service. Things can get much more complicated. Have a play around with the options here as you figure things out. Most of the controls have tooltips or longer explainers in sub-dialogs, so don't be afraid to try things.
It is easy to get tens of thousands of tags by downloading this way. Different sites offer different kinds and qualities of tags, and the client's downloaders (which were designed by me, the dev, or a user) may parse all or only some of them. Many users like to just get everything on offer, but others only ever want, say, `creator`, `series`, and `character` tags. If you feel brave, click that 'all tags' button, which will take you into hydrus's advanced 'tag filter', which allows you to select which of the incoming list of tags will be added.
The blacklist button will let you skip downloading files that have certain tags (perhaps you would like to auto-skip all images with `gore`, `scat`, or `diaper`?), again using the tag filter, while the whitelist enables you to only allow files that have at least one of a set of tags. The 'additional tags' adds some fixed personal tags to all files coming in--for instance, you might like to add 'process into favourites' to your 'my tags' for some query you really like so you can find those files again later and process them separately. That little 'cog' icon button can also do some advanced things.
To edit the defaults, hit up _network->downloaders->manage default tag import options_. You should do this as you get a better idea of your preferences. You can set them for all file posts generally, all watchers, and for specific sites as well.
!!! warning
The file limit and file/tag import options on the upper panel, if changed, will only apply to **new** queries. If you want to change the options for an existing queue, either do so on its highlight panel below or use the 'set options to queries' button.
## URL download
### URL download
The **url downloader** works like the gallery downloader but does not do searches. You can paste downloadable URLs to it, and it will work through them as one list. Dragging and dropping recognisable URLs onto the client (e.g. from your web browser) will also spawn and use this downloader.
The button next to the input field lets you paste multiple URLs at once such as if you've copied from a document or browser bookmarks. The URLs need to be newline separated.
### API
#### API
If you use [API-connected](client_api.md) programs such as the Hydrus Companion, then any [non-watchable](downloader_url_classes.md#url_types) URLs sent to Hydrus through them will end up in an URL downloader page, the specifics depending on the program's settings. You can't use this to force Hydrus to download paged galleries since the URL downloader page doesn't support traversing to the next page, use the gallery downloader for this.
## Gallery download
### Gallery download
![](images/downloader_page.png)
The gallery page can download from multiple sources at the same time. Each entry in the list represents a basic combination of two things:
@ -100,7 +63,7 @@ _Note that some sites only serve 25 or 50 pages of results, despite their indice
**In general, particularly when starting out, artist searches are best.** They are usually fewer than a thousand files and have fairly uniform quality throughout.
## Subscriptions { id="subscriptions" }
### Subscriptions { id="subscriptions" }
Let's say you found an artist you like. You downloaded everything of theirs from some site, but every week, one or two new pieces is posted. You'd like to keep up with the new stuff, but you don't want to manually make a new download job every week for every single artist you like.
Subscriptions are a way to automatically recheck a good query in future, to keep up with new files. Many users come to use them. You set up a number of saved queries, and the client will 'sync' with the latest files in the gallery and download anything new, just as if you were running the download yourself.
@ -112,7 +75,7 @@ Subscriptions only work for booru-like galleries that put the newest files first
It is important to note that while subscriptions can have multiple queries (even hundreds!), they _generally_ only work on one site. Expect to create one subscription for safebooru, one for artstation, one for paheal, and so on for every site you care about. Advanced users may be able to think of ways to get around this, but I recommend against it as it throws off some of the internal check timing calculations.
### Setting up subscriptions
#### Setting up subscriptions
Here's the dialog, which is under _network->manage subscriptions_:
@ -136,7 +99,7 @@ Despite all the controls, the basic idea is simple: Up top, I have selected the
You might want to put subscriptions off until you are more comfortable with galleries. There is more help [here](getting_started_subscriptions.md).
## Watchers
### Watchers
If you are an imageboard user, try going to a thread you like and drag-and-drop its URL (straight from your web browser's address bar) onto the hydrus client. It should open up a new 'watcher' page and import the thread's files!
![](images/watcher_page.png)
@ -145,12 +108,60 @@ With only one URL to check, watchers are a little simpler than gallery searches,
In general, you can leave the checker options alone, but you might like to revisit them if you are always visiting faster or slower boards and find you are missing files or getting DEAD too early.
### API
#### API
If you use [API-connected](client_api.md) programs such as the Hydrus Companion, then any [watchable](downloader_url_classes.md#url_types) URLs sent to Hydrus through them will end up in a watcher page, the specifics depending on the program's settings.
## Simple downloader
### Simple downloader
The **simple downloader** will do very simple parsing for unusual jobs. If you want to download all the images in a page, or all the image link destinations, this is the one to use. There are several default parsing rules to choose from, and if you learn the downloader system yourself, it will be easy to make more.
## Import options
Every importer in Hydrus has some 'import options' that change what is allowed, what is blacklisted, and whether tags or notes should be saved.
In previous versions these were split into completely different windows called `file import options` and `tag import options` so if you see those anywhere, this is what they're talking about and not some hidden menu anywhere.
Importers that download from websites rely on a flexible 'defaults' system, so you do not have to set them up every time you start a new downloader. While you should play around with your import options, once you know what works for you, you should set that as the default under _network->downloaders->manage default import options_. You can set them for all file posts generally, all watchers, and for specific sites as well.
### File import options
This deals with the files being downloaded and what should happen to them. There's a few more tickboxes if you turn on advanced mode.
![](images/file_import.png)
**pre-import checks**
: Pretty self-explanatory for the most part. If you want to redownload previously deleted files turning off `exclude previously deleted files` will have Hydrus ignore deletion status.
A few of the options have more information if you hover over them.
**import destinations**
: See [multiple file services](advanced_multiple_local_file_services.md), an advanced feature.
**post import actions**
: See the [files section on filtering](getting_started_files.md#inbox-and-archive) for the first option, the other two have information if you hover over them.
### Tag Parsing
By default, hydrus now starts with a local tag service called 'downloader tags' and it will parse (get) all the tags from normal gallery sites and put them in this service. You don't have to do anything, you will get some decent tags. As you use the client, you will figure out which tags you like and where you want them. On the downloader page, click `import options`:
![](images/tag_import_options_default.png)
This is an important dialog, although you will not need to use it much. It governs which tags are parsed and where they go. To keep things easy to manage, a new downloader will refer to the 'default' tag import options for a website, but for now let's set some values just for this downloader:
![](images/tag_import_options_specific.png)
You can see that each tag service on your client has a separate section. If you add the PTR, that will get a new box too. A new client is set to _get all tags_ for 'downloader tags' service. Things can get much more complicated. Have a play around with the options here as you figure things out. Most of the controls have tooltips or longer explainers in sub-dialogs, so don't be afraid to try things.
It is easy to get tens of thousands of tags by downloading this way. Different sites offer different kinds and qualities of tags, and the client's downloaders (which were designed by me, the dev, or a user) may parse all or only some of them. Many users like to just get everything on offer, but others only ever want, say, `creator`, `series`, and `character` tags. If you feel brave, click that 'all tags' button, which will take you into hydrus's advanced 'tag filter', which allows you to select which of the incoming list of tags will be added.
The blacklist button will let you skip downloading files that have certain tags (perhaps you would like to auto-skip all images with `gore`, `scat`, or `diaper`?), again using the tag filter, while the whitelist enables you to only allow files that have at least one of a set of tags. The 'additional tags' adds some fixed personal tags to all files coming in--for instance, you might like to add 'process into favourites' to your 'my tags' for some query you really like so you can find those files again later and process them separately. That little 'cog' icon button can also do some advanced things.
!!! warning
The file limit and import options on the upper panel of a gallery or watcher page, if changed, will only apply to **new** queries. If you want to change the options for an existing queue, either do so on its highlight panel below or use the 'set options to queries' button.
### Note Parsing
Hydrus alsos parse 'notes' from some sites. This is a young feature, and a little advanced at times, but it generally means the comments that artists leave on certain gallery sites, or something like a tweet text. Notes are editable by you and appear in a hovering window on the right side of the media viewer.
![](images/note_import_options_normal.png)
Most of the controls here ensure that successive parses do not duplicate existing notes. The default settings are fine for all normal purposes, and you can leave them alone unless you know you want something special (e.g. turning note parsing off completely).
## Bandwidth
It will not be too long until you see a "bandwidth free in xxxxx..." message. As a long-term storage solution, hydrus is designed to be polite in its downloading--both to the source server and your computer. The client's default bandwidth rules have some caps to stop big mistakes, spread out larger jobs, and at a bare minimum, no domain will be hit more than once a second.

BIN
docs/images/db_migration.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 24 KiB

After

Width:  |  Height:  |  Size: 30 KiB

BIN
docs/images/db_migration_example.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 24 KiB

After

Width:  |  Height:  |  Size: 31 KiB

BIN
docs/images/note_import_options_normal.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 25 KiB

41
docs/old_changelog.html
View File

@ -34,6 +34,47 @@
<div class="content">
<h1 id="changelog"><a href="#changelog">changelog</a></h1>
<ul>
<li>
<h2 id="version_541"><a href="#version_541">version 541</a></h2>
<ul>
<li>misc</li>
<li>fixed the gallery downloader and thread watcher loading with the 'clear highlight' button enabled despite there being nothing currently highlighted</li>
<li>to fix the darkmode tooltips on the new Qt 6.5.2 on Windows (the text is stuck on a dark grey, which is unreadable in darkmodes), all the default darkmode styles now have an 'alternate-tooltip-colour' variant, which swaps out the tooltip background colour for the much brighter normal widget text colour</li>
<li>rewrote the apng parser to work much faster on large files. someone encountered a 200MB giga apng that locked up the client for minutes. now it takes a second or two (unfortunately it looks like that huge apng breaks mpv, but there we go)</li>
<li>the 'media' options page has two new checkboxes--'hide uninteresting import/modified times'--which allow you to turn off the media viewer behaivour where import and modified times similar to the 'added to my files xxx days ago' are hidden</li>
<li>reworked the layout of the 'media' options page. everything is in sections now and re-ordered a bit</li>
<li>the 'other file is a pixel-for-pixel duplicate png!' statements will now only show if the complement is a jpeg, gif, or webp. this statement isn't so appropriate for formats like PSD</li>
<li>a variety of tricky tags like `:>=` are now searchable in normal autocomplete lookup. a test that determined whether to use a slower but more capable search was misfiring</li>
<li>the client api key editing window has a new 'check all permissions' button</li>
<li>fixed the updates I made last week to the missing-master-file-id recovery system. I made a stupid typo and didn't test it properly, fixed now. sorry for the trouble!</li>
<li>thanks to a user, the help has a bunch of updated screenshots and fixed references to old concepts</li>
<li>did a little more reformatting and cleanup of 'getting started with downloading' help document and added a short section on note import options</li>
<li>cleaned up some of the syntax in our various batch files. fingers crossed, the setup_venv.bat script will absolutely retain the trailing space after its questions now, no matter what whitespace my IDE and github want to trim</li>
<li><h3>string joiner</h3></li>
<li>the parsing system has a new String Processor object--the 'String Joiner'. this is a simple concatenator that takes the list of strings and joins them together. it has two variables: what joining text to use, e.g. ', ', or '-', or empty string '' for simple concatenation; and an optional 'group size', which lets you join every two or three or n strings in 1-2-3, 1-2-3, 1-2-3 style patterns</li>
<li><h3>new file types</h3></li>
<li>thanks to a user; we now have support for QOI (a png-like lossless image type) and procreate (Apple image project file) files. the former has full support; the latter has thumbnails</li>
<li>QOI needs Pillow 9.5 at least, so if you are on a super old 'running from source' version, try rebuilding your venv; or cope with you QOI-lessness</li>
<li><h3>client api</h3></li>
<li>thanks to a user, we now have `/add_tags/get_siblings_and_parents`, which, given a set of tags, shows their sibling and parent display rules for each service</li>
<li>I wrote some help and unit tests for this</li>
<li>client api version is now 51</li>
<li><h3>file storage (mostly boring)</h3></li>
<li>the file storage system is creaky and ugly to use. I have prepped some longer-term upgrades, mostly by writing new tools and cleaning and reworking existing code. I am nowhere near to done, but I'd like us to have four new features in the nearish future: </li>
<li>- dynamic-length subfolders (where instead of a fixed set of 256 x00-xff folders, we can bump up to 4096 x000-xfff, and beyond, based on total number of files)</li>
<li>- setting fixed space limits on particular database locations (e.g. 'no more than 200GB of files here') to complement the current weight system</li>
<li>- permitting multiple valid locations for a particular subfolder prefix</li>
<li>- slow per-file background migration between valid subfolders, rather than the giganto folder-atomic program-blocking 'move files now' button in database maintenance</li>
<li>so, it is pretty boring so far, but I did the following: </li>
<li>wrote a new class to handle a specific file storage subfolder and spammed it everywhere, replacing previous location and prefix juggling</li>
<li>wrote some new tools to scan and check the coverage of multiple locations and dynamic-length subfolders</li>
<li>rewrote the file location database initialisation, storage, testing, updating, and repair to support multiple valid locations</li>
<li>updated the database to hold 'max num bytes' per file storage location</li>
<li>the feature to migrate the SQLite database files and then restart is removed from the 'migrate database' dialog. it was always ultrajank in a place that really shouldn't be, and it was completely user-unfriendly. just move things manually, while the client is closed</li>
<li>the old 'recover and merge surplus database locations into the correct position' side feature in 'move files now' is removed. it was always a little jank, was very rarely actually helpful, and had zero reporting. it will return in the new system as a better one-shot maintenance job</li>
<li>touched up the migrated database help a little</li>
</ul>
</li>
<li>
<h2 id="version_540"><a href="#version_540">version 540</a></h2>
<ul>

1
docs/running_from_source.md
View File

@ -57,6 +57,7 @@ There are three special external libraries. You just have to get them and put th
1. If you are on Windows 8.1 or older, [this](https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20210228-git-d1be8bb.7z) is known safe.
2. If you are on Windows 10 or newer and want the simple answer, try [this](https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20220501-git-9ffaa6b.7z).
3. Ideally, go for [this](https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20230212-git-a40958c.7z), but you have to rename the dll to `mpv-2.dll`.
4. I have been testing [this newer version](https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20230820-git-19384e0.7z) and things seem to be fine too, at least on updated Windows.
Then open that archive and place the 'mpv-1.dll' or 'mpv-2.dll' into `install_dir`.

4
git_pull.bat
View File

@ -5,7 +5,7 @@ pushd "%~dp0"
where /q git
IF ERRORLEVEL 1 (
SET /P gumpf=You do not seem to have git installed!
SET /P gumpf="You do not seem to have git installed!"
popd
@ -17,4 +17,4 @@ git pull
popd
SET /P done=Done!
SET /P done="Done!"

14
hydrus/client/ClientController.py
View File

@ -976,11 +976,11 @@ class Controller( HydrusController.HydrusController ):
def InitClientFilesManager( self ):
def qt_code( missing_locations ):
def qt_code( missing_subfolders ):
with ClientGUITopLevelWindowsPanels.DialogManage( None, 'repair file system' ) as dlg:
panel = ClientGUIScrolledPanelsManagement.RepairFileSystemPanel( dlg, missing_locations )
panel = ClientGUIScrolledPanelsManagement.RepairFileSystemPanel( dlg, missing_subfolders )
dlg.SetPanel( panel )
@ -988,7 +988,7 @@ class Controller( HydrusController.HydrusController ):
self.client_files_manager = ClientFiles.ClientFilesManager( self )
missing_locations = self.client_files_manager.GetMissing()
missing_subfolders = self.client_files_manager.GetMissingSubfolders()
else:
@ -996,18 +996,18 @@ class Controller( HydrusController.HydrusController ):
return missing_locations
return missing_subfolders
self.client_files_manager = ClientFiles.ClientFilesManager( self )
self.files_maintenance_manager = ClientFiles.FilesMaintenanceManager( self )
missing_locations = self.client_files_manager.GetMissing()
missing_subfolders = self.client_files_manager.GetMissingSubfolders()
while len( missing_locations ) > 0:
while len( missing_subfolders ) > 0:
missing_locations = self.CallBlockingToQt( self._splash, qt_code, missing_locations )
missing_subfolders = self.CallBlockingToQt( self._splash, qt_code, missing_subfolders )

11
hydrus/client/ClientDuplicates.py
View File

@ -91,17 +91,24 @@ def GetDuplicateComparisonStatements( shown_media, comparison_media ):
s_pixel_hash = hashes_to_pixel_hashes[ s_hash ]
c_pixel_hash = hashes_to_pixel_hashes[ c_hash ]
# this is not appropriate for, say, PSD files
other_file_is_pixel_png_appropriate_filetypes = {
HC.IMAGE_JPEG,
HC.IMAGE_GIF,
HC.IMAGE_WEBP
}
if s_pixel_hash == c_pixel_hash:
is_a_pixel_dupe = True
if s_mime == HC.IMAGE_PNG and c_mime != HC.IMAGE_PNG:
if s_mime == HC.IMAGE_PNG and c_mime in other_file_is_pixel_png_appropriate_filetypes:
statement = 'this is a pixel-for-pixel duplicate png!'
score = -100
elif s_mime != HC.IMAGE_PNG and c_mime == HC.IMAGE_PNG:
elif s_mime in other_file_is_pixel_png_appropriate_filetypes and c_mime == HC.IMAGE_PNG:
statement = 'other file is a pixel-for-pixel duplicate png!'

369
hydrus/client/ClientFiles.py
View File

@ -1,6 +1,5 @@
import collections
import os
import queue
import random
import threading
import time
@ -20,6 +19,7 @@ from hydrus.core import HydrusTime
from hydrus.core.networking import HydrusNetworking
from hydrus.client import ClientConstants as CC
from hydrus.client import ClientFilesPhysical
from hydrus.client import ClientImageHandling
from hydrus.client import ClientPaths
from hydrus.client import ClientSVGHandling # important to keep this in, despite not being used, since there's initialisation stuff in here
@ -212,8 +212,6 @@ def GetAllFilePaths( raw_paths, do_human_sort = True, clear_out_sidecars = True
file_paths = []
num_sidecars = 0
paths_to_process = list( raw_paths )
while len( paths_to_process ) > 0:
@ -303,6 +301,7 @@ def GetAllFilePaths( raw_paths, do_human_sort = True, clear_out_sidecars = True
return ( file_paths, num_sidecars )
class ClientFilesManager( object ):
def __init__( self, controller ):
@ -311,27 +310,47 @@ class ClientFilesManager( object ):
self._rwlock = ClientThreading.FileRWLock()
self._prefixes_to_locations = {}
self._prefixes_to_client_files_subfolders = collections.defaultdict( list )
self._smallest_prefix = 2
self._largest_prefix = 2
self._physical_file_delete_wait = threading.Event()
self._locations_to_free_space = {}
self._bad_error_occurred = False
self._missing_locations = set()
self._missing_subfolders = set()
self._Reinit()
self._controller.sub( self, 'shutdown', 'shutdown' )
def _GetCurrentSubfolderLocations( self, only_files = False ):
known_locations = set()
for ( prefix, subfolders ) in self._prefixes_to_client_files_subfolders.items():
if only_files and not prefix.startswith( 'f' ):
continue
for subfolder in subfolders:
known_locations.add( subfolder.location )
return known_locations
def _GetFileStorageFreeSpace( self, hash: bytes ) -> int:
hash_encoded = hash.hex()
subfolder = self._GetSubfolderForFile( hash, 'f' )
prefix = 'f' + hash_encoded[:2]
location = self._prefixes_to_locations[ prefix ]
location = subfolder.location
if location in self._locations_to_free_space:
@ -367,6 +386,52 @@ class ClientFilesManager( object ):
return free_space
def _GetPossibleSubfoldersForFile( self, hash: bytes, prefix_type: str ) -> typing.List[ ClientFilesPhysical.FilesStorageSubfolder ]:
hash_encoded = hash.hex()
result = []
for i in range( self._smallest_prefix, self._largest_prefix + 1 ):
prefix = prefix_type + hash_encoded[ : i ]
if prefix in self._prefixes_to_client_files_subfolders:
result.extend( self._prefixes_to_client_files_subfolders[ prefix ] )
return result
def _GetAllSubfolders( self ):
result = []
for ( prefix, subfolders ) in self._prefixes_to_client_files_subfolders.items():
result.extend( subfolders )
return result
def _GetSubfolderForFile( self, hash: bytes, prefix_type: str ) -> ClientFilesPhysical.FilesStorageSubfolder:
# TODO: So this will be a crux of the more complicated system
# might even want a media result eventually, for various 'ah, because it is archived, it should go here'
# for now it is a patch to navigate multiples into our currently mutually exclusive storage dataset
# we probably need to break this guy into variants of 'getpossiblepaths' vs 'getidealpath' for different callers
# getideal would be testing purge states and client files locations max num bytes stuff
# there should, in all circumstances, be a place to put a file, so there should always be at least one non-num_bytes'd location with weight to handle 100% coverage of the spillover
# if we are over the limit on the place the directory is supposed to be, I think we are creating a stub subfolder in the spillover place and writing there, but that'll mean saving a new subfolder, so be careful
# maybe the spillover should always have 100% coverage no matter what, and num_bytes'd locations should always just have extensions. something to think about
return self._GetPossibleSubfoldersForFile( hash, prefix_type )[0]
def _HandleCriticalDriveError( self ):
self._controller.new_options.SetBoolean( 'pause_import_folders_sync', True )
@ -436,13 +501,9 @@ class ClientFilesManager( object ):
except Exception as e:
hash_encoded = hash.hex()
subfolder = self._GetSubfolderForFile( hash, 't' )
prefix = 't' + hash_encoded[:2]
location = self._prefixes_to_locations[ prefix ]
thumb_dir = os.path.join( location, prefix )
thumb_dir = subfolder.GetDirectory()
if not os.path.exists( thumb_dir ):
@ -468,9 +529,7 @@ class ClientFilesManager( object ):
fixes_counter = collections.Counter()
known_locations = set()
known_locations.update( self._prefixes_to_locations.values() )
known_locations = self._GetCurrentSubfolderLocations()
( locations_to_ideal_weights, thumbnail_override ) = self._controller.Read( 'ideal_client_files_locations' )
@ -481,7 +540,10 @@ class ClientFilesManager( object ):
known_locations.add( thumbnail_override )
for ( missing_location, prefix ) in self._missing_locations:
for subfolder in self._missing_subfolders:
missing_location = subfolder.location
prefix = subfolder.prefix
potential_correct_locations = []
@ -504,7 +566,7 @@ class ClientFilesManager( object ):
correct_location = potential_correct_locations[0]
correct_rows.append( ( prefix, correct_location ) )
correct_rows.append( ( prefix, missing_location, correct_location ) )
fixes_counter[ ( missing_location, correct_location ) ] += 1
@ -523,13 +585,13 @@ class ClientFilesManager( object ):
if len( correct_rows ) > 0:
summaries = sorted( ( '{} moved from {} to {}'.format( HydrusData.ToHumanInt( count ), missing_location, correct_location ) for ( ( missing_location, correct_location ), count ) in fixes_counter.items() ) )
summaries = sorted( ( '{} folders seem to have moved from {} to {}'.format( HydrusData.ToHumanInt( count ), missing_location, correct_location ) for ( ( missing_location, correct_location ), count ) in fixes_counter.items() ) )
summary_message = 'Some client file folders were missing, but they seem to be in other known locations! The folders are:'
summary_message = 'Some client file folders were missing, but they appear to be in other known locations! The folders are:'
summary_message += os.linesep * 2
summary_message += os.linesep.join( summaries )
summary_message += os.linesep * 2
summary_message += 'Assuming you did this on purpose, Hydrus is ready to update its internal knowledge to reflect these new mappings as soon as this dialog closes. If you know these proposed fixes are incorrect, terminate the program now.'
summary_message += 'Assuming you did this on purpose, or hydrus recently inserted stub values after database corruption, Hydrus is ready to update its internal knowledge to reflect these new mappings as soon as this dialog closes. If you know these proposed fixes are incorrect, terminate the program now.'
HydrusData.Print( summary_message )
@ -585,13 +647,13 @@ class ClientFilesManager( object ):
self._WaitOnWakeup()
subfolder = self._GetSubfolderForFile( hash, 'f' )
file_dir = subfolder.GetDirectory()
hash_encoded = hash.hex()
prefix = 'f' + hash_encoded[:2]
location = self._prefixes_to_locations[ prefix ]
path = os.path.join( location, prefix, hash_encoded + HC.mime_ext_lookup[ mime ] )
path = os.path.join( file_dir, hash_encoded + HC.mime_ext_lookup[ mime ] )
return path
@ -600,13 +662,13 @@ class ClientFilesManager( object ):
self._WaitOnWakeup()
subfolder = self._GetSubfolderForFile( hash, 't' )
thumb_dir = subfolder.GetDirectory()
hash_encoded = hash.hex()
prefix = 't' + hash_encoded[:2]
location = self._prefixes_to_locations[ prefix ]
path = os.path.join( location, prefix, hash_encoded ) + '.thumbnail'
path = os.path.join( thumb_dir, hash_encoded ) + '.thumbnail'
return path
@ -639,52 +701,10 @@ class ClientFilesManager( object ):
return thumbnail_bytes
def _GetRecoverTuple( self ):
all_locations = { location for location in self._prefixes_to_locations.values() }
all_prefixes = list(self._prefixes_to_locations.keys())
for possible_location in all_locations:
if not os.path.exists( possible_location ):
continue
for prefix in all_prefixes:
correct_location = self._prefixes_to_locations[ prefix ]
if correct_location == possible_location:
continue
if os.path.exists( os.path.join( possible_location, prefix ) ):
if not os.path.exists( correct_location ):
continue
if os.path.samefile( possible_location, correct_location ):
continue
recoverable_location = possible_location
return ( prefix, recoverable_location, correct_location )
return None
def _GetRebalanceTuple( self ):
# TODO: obviously this will change radically when we move to multiple folders for real and background migration. hacks for now
( locations_to_ideal_weights, thumbnail_override ) = self._controller.Read( 'ideal_client_files_locations' )
total_weight = sum( locations_to_ideal_weights.values() )
@ -693,16 +713,20 @@ class ClientFilesManager( object ):
current_locations_to_normalised_weights = collections.defaultdict( lambda: 0 )
file_prefixes = [ prefix for prefix in self._prefixes_to_locations if prefix.startswith( 'f' ) ]
file_prefixes = [ prefix for prefix in self._prefixes_to_client_files_subfolders.keys() if prefix.startswith( 'f' ) ]
for file_prefix in file_prefixes:
location = self._prefixes_to_locations[ file_prefix ]
subfolders = self._prefixes_to_client_files_subfolders[ file_prefix ]
subfolder = subfolders[0]
location = subfolder.location
current_locations_to_normalised_weights[ location ] += 1.0 / 256
for location in list(current_locations_to_normalised_weights.keys()):
for location in list( current_locations_to_normalised_weights.keys() ):
if location not in ideal_locations_to_normalised_weights:
@ -747,7 +771,11 @@ class ClientFilesManager( object ):
for file_prefix in file_prefixes:
location = self._prefixes_to_locations[ file_prefix ]
subfolders = self._prefixes_to_client_files_subfolders[ file_prefix ]
subfolder = subfolders[0]
location = subfolder.location
if location == overweight_location:
@ -757,6 +785,9 @@ class ClientFilesManager( object ):
else:
# TODO: this needs work too. either we guarantee we split thumb and file dirs at the same time, so we know there is a matching file prefix for any thumb one,
# or we learn how to match thumb dirs with their fxx000 parent or whatever, _and vice versa_
for hex_prefix in HydrusData.IterateHexPrefixes():
thumbnail_prefix = 't' + hex_prefix
@ -765,14 +796,22 @@ class ClientFilesManager( object ):
file_prefix = 'f' + hex_prefix
correct_location = self._prefixes_to_locations[ file_prefix ]
subfolders = self._prefixes_to_client_files_subfolders[ file_prefix ]
subfolder = subfolders[0]
correct_location = subfolder.location
else:
correct_location = thumbnail_override
current_thumbnails_location = self._prefixes_to_locations[ thumbnail_prefix ]
subfolders = self._prefixes_to_client_files_subfolders[ thumbnail_prefix ]
subfolder = subfolders[0]
current_thumbnails_location = subfolder.location
if current_thumbnails_location != correct_location:
@ -786,17 +825,20 @@ class ClientFilesManager( object ):
def _IterateAllFilePaths( self ):
for ( prefix, location ) in list(self._prefixes_to_locations.items()):
for ( prefix, subfolders ) in self._prefixes_to_client_files_subfolders.items():
if prefix.startswith( 'f' ):
dir = os.path.join( location, prefix )
filenames = list( os.listdir( dir ) )
for filename in filenames:
for subfolder in subfolders:
yield os.path.join( dir, filename )
files_dir = subfolder.GetDirectory()
filenames = list( os.listdir( files_dir ) )
for filename in filenames:
yield os.path.join( files_dir, filename )
@ -804,17 +846,20 @@ class ClientFilesManager( object ):
def _IterateAllThumbnailPaths( self ):
for ( prefix, location ) in list(self._prefixes_to_locations.items()):
for ( prefix, subfolders ) in self._prefixes_to_client_files_subfolders.items():
if prefix.startswith( 't' ):
dir = os.path.join( location, prefix )
filenames = list( os.listdir( dir ) )
for filename in filenames:
for subfolder in subfolders:
yield os.path.join( dir, filename )
files_dir = subfolder.GetDirectory()
filenames = list( os.listdir( files_dir ) )
for filename in filenames:
yield os.path.join( files_dir, filename )
@ -832,17 +877,16 @@ class ClientFilesManager( object ):
hash_encoded = hash.hex()
subfolders = self._GetPossibleSubfoldersForFile( hash, 'f' )
prefix = 'f' + hash_encoded[:2]
location = self._prefixes_to_locations[ prefix ]
subdir = os.path.join( location, prefix )
if not os.path.exists( subdir ):
for subfolder in subfolders:
raise HydrusExceptions.DirectoryMissingException( 'The directory {} was not found! Reconnect the missing location or shut down the client immediately!'.format( subdir ) )
files_dir = subfolder.GetDirectory()
if not os.path.exists( files_dir ):
raise HydrusExceptions.DirectoryMissingException( 'The directory {} was not found! Reconnect the missing location or shut down the client immediately!'.format( files_dir ) )
raise HydrusExceptions.FileMissingException( 'File for ' + hash.hex() + ' not found!' )
@ -850,24 +894,39 @@ class ClientFilesManager( object ):
def _Reinit( self ):
self._prefixes_to_locations = self._controller.Read( 'client_files_locations' )
self._ReinitSubfolders()
if HG.client_controller.IsFirstStart():
try:
for ( prefix, location ) in list( self._prefixes_to_locations.items() ):
dirs_to_test = set()
for subfolder in self._GetAllSubfolders():
HydrusPaths.MakeSureDirectoryExists( location )
dirs_to_test.add( subfolder.location )
dirs_to_test.add( subfolder.GetDirectory() )
subdir = os.path.join( location, prefix )
for dir_to_test in dirs_to_test:
HydrusPaths.MakeSureDirectoryExists( subdir )
try:
HydrusPaths.MakeSureDirectoryExists( dir_to_test )
except:
text = 'Attempting to create the database\'s client_files folder structure in {} failed!'.format( dir_to_test )
self._controller.SafeShowCriticalMessage( 'unable to create file structure', text )
raise
except:
text = 'Attempting to create the database\'s client_files folder structure in {} failed!'.format( location )
text = 'Attempting to create the database\'s client_files folder structure failed!'
self._controller.SafeShowCriticalMessage( 'unable to create file structure', text )
@ -878,22 +937,22 @@ class ClientFilesManager( object ):
self._ReinitMissingLocations()
if len( self._missing_locations ) > 0:
if len( self._missing_subfolders ) > 0:
self._AttemptToHealMissingLocations()
self._prefixes_to_locations = self._controller.Read( 'client_files_locations' )
self._ReinitSubfolders()
self._ReinitMissingLocations()
if len( self._missing_locations ) > 0:
if len( self._missing_subfolders ) > 0:
self._bad_error_occurred = True
#
missing_dict = HydrusData.BuildKeyToListDict( self._missing_locations )
missing_dict = HydrusData.BuildKeyToListDict( [ ( subfolder.location, subfolder.prefix ) for subfolder in self._missing_subfolders ] )
missing_locations = sorted( missing_dict.keys() )
@ -913,7 +972,7 @@ class ClientFilesManager( object ):
#
if len( self._missing_locations ) > 4:
if len( self._missing_subfolders ) > 4:
text = 'When initialising the client files manager, some file locations did not exist! They have all been written to the log!'
text += os.linesep * 2
@ -938,22 +997,35 @@ class ClientFilesManager( object ):
def _ReinitSubfolders( self ):
subfolders = self._controller.Read( 'client_files_subfolders' )
self._prefixes_to_client_files_subfolders = collections.defaultdict( list )
for subfolder in subfolders:
self._prefixes_to_client_files_subfolders[ subfolder.prefix ].append( subfolder )
self._smallest_prefix = min( ( len( prefix ) for prefix in self._prefixes_to_client_files_subfolders.keys() ) ) - 1
self._largest_prefix = max( ( len( prefix ) for prefix in self._prefixes_to_client_files_subfolders.keys() ) ) - 1
def _ReinitMissingLocations( self ):
self._missing_locations = set()
self._missing_subfolders = set()
for ( prefix, location ) in self._prefixes_to_locations.items():
for ( prefix, subfolders ) in self._prefixes_to_client_files_subfolders.items():
if os.path.exists( location ):
for subfolder in subfolders:
if os.path.exists( os.path.join( location, prefix ) ):
if not os.path.exists( subfolder.GetDirectory() ):
continue
self._missing_subfolders.add( subfolder )
self._missing_locations.add( ( location, prefix ) )
def _WaitOnWakeup( self ):
@ -977,7 +1049,7 @@ class ClientFilesManager( object ):
client_files_default = os.path.join( db_dir, 'client_files' )
all_locations = set( self._prefixes_to_locations.values() )
all_locations = self._GetCurrentSubfolderLocations()
return False not in ( location.startswith( client_files_default ) for location in all_locations )
@ -1329,17 +1401,7 @@ class ClientFilesManager( object ):
with self._rwlock.read:
locations = set()
for ( prefix, location ) in self._prefixes_to_locations.items():
if prefix.startswith( 'f' ):
locations.add( location )
return locations
return self._GetCurrentSubfolderLocations( only_files = True )
@ -1383,9 +1445,9 @@ class ClientFilesManager( object ):
def GetMissing( self ):
def GetMissingSubfolders( self ):
return self._missing_locations
return self._missing_subfolders
def GetThumbnailPath( self, media ):
@ -1465,33 +1527,6 @@ class ClientFilesManager( object ):
time.sleep( 0.01 )
recover_tuple = self._GetRecoverTuple()
while recover_tuple is not None:
if job_key.IsCancelled():
break
( prefix, recoverable_location, correct_location ) = recover_tuple
text = 'Recovering \'' + prefix + '\' from ' + recoverable_location + ' to ' + correct_location
HydrusData.Print( text )
job_key.SetStatusText( text )
recoverable_path = os.path.join( recoverable_location, prefix )
correct_path = os.path.join( correct_location, prefix )
HydrusPaths.MergeTree( recoverable_path, correct_path )
recover_tuple = self._GetRecoverTuple()
time.sleep( 0.01 )
finally:
@ -1735,6 +1770,8 @@ class FilesMaintenanceManager( object ):
file_is_missing = False
file_is_invalid = False
path = ''
try:
path = self._controller.client_files_manager.GetFilePath( hash, mime )
@ -2331,8 +2368,6 @@ class FilesMaintenanceManager( object ):
last_time_jobs_were_cleared = HydrusTime.GetNow()
num_to_do = sum( len( job_types ) for ( media_result, job_types ) in media_results_to_job_types.items() )
for ( media_result, job_types ) in media_results_to_job_types.items():
big_pauser.Pause()

79
hydrus/client/ClientFilesPhysical.py Normal file
View File

@ -0,0 +1,79 @@
import os
import typing
from hydrus.core import HydrusData
from hydrus.core import HydrusExceptions
def CheckFullPrefixCoverage( merge_target, prefixes ):
missing_prefixes = GetMissingPrefixes( merge_target, prefixes )
if len( missing_prefixes ) > 0:
list_of_problems = ', '.join( missing_prefixes )
raise HydrusExceptions.DataMissing( 'Missing storage spaces! They are, or are sub-divisions of:' + list_of_problems )
def GetMissingPrefixes( merge_target: str, prefixes: typing.Collection[ str ], min_prefix_length_allowed = 3, prefixes_are_filtered: bool = False ):
# given a merge target of 'tf'
# do these prefixes, let's say { tf0, tf1, tf2, tf3, tf4, tf5, tf6, tf7, tf8, tf9, tfa, tfb, tfc, tfd, tfe, tff }, add up to 'tf'?
hex_chars = '0123456789abcdef'
if prefixes_are_filtered:
matching_prefixes = prefixes
else:
matching_prefixes = { prefix for prefix in prefixes if prefix.startswith( merge_target ) }
missing_prefixes = []
for char in hex_chars:
expected_prefix = merge_target + char
if expected_prefix in matching_prefixes:
# we are good
pass
else:
matching_prefixes_for_this_char = { prefix for prefix in prefixes if prefix.startswith( expected_prefix ) }
if len( matching_prefixes_for_this_char ) > 0 or len( expected_prefix ) < min_prefix_length_allowed:
missing_for_this_char = GetMissingPrefixes( expected_prefix, matching_prefixes_for_this_char, prefixes_are_filtered = True )
missing_prefixes.extend( missing_for_this_char )
else:
missing_prefixes.append( expected_prefix )
return missing_prefixes
class FilesStorageSubfolder( object ):
def __init__( self, prefix: str, location: str, purge: bool ):
self.prefix = prefix
self.location = location
self.purge = purge
def GetDirectory( self ):
return os.path.join( self.location, self.prefix )

3
hydrus/client/ClientOptions.py
View File

@ -292,6 +292,9 @@ class ClientOptions( HydrusSerialisable.SerialisableBase ):
self._dictionary[ 'booleans' ][ 'fade_sibling_connector' ] = True
self._dictionary[ 'booleans' ][ 'use_custom_sibling_connector_colour' ] = False
self._dictionary[ 'booleans' ][ 'hide_uninteresting_local_import_time' ] = True
self._dictionary[ 'booleans' ][ 'hide_uninteresting_modified_time' ] = True
from hydrus.client.gui.canvas import ClientGUIMPV
self._dictionary[ 'booleans' ][ 'mpv_available_at_start' ] = ClientGUIMPV.MPV_IS_AVAILABLE

122
hydrus/client/ClientStrings.py
View File

@ -71,6 +71,7 @@ class StringProcessingStep( HydrusSerialisable.SerialisableBase ):
raise NotImplementedError()
class StringConverter( StringProcessingStep ):
SERIALISABLE_TYPE = HydrusSerialisable.SERIALISABLE_TYPE_STRING_CONVERTER
@ -453,8 +454,112 @@ class StringConverter( StringProcessingStep ):
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_STRING_CONVERTER ] = StringConverter
class StringJoiner( StringProcessingStep ):
SERIALISABLE_TYPE = HydrusSerialisable.SERIALISABLE_TYPE_STRING_JOINER
SERIALISABLE_NAME = 'String Concatenator'
SERIALISABLE_VERSION = 1
def __init__( self, joiner: str = '', join_tuple_size = None ):
StringProcessingStep.__init__( self )
self._joiner = joiner
self._join_tuple_size = join_tuple_size
def _GetSerialisableInfo( self ):
return ( self._joiner, self._join_tuple_size )
def _InitialiseFromSerialisableInfo( self, serialisable_info ):
( self._joiner, self._join_tuple_size ) = serialisable_info
def GetJoiner( self ):
return self._joiner
def GetJoinTupleSize( self ):
return self._join_tuple_size
def MakesChanges( self ) -> bool:
return True
def Join( self, texts: typing.Collection[ str ] ) -> typing.List[ str ]:
for text in texts:
if isinstance( text, bytes ):
raise HydrusExceptions.StringJoinerException( 'Got a bytes value in a string joiner!' )
try:
joined_texts = []
if self._join_tuple_size is None:
joined_texts.append( self._joiner.join( texts ) )
else:
for chunk_of_texts in HydrusData.SplitIteratorIntoChunks( texts, self._join_tuple_size ):
if len( chunk_of_texts ) == self._join_tuple_size:
joined_texts.append( self._joiner.join( chunk_of_texts ) )
except Exception as e:
raise HydrusExceptions.StringJoinerException( 'Problem when joining text: {}'.format( e ) )
return joined_texts
def ToString( self, simple = False, with_type = False ) -> str:
if simple:
return 'joiner'
if self._join_tuple_size is None:
result = f'joining all strings using "{self._joiner}"'
else:
result = f'joining every {self._join_tuple_size} strings using "{self._joiner}"'
if with_type:
result = 'JOIN: {}'.format( result )
return result
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_STRING_JOINER ] = StringJoiner
STRING_MATCH_FIXED = 0
STRING_MATCH_FLEXIBLE = 1
STRING_MATCH_REGEX = 2
@ -1090,6 +1195,7 @@ class StringSplitter( StringProcessingStep ):
return result
HydrusSerialisable.SERIALISABLE_TYPES_TO_OBJECT_TYPES[ HydrusSerialisable.SERIALISABLE_TYPE_STRING_SPLITTER ] = StringSplitter
class StringTagFilter( StringProcessingStep ):
@ -1325,6 +1431,17 @@ class StringProcessor( StringProcessingStep ):
next_strings = current_strings
elif isinstance( processing_step, StringJoiner ):
try:
next_strings = processing_step.Join( current_strings )
except:
next_strings = current_strings
else:
next_strings = []
@ -1410,6 +1527,11 @@ class StringProcessor( StringProcessingStep ):
components.append( 'conversion' )
if True in ( isinstance( ps, StringJoiner ) for ps in self._processing_steps ):
components.append( 'joining' )
if True in ( isinstance( ps, StringMatch ) for ps in self._processing_steps ):
components.append( 'filtering' )

25
hydrus/client/db/ClientDB.py
View File

@ -6288,7 +6288,7 @@ class DB( HydrusDB.HydrusDB ):
if action == 'autocomplete_predicates': result = self.modules_tag_search.GetAutocompletePredicates( *args, **kwargs )
elif action == 'boned_stats': result = self._GetBonedStats( *args, **kwargs )
elif action == 'client_files_locations': result = self.modules_files_physical_storage.GetClientFilesLocations( *args, **kwargs )
elif action == 'client_files_subfolders': result = self.modules_files_physical_storage.GetClientFilesSubfolders( *args, **kwargs )
elif action == 'deferred_delete_data': result = self.modules_db_maintenance.GetDeferredDeleteTableData( *args, **kwargs )
elif action == 'deferred_physical_delete': result = self.modules_files_storage.GetDeferredPhysicalDelete( *args, **kwargs )
elif action == 'duplicate_pairs_for_filtering': result = self._DuplicatesGetPotentialDuplicatePairsForFiltering( *args, **kwargs )
@ -9454,6 +9454,29 @@ class DB( HydrusDB.HydrusDB ):
if version == 540:
result = self._Execute( 'SELECT 1 FROM main.sqlite_master WHERE name = ?;', ( 'client_files_subfolders', ) ).fetchone()
if result is None:
self._Execute( 'CREATE TABLE IF NOT EXISTS main.client_files_subfolders ( prefix TEXT, location TEXT, purge INTEGER_BOOLEAN, PRIMARY KEY ( prefix, location ) );' )
self._Execute( 'INSERT INTO client_files_subfolders SELECT prefix, location, ? FROM client_files_locations;', ( False, ) )
self._Execute( 'DROP TABLE client_files_locations;' )
result = self._Execute( 'SELECT * FROM ideal_client_files_locations;' ).fetchone()
if len( result ) == 2:
self._Execute( 'ALTER TABLE ideal_client_files_locations ADD COLUMN max_num_bytes INTEGER;' )
self._Execute( 'UPDATE ideal_client_files_locations SET max_num_bytes = ?;', ( None, ) )
self._controller.frame_splash_status.SetTitleText( 'updated db to v{}'.format( HydrusData.ToHumanInt( version + 1 ) ) )
self._Execute( 'UPDATE version SET version = ?;', ( version + 1, ) )

72
hydrus/client/db/ClientDBFilesPhysicalStorage.py
View File

@ -4,8 +4,8 @@ import typing
from hydrus.core import HydrusData
from hydrus.core import HydrusPaths
from hydrus.core import HydrusTime
from hydrus.client import ClientFilesPhysical
from hydrus.client.db import ClientDBModule
class ClientDBFilesPhysicalStorage( ClientDBModule.ClientDBModule ):
@ -24,19 +24,24 @@ class ClientDBFilesPhysicalStorage( ClientDBModule.ClientDBModule ):
def _GetInitialTableGenerationDict( self ) -> dict:
return {
'main.client_files_locations' : ( 'CREATE TABLE IF NOT EXISTS {} ( prefix TEXT, location TEXT );', 400 ),
'main.ideal_client_files_locations' : ( 'CREATE TABLE IF NOT EXISTS {} ( location TEXT, weight INTEGER );', 400 ),
'main.client_files_subfolders' : ( 'CREATE TABLE IF NOT EXISTS {} ( prefix TEXT, location TEXT, purge INTEGER_BOOLEAN, PRIMARY KEY ( prefix, location ) );', 541 ),
'main.ideal_client_files_locations' : ( 'CREATE TABLE IF NOT EXISTS {} ( location TEXT, weight INTEGER, max_num_bytes INTEGER );', 400 ),
'main.ideal_thumbnail_override_location' : ( 'CREATE TABLE IF NOT EXISTS {} ( location TEXT );', 400 )
}
def GetClientFilesLocations( self ):
def GetClientFilesSubfolders( self ):
result = { prefix : HydrusPaths.ConvertPortablePathToAbsPath( portable_location ) for ( prefix, portable_location ) in self._Execute( 'SELECT prefix, location FROM client_files_locations;' ) }
subfolders = { ClientFilesPhysical.FilesStorageSubfolder( prefix, HydrusPaths.ConvertPortablePathToAbsPath( portable_location ), purge ) for ( prefix, portable_location, purge ) in self._Execute( 'SELECT prefix, location, purge FROM client_files_subfolders;' ) }
if len( result ) < 512:
all_prefixes = { subfolder.prefix for subfolder in subfolders }
missing_prefixes_f = ClientFilesPhysical.GetMissingPrefixes( 'f', all_prefixes )
missing_prefixes_t = ClientFilesPhysical.GetMissingPrefixes( 't', all_prefixes )
if len( missing_prefixes_f ) > 0 or len( missing_prefixes_t ) > 0:
message = 'When fetching the directories where your files are stored, the database discovered some entries were missing!'
message = 'When fetching the directories where your files are stored, the database discovered that some entries were missing! If you did not fiddle with the database yourself, this probably happened due to database corruption.'
message += os.linesep * 2
message += 'Default values will now be inserted. If you have previously migrated your files or thumbnails, and assuming this is occuring on boot, you will next be presented with a dialog to remap them to the correct location.'
message += os.linesep * 2
@ -50,30 +55,27 @@ class ClientDBFilesPhysicalStorage( ClientDBModule.ClientDBModule ):
portable_path = HydrusPaths.ConvertAbsPathToPortablePath( client_files_default )
for hex_prefix in HydrusData.IterateHexPrefixes():
for missing_prefix in missing_prefixes_f:
if 'f' + hex_prefix not in result:
self._Execute( 'INSERT OR IGNORE INTO client_files_locations ( prefix, location ) VALUES ( ?, ? );', ( 'f' + hex_prefix, portable_path ) )
if 't' + hex_prefix not in result:
self._Execute( 'INSERT OR IGNORE INTO client_files_locations ( prefix, location ) VALUES ( ?, ? );', ( 't' + hex_prefix, portable_path ) )
self._Execute( 'INSERT OR IGNORE INTO client_files_subfolders ( prefix, location, purge ) VALUES ( ?, ?, ? );', ( missing_prefix, portable_path, False ) )
result = { prefix : HydrusPaths.ConvertPortablePathToAbsPath( portable_location ) for ( prefix, portable_location ) in self._Execute( 'SELECT prefix, location FROM client_files_locations;' ) }
for missing_prefix in missing_prefixes_t:
self._Execute( 'INSERT OR IGNORE INTO client_files_subfolders ( prefix, location, purge ) VALUES ( ?, ?, ? );', ( missing_prefix, portable_path, False ) )
subfolders = { ClientFilesPhysical.FilesStorageSubfolder( prefix, HydrusPaths.ConvertPortablePathToAbsPath( portable_location ), purge ) for ( prefix, portable_location, purge ) in self._Execute( 'SELECT prefix, location, purge FROM client_files_subfolders;' ) }
return result
return subfolders
def GetIdealClientFilesLocations( self ):
abs_locations_to_ideal_weights = {}
for ( portable_location, weight ) in self._Execute( 'SELECT location, weight FROM ideal_client_files_locations;' ):
for ( portable_location, weight, max_num_bytes ) in self._Execute( 'SELECT location, weight, max_num_bytes FROM ideal_client_files_locations;' ):
abs_location = HydrusPaths.ConvertPortablePathToAbsPath( portable_location )
@ -113,15 +115,21 @@ class ClientDBFilesPhysicalStorage( ClientDBModule.ClientDBModule ):
for hex_prefix in HydrusData.IterateHexPrefixes():
self._Execute( 'INSERT INTO client_files_locations ( prefix, location ) VALUES ( ?, ? );', ( 'f' + hex_prefix, portable_path ) )
self._Execute( 'INSERT INTO client_files_locations ( prefix, location ) VALUES ( ?, ? );', ( 't' + hex_prefix, portable_path ) )
self._Execute( 'INSERT INTO client_files_subfolders ( prefix, location, purge ) VALUES ( ?, ?, ? );', ( 'f' + hex_prefix, portable_path, False ) )
self._Execute( 'INSERT INTO client_files_subfolders ( prefix, location, purge ) VALUES ( ?, ?, ? );', ( 't' + hex_prefix, portable_path, False ) )
self._Execute( 'INSERT INTO ideal_client_files_locations ( location, weight ) VALUES ( ?, ? );', ( portable_path, 1 ) )
self._Execute( 'INSERT INTO ideal_client_files_locations ( location, weight, max_num_bytes ) VALUES ( ?, ?, ? );', ( portable_path, 1, None ) )
def RelocateClientFiles( self, prefix, abs_source, abs_dest ):
# TODO: so this guy is going to be replaces with slow migration, which will be something like:
# Add a new valid subfolder
# Set dupe subfolder to purge = True
# Ask database for valid purge paths
# once a source is fully purged, remove the now purged subfolder
if not os.path.exists( abs_source ):
raise Exception( 'Was commanded to move prefix "{}" from "{}" to "{}", but that source does not exist!'.format( prefix, abs_source, abs_dest ) )
@ -147,13 +155,15 @@ class ClientDBFilesPhysicalStorage( ClientDBModule.ClientDBModule ):
portable_source = HydrusPaths.ConvertAbsPathToPortablePath( abs_source )
portable_dest = HydrusPaths.ConvertAbsPathToPortablePath( abs_dest )
self._Execute( 'UPDATE client_files_locations SET location = ? WHERE prefix = ?;', ( portable_dest, prefix ) )
self._Execute( 'DELETE FROM client_files_subfolders WHERE prefix = ? AND location = ?;', ( prefix, portable_source ) )
self._Execute( 'INSERT OR IGNORE INTO client_files_subfolders ( prefix, location, purge ) VALUES ( ?, ?, ? );', ( prefix, portable_dest, False ) )
if not os.path.samefile( abs_source, abs_dest ):
if os.path.exists( full_source ):
if os.path.exists( full_source ) and len( os.listdir( full_source ) ) == 0:
try: HydrusPaths.RecyclePath( full_source )
except: pass
@ -163,15 +173,19 @@ class ClientDBFilesPhysicalStorage( ClientDBModule.ClientDBModule ):
def RepairClientFiles( self, correct_rows ):
for ( prefix, abs_correct_location ) in correct_rows:
# TODO: as we move to multiple valid locations, this should probably become something else, or the things that feed it should have more sophisticated discovery of the correct
for ( prefix, abs_incorrect_location, abs_correct_location ) in correct_rows:
full_abs_correct_location = os.path.join( abs_correct_location, prefix )
HydrusPaths.MakeSureDirectoryExists( full_abs_correct_location )
portable_incorrect_location = HydrusPaths.ConvertAbsPathToPortablePath( abs_incorrect_location )
portable_correct_location = HydrusPaths.ConvertAbsPathToPortablePath( abs_correct_location )
self._Execute( 'UPDATE client_files_locations SET location = ? WHERE prefix = ?;', ( portable_correct_location, prefix ) )
self._Execute( 'DELETE FROM client_files_subfolders WHERE prefix = ? AND location = ?;', ( prefix, portable_incorrect_location ) )
self._Execute( 'INSERT OR IGNORE INTO client_files_subfolders ( prefix, location, purge ) VALUES ( ?, ?, ? );', ( prefix, portable_correct_location, False ) )
@ -184,11 +198,13 @@ class ClientDBFilesPhysicalStorage( ClientDBModule.ClientDBModule ):
self._Execute( 'DELETE FROM ideal_client_files_locations;' )
max_num_bytes = None
for ( abs_location, weight ) in abs_locations_to_ideal_weights.items():
portable_location = HydrusPaths.ConvertAbsPathToPortablePath( abs_location )
self._Execute( 'INSERT INTO ideal_client_files_locations ( location, weight ) VALUES ( ?, ? );', ( portable_location, weight ) )
self._Execute( 'INSERT INTO ideal_client_files_locations ( location, weight, max_num_bytes ) VALUES ( ?, ?, ? );', ( portable_location, weight, max_num_bytes ) )
self._Execute( 'DELETE FROM ideal_thumbnail_override_location;' )

8
hydrus/client/db/ClientDBMaster.py
View File

@ -103,6 +103,8 @@ class ClientDBMasterHashes( ClientDBModule.ClientDBModule ):
if result is None:
hash = bytes.fromhex( 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' ) + os.urandom( 16 )
if not pubbed_error:
HydrusData.ShowText( 'A file identifier was missing! This is a serious error that means your client database had an orphan file id! You have very likely encountered database corruption, perhaps recently, or perhaps years ago, please check the "help my db is broke.txt" document under install_dir/db folder as background reading. Additional info has been written to the log.' )
@ -112,10 +114,10 @@ class ClientDBMasterHashes( ClientDBModule.ClientDBModule ):
HydrusData.DebugPrint( 'Database master hash definition error: hash_id {} was missing! Replaced with hash {}.'.format( hash_id, hash.hex() ) )
hash = bytes.fromhex( 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' ) + os.urandom( 16 )
else:
( hash, ) = result
if not pubbed_error:
HydrusData.ShowText( 'A file identifier was missing! This is a serious error that means your client database had an orphan file id! Luckily, I was able to find a duplicate record in another location and fill in the missing record. You have, however, very likely encountered database corruption, perhaps recently, or perhaps years ago, please check the "help my db is broke.txt" document under install_dir/db folder as background reading. Additional info has been written to the log.' )
@ -125,8 +127,6 @@ class ClientDBMasterHashes( ClientDBModule.ClientDBModule ):
HydrusData.DebugPrint( 'Database master hash definition error: hash_id {} was missing! Recovered from local hash cache with hash {}.'.format( hash_id, hash.hex() ) )
( hash, ) = result
self._Execute( 'INSERT OR IGNORE INTO hashes ( hash_id, hash ) VALUES ( ?, ? );', ( hash_id, sqlite3.Binary( hash ) ) )

7
hydrus/client/db/ClientDBTagSearch.py
View File

@ -135,7 +135,12 @@ def WildcardHasFTS4SearchableCharacters( wildcard: str ):
for c in wildcard:
if c.isalnum() or ord( c ) >= 128 or c == '*':
if c == '*':
continue
if c.isalnum() or ord( c ) >= 128:
return True

21
hydrus/client/gui/ClientGUIAPI.py
View File

@ -89,6 +89,8 @@ class EditAPIPermissionsPanel( ClientGUIScrolledPanels.EditPanel ):
self._basic_permissions.sortItems()
self._check_all_permissions_button = ClientGUICommon.BetterButton( self, 'check all permissions', self._CheckAllPermissions )
search_tag_filter = api_permissions.GetSearchTagFilter()
message = 'The API will only permit searching for tags that pass through this filter.'
@ -118,6 +120,7 @@ class EditAPIPermissionsPanel( ClientGUIScrolledPanels.EditPanel ):
rows.append( ( 'access key: ', self._access_key ) )
rows.append( ( 'name: ', self._name ) )
rows.append( ( 'permissions: ', self._basic_permissions ) )
rows.append( ( '', self._check_all_permissions_button ) )
rows.append( ( 'tag search permissions: ', self._search_tag_filter ) )
gridbox = ClientGUICommon.WrapInGrid( self, rows )
@ -135,12 +138,30 @@ class EditAPIPermissionsPanel( ClientGUIScrolledPanels.EditPanel ):
self._basic_permissions.checkBoxListChanged.connect( self._UpdateEnabled )
def _CheckAllPermissions( self ):
for i in range( self._basic_permissions.count() ):
self._basic_permissions.Check( i )
def _UpdateEnabled( self ):
can_search = ClientAPI.CLIENT_API_PERMISSION_SEARCH_FILES in self._basic_permissions.GetValue()
self._search_tag_filter.setEnabled( can_search )
self._check_all_permissions_button.setEnabled( False )
for i in range( self._basic_permissions.count() ):
if not self._basic_permissions.IsChecked( i ):
self._check_all_permissions_button.setEnabled( True )
def _GetValue( self ):

193
hydrus/client/gui/ClientGUIScrolledPanelsManagement.py
View File

@ -2,6 +2,7 @@ import collections
import os
import random
import traceback
import typing
from qtpy import QtCore as QC
from qtpy import QtWidgets as QW
@ -20,6 +21,7 @@ from hydrus.core import HydrusTime
from hydrus.client import ClientApplicationCommand as CAC
from hydrus.client import ClientConstants as CC
from hydrus.client import ClientFilesPhysical
from hydrus.client.importing.options import FileImportOptions
from hydrus.client.gui import ClientGUIDialogs
from hydrus.client.gui import ClientGUIDialogsQuick
@ -2059,58 +2061,78 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
self._new_options = HG.client_controller.new_options
self._animation_start_position = ClientGUICommon.BetterSpinBox( self, min=0, max=100 )
#
self._disable_cv_for_gifs = QW.QCheckBox( self )
self._disable_cv_for_gifs.setToolTip( 'OpenCV is good at rendering gifs, but if you have problems with it and your graphics card, check this and the less reliable and slower PIL will be used instead. EDIT: OpenCV is much better these days--this is mostly not needed.' )
animations_panel = ClientGUICommon.StaticBox( self, 'animations' )
self._load_images_with_pil = QW.QCheckBox( self )
self._load_images_with_pil.setToolTip( 'OpenCV is much faster than PIL, but it is sometimes less reliable. Switch this on if you experience crashes or other unusual problems while importing or viewing certain images. EDIT: OpenCV is much better these days--this is mostly not needed.' )
self._animated_scanbar_height = ClientGUICommon.BetterSpinBox( animations_panel, min=1, max=255 )
self._animated_scanbar_hide_height = ClientGUICommon.NoneableSpinCtrl( animations_panel, none_phrase = 'no, hide it', min = 1, max = 255, unit = 'px' )
self._animated_scanbar_nub_width = ClientGUICommon.BetterSpinBox( animations_panel, min=1, max=63 )
self._use_system_ffmpeg = QW.QCheckBox( self )
self._use_system_ffmpeg.setToolTip( 'Check this to always default to the system ffmpeg in your path, rather than using the static ffmpeg in hydrus\'s bin directory. (requires restart)' )
self._animation_start_position = ClientGUICommon.BetterSpinBox( animations_panel, min=0, max=100 )
self._always_loop_animations = QW.QCheckBox( self )
self._always_loop_animations = QW.QCheckBox( animations_panel )
self._always_loop_animations.setToolTip( 'Some GIFS and APNGs have metadata specifying how many times they should be played, usually 1. Uncheck this to obey that number.' )
self._draw_transparency_checkerboard_media_canvas = QW.QCheckBox( self )
self._draw_transparency_checkerboard_media_canvas.setToolTip( 'If unchecked, will fill in with the normal background colour. Does not apply to MPV.' )
#
self._media_viewer_cursor_autohide_time_ms = ClientGUICommon.NoneableSpinCtrl( self, none_phrase = 'do not autohide', min = 100, max = 100000, unit = 'ms' )
system_panel = ClientGUICommon.StaticBox( self, 'system' )
self._anchor_and_hide_canvas_drags = QW.QCheckBox( self )
self._touchscreen_canvas_drags_unanchor = QW.QCheckBox( self )
self._mpv_conf_path = QP.FilePickerCtrl( system_panel, starting_directory = os.path.join( HC.STATIC_DIR, 'mpv-conf' ) )
self._use_system_ffmpeg = QW.QCheckBox( system_panel )
self._use_system_ffmpeg.setToolTip( 'Check this to always default to the system ffmpeg in your path, rather than using the static ffmpeg in hydrus\'s bin directory. (requires restart)' )
self._disable_cv_for_gifs = QW.QCheckBox( system_panel )
self._disable_cv_for_gifs.setToolTip( 'OpenCV is good at rendering gifs, but if you have problems with it and your graphics card, check this and the less reliable and slower PIL will be used instead. EDIT: OpenCV is much better these days--this is mostly not needed.' )
self._load_images_with_pil = QW.QCheckBox( system_panel )
self._load_images_with_pil.setToolTip( 'OpenCV is much faster than PIL, but it is sometimes less reliable. Switch this on if you experience crashes or other unusual problems while importing or viewing certain images. EDIT: OpenCV is much better these days--this is mostly not needed.' )
#
media_viewer_panel = ClientGUICommon.StaticBox( self, 'media viewer' )
self._media_viewer_cursor_autohide_time_ms = ClientGUICommon.NoneableSpinCtrl( media_viewer_panel, none_phrase = 'do not autohide', min = 100, max = 100000, unit = 'ms' )
self._slideshow_durations = QW.QLineEdit( media_viewer_panel )
self._slideshow_durations.setToolTip( 'This is a bit hacky, but whatever you have here, in comma-separated floats, will end up in the slideshow menu in the media viewer.' )
self._slideshow_durations.textChanged.connect( self.EventSlideshowDurationsChanged )
self._media_zooms = QW.QLineEdit( media_viewer_panel )
self._media_zooms.setToolTip( 'This is a bit hacky, but whatever you have here, in comma-separated floats, will be what the program steps through as you zoom a media up and down.' )
self._media_zooms.textChanged.connect( self.EventZoomsChanged )
from hydrus.client.gui.canvas import ClientGUICanvasMedia
self._media_viewer_zoom_center = ClientGUICommon.BetterChoice()
self._media_viewer_zoom_center = ClientGUICommon.BetterChoice( media_viewer_panel )
for zoom_centerpoint_type in ClientGUICanvasMedia.ZOOM_CENTERPOINT_TYPES:
self._media_viewer_zoom_center.addItem( ClientGUICanvasMedia.zoom_centerpoints_str_lookup[ zoom_centerpoint_type ], zoom_centerpoint_type )
tt = 'When you zoom in or out, there is a centerpoint about which the image zooms. This point \'stays still\' while the image expands or shrinks around it. Different centerpoints give different feels, especially if you drag images around a bit.'
tt = 'When you zoom in or out, there is a centerpoint about which the image zooms. This point \'stays still\' while the image expands or shrinks around it. Different centerpoints give different feels, especially if you drag images around a bit before zooming.'
self._media_viewer_zoom_center.setToolTip( tt )
self._slideshow_durations = QW.QLineEdit( self )
self._slideshow_durations.setToolTip( 'This is a bit hacky, but whatever you have here, in comma-separated floats, will end up in the slideshow menu in the media viewer.' )
self._slideshow_durations.textChanged.connect( self.EventSlideshowDurationsChanged )
self._draw_transparency_checkerboard_media_canvas = QW.QCheckBox( media_viewer_panel )
self._draw_transparency_checkerboard_media_canvas.setToolTip( 'If unchecked, will fill in with the normal background colour. Does not apply to MPV.' )
self._media_zooms = QW.QLineEdit( self )
self._media_zooms.setToolTip( 'This is a bit hacky, but whatever you have here, in comma-separated floats, will be what the program steps through as you zoom a media up and down.' )
self._media_zooms.textChanged.connect( self.EventZoomsChanged )
self._hide_uninteresting_local_import_time = QW.QCheckBox( media_viewer_panel )
self._hide_uninteresting_local_import_time.setToolTip( 'If the file was imported at a similar time to when it was added to its current services (i.e. the number of seconds since both events differs by less than 10%), hide the import time in the top of the media viewer.' )
self._mpv_conf_path = QP.FilePickerCtrl( self, starting_directory = os.path.join( HC.STATIC_DIR, 'mpv-conf' ) )
self._hide_uninteresting_modified_time = QW.QCheckBox( media_viewer_panel )
self._hide_uninteresting_modified_time.setToolTip( 'If the file has a modified time similar to its import time (i.e. the number of seconds since both events differs by less than 10%), hide the modified time in the top of the media viewer.' )
self._animated_scanbar_height = ClientGUICommon.BetterSpinBox( self, min=1, max=255 )
self._animated_scanbar_hide_height = ClientGUICommon.NoneableSpinCtrl( self, none_phrase = 'no, hide it', min = 1, max = 255, unit = 'px' )
self._animated_scanbar_nub_width = ClientGUICommon.BetterSpinBox( self, min=1, max=63 )
self._anchor_and_hide_canvas_drags = QW.QCheckBox( media_viewer_panel )
self._touchscreen_canvas_drags_unanchor = QW.QCheckBox( media_viewer_panel )
self._media_viewer_panel = ClientGUICommon.StaticBox( self, 'media viewer filetype handling' )
#
media_viewer_list_panel = ClientGUIListCtrl.BetterListCtrlPanel( self._media_viewer_panel )
filetype_handling_panel = ClientGUICommon.StaticBox( media_viewer_panel, 'media viewer filetype handling' )
media_viewer_list_panel = ClientGUIListCtrl.BetterListCtrlPanel( filetype_handling_panel )
self._media_viewer_options = ClientGUIListCtrl.BetterListCtrl( media_viewer_list_panel, CGLC.COLUMN_LIST_MEDIA_VIEWER_OPTIONS.ID, 20, data_to_tuples_func = self._GetListCtrlData, activation_callback = self.EditMediaViewerOptions, use_simple_delete = True )
@ -2123,6 +2145,8 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
#
self._animation_start_position.setValue( int( HC.options['animation_start_position'] * 100.0 ) )
self._hide_uninteresting_local_import_time.setChecked( self._new_options.GetBoolean( 'hide_uninteresting_local_import_time' ) )
self._hide_uninteresting_modified_time.setChecked( self._new_options.GetBoolean( 'hide_uninteresting_modified_time' ) )
self._disable_cv_for_gifs.setChecked( self._new_options.GetBoolean( 'disable_cv_for_gifs' ) )
self._load_images_with_pil.setChecked( self._new_options.GetBoolean( 'load_images_with_pil' ) )
self._use_system_ffmpeg.setChecked( self._new_options.GetBoolean( 'use_system_ffmpeg' ) )
@ -2162,32 +2186,61 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
vbox = QP.VBoxLayout()
#
rows = []
rows.append( ( 'Start animations this % in:', self._animation_start_position ) )
rows.append( ( 'Prefer system FFMPEG:', self._use_system_ffmpeg ) )
rows.append( ( 'Always Loop GIFs/APNGs:', self._always_loop_animations ) )
rows.append( ( 'Draw image transparency as checkerboard:', self._draw_transparency_checkerboard_media_canvas ) )
rows.append( ( 'Centerpoint for media zooming:', self._media_viewer_zoom_center ) )
rows.append( ( 'Time until mouse cursor autohides on media viewer:', self._media_viewer_cursor_autohide_time_ms ) )
rows.append( ( 'Slideshow durations:', self._slideshow_durations ) )
rows.append( ( 'Media zooms:', self._media_zooms ) )
rows.append( ( 'Set a new mpv.conf on dialog ok?:', self._mpv_conf_path ) )
rows.append( ( 'Centerpoint for media zooming:', self._media_viewer_zoom_center ) )
rows.append( ( 'Draw image transparency as checkerboard:', self._draw_transparency_checkerboard_media_canvas ) )
rows.append( ( 'Hide uninteresting import times:', self._hide_uninteresting_local_import_time ) )
rows.append( ( 'Hide uninteresting modified times:', self._hide_uninteresting_modified_time ) )
rows.append( ( 'RECOMMEND WINDOWS ONLY: Hide and anchor mouse cursor on media viewer drags:', self._anchor_and_hide_canvas_drags ) )
rows.append( ( 'RECOMMEND WINDOWS ONLY: If set to hide and anchor, undo on apparent touchscreen drag:', self._touchscreen_canvas_drags_unanchor ) )
gridbox = ClientGUICommon.WrapInGrid( media_viewer_panel, rows )
filetype_handling_panel.Add( media_viewer_list_panel, CC.FLAGS_EXPAND_BOTH_WAYS )
media_viewer_panel.Add( gridbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
media_viewer_panel.Add( filetype_handling_panel, CC.FLAGS_EXPAND_BOTH_WAYS )
QP.AddToLayout( vbox, media_viewer_panel, CC.FLAGS_EXPAND_BOTH_WAYS )
#
rows = []
rows.append( ( 'Animation scanbar height:', self._animated_scanbar_height ) )
rows.append( ( 'Animation scanbar height when mouse away:', self._animated_scanbar_hide_height ) )
rows.append( ( 'Animation scanbar nub width:', self._animated_scanbar_nub_width ) )
rows.append( ( 'Time until mouse cursor autohides on media viewer:', self._media_viewer_cursor_autohide_time_ms ) )
rows.append( ( 'RECOMMEND WINDOWS ONLY: Hide and anchor mouse cursor on media viewer drags:', self._anchor_and_hide_canvas_drags ) )
rows.append( ( 'RECOMMEND WINDOWS ONLY: If set to hide and anchor, undo on apparent touchscreen drag:', self._touchscreen_canvas_drags_unanchor ) )
rows.append( ( 'BUGFIX: Load images with PIL (slower):', self._load_images_with_pil ) )
rows.append( ( 'Start animations this % in:', self._animation_start_position ) )
rows.append( ( 'Always Loop GIFs/APNGs:', self._always_loop_animations ) )
gridbox = ClientGUICommon.WrapInGrid( animations_panel, rows )
animations_panel.Add( gridbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
QP.AddToLayout( vbox, animations_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
#
rows = []
rows.append( ( 'Set a new mpv.conf on dialog ok?:', self._mpv_conf_path ) )
rows.append( ( 'Prefer system FFMPEG:', self._use_system_ffmpeg ) )
rows.append( ( 'BUGFIX: Load gifs with PIL instead of OpenCV (slower, bad transparency):', self._disable_cv_for_gifs ) )
rows.append( ( 'BUGFIX: Load images with PIL (slower):', self._load_images_with_pil ) )
gridbox = ClientGUICommon.WrapInGrid( self, rows )
gridbox = ClientGUICommon.WrapInGrid( system_panel, rows )
QP.AddToLayout( vbox, gridbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
system_panel.Add( gridbox, CC.FLAGS_EXPAND_SIZER_PERPENDICULAR )
self._media_viewer_panel.Add( media_viewer_list_panel, CC.FLAGS_EXPAND_BOTH_WAYS )
QP.AddToLayout( vbox, system_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( vbox, self._media_viewer_panel, CC.FLAGS_EXPAND_BOTH_WAYS )
#
self.setLayout( vbox )
@ -2411,6 +2464,8 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
HC.options[ 'animation_start_position' ] = self._animation_start_position.value() / 100.0
self._new_options.SetBoolean( 'hide_uninteresting_local_import_time', self._hide_uninteresting_local_import_time.isChecked() )
self._new_options.SetBoolean( 'hide_uninteresting_modified_time', self._hide_uninteresting_modified_time.isChecked() )
self._new_options.SetBoolean( 'disable_cv_for_gifs', self._disable_cv_for_gifs.isChecked() )
self._new_options.SetBoolean( 'load_images_with_pil', self._load_images_with_pil.isChecked() )
self._new_options.SetBoolean( 'use_system_ffmpeg', self._use_system_ffmpeg.isChecked() )
@ -3398,6 +3453,8 @@ class ManageOptionsPanel( ClientGUIScrolledPanels.ManagePanel ):
#
text = 'The current styles are what your Qt has available, the stylesheets are what .css and .qss files are currently in install_dir/static/qss.'
text += '\n' * 2
text += 'Note that there are several colours not handled by this yet. Check out the "colours" page of this options to change them.'
st = ClientGUICommon.BetterStaticText( self, label = text )
@ -4942,25 +4999,17 @@ class ManageURLsPanel( CAC.ApplicationCommandProcessorMixin, ClientGUIScrolledPa
class RepairFileSystemPanel( ClientGUIScrolledPanels.ManagePanel ):
def __init__( self, parent, missing_locations ):
def __init__( self, parent, missing_subfolders: typing.Collection[ ClientFilesPhysical.FilesStorageSubfolder ] ):
ClientGUIScrolledPanels.ManagePanel.__init__( self, parent )
# TODO: This needs another pass as we move to multiple locations and other tech
# if someone has f10 and we are expecting 16 lots of f10x, or vice versa, (e.g. on an out of sync db recovery, not uncommon) we'll need to handle that
self._only_thumbs = True
self._incorrect_locations = {}
self._correct_locations = {}
for ( incorrect_location, prefix ) in missing_locations:
self._incorrect_locations[ prefix ] = incorrect_location
if prefix.startswith( 'f' ):
self._only_thumbs = False
text = 'This dialog has launched because some expected file storage directories were not found. This is a serious error. You have two options:'
text += os.linesep * 2
text += '1) If you know what these should be (e.g. you recently remapped their external drive to another location), update the paths here manually. For most users, this will be clicking _add a possibly correct location_ and then select the new folder where the subdirectories all went. You can repeat this if your folders are missing in multiple locations. Check everything reports _ok!_'
@ -4989,7 +5038,7 @@ class RepairFileSystemPanel( ClientGUIScrolledPanels.ManagePanel ):
#
self._locations.AddDatas( [ prefix for ( incorrect_location, prefix ) in missing_locations ] )
self._locations.AddDatas( missing_subfolders )
self._locations.Sort()
@ -5013,13 +5062,15 @@ class RepairFileSystemPanel( ClientGUIScrolledPanels.ManagePanel ):
path = dlg.GetPath()
for prefix in self._locations.GetData():
for subfolder in self._locations.GetData():
prefix = subfolder.prefix
ok = os.path.exists( os.path.join( path, prefix ) )
if ok:
self._correct_locations[ prefix ] = ( path, ok )
self._correct_locations[ subfolder ] = ( path, ok )
@ -5028,13 +5079,14 @@ class RepairFileSystemPanel( ClientGUIScrolledPanels.ManagePanel ):
def _ConvertPrefixToListCtrlTuples( self, prefix ):
def _ConvertPrefixToListCtrlTuples( self, subfolder ):
incorrect_location = self._incorrect_locations[ prefix ]
prefix = subfolder.prefix
incorrect_location = subfolder.location
if prefix in self._correct_locations:
if subfolder in self._correct_locations:
( correct_location, ok ) = self._correct_locations[ prefix ]
( correct_location, ok ) = self._correct_locations[ subfolder ]
if ok:
@ -5068,11 +5120,12 @@ class RepairFileSystemPanel( ClientGUIScrolledPanels.ManagePanel ):
thumb_problems = False
for prefix in self._locations.GetData():
for subfolder in self._locations.GetData():
incorrect_location = self._incorrect_locations[ prefix ]
prefix = subfolder.prefix
incorrect_location = subfolder.location
if prefix not in self._correct_locations:
if subfolder not in self._correct_locations:
if prefix.startswith( 'f' ):
@ -5087,7 +5140,7 @@ class RepairFileSystemPanel( ClientGUIScrolledPanels.ManagePanel ):
else:
( correct_location, ok ) = self._correct_locations[ prefix ]
( correct_location, ok ) = self._correct_locations[ subfolder ]
if not ok:
@ -5102,7 +5155,7 @@ class RepairFileSystemPanel( ClientGUIScrolledPanels.ManagePanel ):
correct_rows.append( ( prefix, correct_location ) )
correct_rows.append( ( prefix, incorrect_location, correct_location ) )
return ( correct_rows, thumb_problems )
@ -5110,9 +5163,9 @@ class RepairFileSystemPanel( ClientGUIScrolledPanels.ManagePanel ):
def _SetLocations( self ):
prefixes = self._locations.GetData( only_selected = True )
subfolders = self._locations.GetData( only_selected = True )
if len( prefixes ) > 0:
if len( subfolders ) > 0:
with QP.DirDialog( self, 'Select correct location.' ) as dlg:
@ -5120,11 +5173,13 @@ class RepairFileSystemPanel( ClientGUIScrolledPanels.ManagePanel ):
path = dlg.GetPath()
for prefix in prefixes:
for subfolder in subfolders:
prefix = subfolder.prefix
ok = os.path.exists( os.path.join( path, prefix ) )
self._correct_locations[ prefix ] = ( path, ok )
self._correct_locations[ subfolder ] = ( path, ok )
self._locations.UpdateDatas()

196
hydrus/client/gui/ClientGUIScrolledPanelsReview.py
View File

@ -132,7 +132,7 @@ class MigrateDatabasePanel( ClientGUIScrolledPanels.ReviewPanel ):
ClientGUIScrolledPanels.ReviewPanel.__init__( self, parent )
self._prefixes_to_locations = HG.client_controller.Read( 'client_files_locations' )
self._client_files_subfolders = HG.client_controller.Read( 'client_files_subfolders' )
( self._locations_to_ideal_weights, self._ideal_thumbnails_location_override ) = self._controller.Read( 'ideal_client_files_locations' )
@ -189,12 +189,6 @@ class MigrateDatabasePanel( ClientGUIScrolledPanels.ReviewPanel ):
#
migration_panel = ClientGUICommon.StaticBox( self, 'migrate database files (and portable media file locations)' )
self._migrate_db_button = ClientGUICommon.BetterButton( migration_panel, 'move entire database and all portable paths', self._MigrateDatabase )
#
info_panel.Add( self._current_install_path_st, CC.FLAGS_EXPAND_PERPENDICULAR )
info_panel.Add( self._current_db_path_st, CC.FLAGS_EXPAND_PERPENDICULAR )
info_panel.Add( self._current_media_paths_st, CC.FLAGS_EXPAND_PERPENDICULAR )
@ -217,10 +211,6 @@ class MigrateDatabasePanel( ClientGUIScrolledPanels.ReviewPanel ):
#
migration_panel.Add( self._migrate_db_button, CC.FLAGS_ON_RIGHT )
#
vbox = QP.VBoxLayout()
message = 'THIS IS ADVANCED. DO NOT CHANGE ANYTHING HERE UNLESS YOU KNOW WHAT IT DOES!'
@ -233,7 +223,6 @@ class MigrateDatabasePanel( ClientGUIScrolledPanels.ReviewPanel ):
QP.AddToLayout( vbox, help_hbox, CC.FLAGS_ON_RIGHT )
QP.AddToLayout( vbox, info_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( vbox, file_locations_panel, CC.FLAGS_EXPAND_BOTH_WAYS )
QP.AddToLayout( vbox, migration_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
self.widget().setLayout( vbox )
@ -577,7 +566,10 @@ class MigrateDatabasePanel( ClientGUIScrolledPanels.ReviewPanel ):
locations_to_file_weights = collections.Counter()
locations_to_thumb_weights = collections.Counter()
for ( prefix, location ) in list(self._prefixes_to_locations.items()):
for client_files_subfolder in self._client_files_subfolders:
prefix = client_files_subfolder.prefix
location = client_files_subfolder.location
if prefix.startswith( 'f' ):
@ -638,92 +630,6 @@ class MigrateDatabasePanel( ClientGUIScrolledPanels.ReviewPanel ):
self._AdjustWeight( 1 )
def _MigrateDatabase( self ):
message = 'This operation will move your database files and any \'portable\' paths. It is a big job that will require a client shutdown and need you to create a new shortcut before you can launch it again.'
message += os.linesep * 2
message += 'If you have not read the database migration help or otherwise do not know what is going on here, turn back now!'
result = ClientGUIDialogsQuick.GetYesNo( self, message, yes_label = 'do it', no_label = 'forget it' )
if result == QW.QDialog.Accepted:
source = self._controller.GetDBDir()
with QP.DirDialog( self, message = 'Choose new database location.' ) as dlg:
dlg.setDirectory( source )
if dlg.exec() == QW.QDialog.Accepted:
dest = dlg.GetPath()
if source == dest:
QW.QMessageBox.warning( self, 'Warning', 'That is the same location!' )
return
if len( os.listdir( dest ) ) > 0:
message = '"{}" is not empty! Please select an empty destination--if your situation is more complicated, please do this move manually! Feel free to ask hydrus dev for help.'.format( dest )
result = ClientGUIDialogsQuick.GetYesNo( self, message )
if result != QW.QDialog.Accepted:
return
message = 'Here is the client\'s best guess at your new launch command. Make sure it looks correct and copy it to your clipboard. Update your program shortcut when the transfer is complete.'
message += os.linesep * 2
message += 'Hit ok to close the client and start the transfer, cancel to back out.'
me = sys.argv[0]
shortcut = '"' + me + '" -d="' + dest + '"'
with ClientGUIDialogs.DialogTextEntry( self, message, default = shortcut ) as dlg_3:
if dlg_3.exec() == QW.QDialog.Accepted:
# The below comment is from before the Qt port and probably obsolote, leaving it for explanation.
#
# careful with this stuff!
# the app's mainloop didn't want to exit for me, for a while, because this dialog didn't have time to exit before the thread's dialog laid a new event loop on top
# the confused event loops lead to problems at a C++ level in ShowModal not being able to do the Destroy because parent stuff had already died
# this works, so leave it alone if you can
QP.CallAfter( self.parentWidget().close )
portable_locations = []
for location in set( self._prefixes_to_locations.values() ):
if not os.path.exists( location ):
continue
portable_location = HydrusPaths.ConvertAbsPathToPortablePath( location )
portable = not os.path.isabs( portable_location )
if portable:
portable_locations.append( portable_location )
HG.client_controller.CallToThreadLongRunning( THREADMigrateDatabase, self._controller, source, portable_locations, dest )
def _Rebalance( self ):
for location in self._GetListCtrlLocations():
@ -882,7 +788,7 @@ class MigrateDatabasePanel( ClientGUIScrolledPanels.ReviewPanel ):
def _Update( self ):
self._prefixes_to_locations = HG.client_controller.Read( 'client_files_locations' )
self._client_files_subfolders = HG.client_controller.Read( 'client_files_subfolders' )
( self._locations_to_ideal_weights, self._ideal_thumbnails_location_override ) = self._controller.Read( 'ideal_client_files_locations' )
@ -941,96 +847,6 @@ class MigrateDatabasePanel( ClientGUIScrolledPanels.ReviewPanel ):
def THREADMigrateDatabase( controller, source, portable_locations, dest ):
time.sleep( 2 ) # important to have this, so the migrate dialog can close itself and clean its event loop, wew
def qt_code( job_key ):
HG.client_controller.CallLaterQtSafe( controller.gui, 3.0, 'close program', controller.Exit )
# no parent because this has to outlive the gui, obvs
with ClientGUITopLevelWindowsPanels.DialogNullipotent( None, 'migrating files' ) as dlg:
panel = ClientGUIPopupMessages.PopupMessageDialogPanel( dlg, job_key )
dlg.SetPanel( panel )
dlg.exec()
db = controller.db
job_key = ClientThreading.JobKey( cancel_on_shutdown = False )
job_key.SetStatusTitle( 'migrating database' )
QP.CallAfter( qt_code, job_key )
try:
job_key.SetStatusText( 'waiting for db shutdown' )
while not db.LoopIsFinished():
time.sleep( 1 )
job_key.SetStatusText( 'doing the move' )
def text_update_hook( text ):
job_key.SetStatusText( text )
for filename in os.listdir( source ):
if filename.startswith( 'client' ) and filename.endswith( '.db' ):
job_key.SetStatusText( 'moving ' + filename )
source_path = os.path.join( source, filename )
dest_path = os.path.join( dest, filename )
if os.path.exists( source_path ) and os.path.exists( dest_path ) and os.path.samefile( source_path, dest_path ):
continue
HydrusPaths.MergeFile( source_path, dest_path )
for portable_location in portable_locations:
source_path = os.path.join( source, portable_location )
dest_path = os.path.join( dest, portable_location )
if os.path.exists( source_path ) and os.path.exists( dest_path ) and os.path.samefile( source_path, dest_path ):
continue
HydrusPaths.MergeTree( source_path, dest_path, text_update_hook = text_update_hook )
job_key.SetStatusText( 'done!' )
except:
QP.CallAfter( QW.QMessageBox.critical, None, 'Error', traceback.format_exc() )
job_key.SetStatusText( 'error!' )
finally:
time.sleep( 3 )
job_key.Finish()
class MigrateTagsPanel( ClientGUIScrolledPanels.ReviewPanel ):
COPY = 0

139
hydrus/client/gui/ClientGUIStringPanels.py
View File

@ -1131,6 +1131,135 @@ class EditStringConverterPanel( ClientGUIScrolledPanels.EditPanel ):
class EditStringJoinerPanel( ClientGUIScrolledPanels.EditPanel ):
def __init__( self, parent, string_joiner: ClientStrings.StringJoiner, test_data: typing.Optional[ typing.Sequence[ str ] ] = None ):
if test_data is None:
test_data = []
ClientGUIScrolledPanels.EditPanel.__init__( self, parent )
#
self._controls_panel = ClientGUICommon.StaticBox( self, 'join text' )
self._joiner = QW.QLineEdit( self._controls_panel )
self._joiner.setToolTip( 'The strings will be joined using this text. For instance, joining "A" "B" "C" with ", " will create "A, B, C". You can enter the empty string, which simply concatenates.' )
self._join_tuple_size = ClientGUICommon.NoneableSpinCtrl( self._controls_panel, none_phrase = 'merge all into one string', min = 2 )
self._join_tuple_size.setToolTip( 'If you want to merge your strings in a 1-2, 1-2, 1-2 (e.g. you have domain-path pairs you want to joint into URLs), or 1-2-3, 1-2-3, 1-2-3 fashion, set the size of your groups here. If the remainder at the end of the list does not fit the group size, it is discarded.' )
self._summary_st = ClientGUICommon.BetterStaticText( self._controls_panel )
#
self._example_panel = ClientGUICommon.StaticBox( self, 'test results' )
self._example_strings = QW.QListWidget( self._example_panel )
self._example_strings.setSelectionMode( QW.QListWidget.NoSelection )
self._example_strings_joined = QW.QListWidget( self._example_panel )
self._example_strings_joined.setSelectionMode( QW.QListWidget.NoSelection )
#
for s in test_data:
self._example_strings.addItem( s )
#
rows = []
rows.append( ( 'text to join with: ', self._joiner ) )
rows.append( ( 'join groups of size: ', self._join_tuple_size ) )
gridbox = ClientGUICommon.WrapInGrid( self._controls_panel, rows )
self._controls_panel.Add( gridbox, CC.FLAGS_EXPAND_PERPENDICULAR )
self._controls_panel.Add( self._summary_st, CC.FLAGS_EXPAND_PERPENDICULAR )
hbox = QP.HBoxLayout()
QP.AddToLayout( hbox, self._example_strings, CC.FLAGS_EXPAND_BOTH_WAYS )
QP.AddToLayout( hbox, self._example_strings_joined, CC.FLAGS_EXPAND_BOTH_WAYS )
self._example_panel.Add( hbox, CC.FLAGS_EXPAND_SIZER_BOTH_WAYS )
vbox = QP.VBoxLayout()
QP.AddToLayout( vbox, self._controls_panel, CC.FLAGS_EXPAND_PERPENDICULAR )
QP.AddToLayout( vbox, self._example_panel, CC.FLAGS_EXPAND_BOTH_WAYS )
self.widget().setLayout( vbox )
#
self.SetValue( string_joiner )
self._joiner.textChanged.connect( self._UpdateControls )
self._join_tuple_size.valueChanged.connect( self._UpdateControls )
def _GetValue( self ):
joiner = self._joiner.text()
join_tuple_size = self._join_tuple_size.GetValue()
string_joiner = ClientStrings.StringJoiner( joiner = joiner, join_tuple_size = join_tuple_size )
return string_joiner
def _UpdateControls( self ):
string_joiner = self._GetValue()
self._summary_st.setText( string_joiner.ToString() )
texts = [ self._example_strings.item( i ).text() for i in range( self._example_strings.count() ) ]
try:
joined_texts = string_joiner.Join( texts )
except Exception as e:
joined_texts = [ 'Error: {}'.format( e ) ]
self._example_strings_joined.clear()
for s in joined_texts:
self._example_strings_joined.addItem( s )
def GetValue( self ):
string_slicer = self._GetValue()
return string_slicer
def SetValue( self, string_joiner: ClientStrings.StringJoiner ):
joiner = string_joiner.GetJoiner()
join_tuple_size = string_joiner.GetJoinTupleSize()
self._joiner.setText( joiner )
self._join_tuple_size.SetValue( join_tuple_size )
self._UpdateControls()
class EditStringMatchPanel( ClientGUIScrolledPanels.EditPanel ):
def __init__( self, parent: QW.QWidget, string_match: ClientStrings.StringMatch, test_data = typing.Optional[ ClientParsing.ParsingTestData ] ):
@ -1564,6 +1693,7 @@ class EditStringSlicerPanel( ClientGUIScrolledPanels.EditPanel ):
self._ShowHideControls()
class EditStringSorterPanel( ClientGUIScrolledPanels.EditPanel ):
def __init__( self, parent, string_sorter: ClientStrings.StringSorter, test_data: typing.Optional[ typing.Sequence[ str ] ] = None ):
@ -1724,6 +1854,7 @@ class EditStringSorterPanel( ClientGUIScrolledPanels.EditPanel ):
self._UpdateControls()
class EditStringSplitterPanel( ClientGUIScrolledPanels.EditPanel ):
def __init__( self, parent, string_splitter: ClientStrings.StringSplitter, example_string: str = '' ):
@ -1847,6 +1978,7 @@ class EditStringSplitterPanel( ClientGUIScrolledPanels.EditPanel ):
self._UpdateControls()
class EditStringTagFilterPanel( ClientGUIScrolledPanels.EditPanel ):
def __init__( self, parent: QW.QWidget, string_tag_filter: ClientStrings.StringTagFilter, test_data = typing.Optional[ ClientParsing.ParsingTestData ] ):
@ -2039,6 +2171,7 @@ class EditStringProcessorPanel( ClientGUIScrolledPanels.EditPanel ):
( 'String Tag Filter', ClientStrings.StringTagFilter, 'An object that filters strings using tag rules.' ),
( 'String Converter', ClientStrings.StringConverter, 'An object that converts strings from one thing to another.' ),
( 'String Splitter', ClientStrings.StringSplitter, 'An object that breaks strings into smaller strings.' ),
( 'String Joiner', ClientStrings.StringJoiner, 'An object that concatenates strings together.' ),
( 'String Sorter', ClientStrings.StringSorter, 'An object that sorts strings.' ),
( 'String Selector/Slicer', ClientStrings.StringSlicer, 'An object that filter-selects from the list of strings. Either absolute index position or a range.' )
]
@ -2102,6 +2235,12 @@ class EditStringProcessorPanel( ClientGUIScrolledPanels.EditPanel ):
panel = EditStringSlicerPanel( dlg, string_processing_step, test_data = test_data )
elif isinstance( string_processing_step, ClientStrings.StringJoiner ):
test_data = self._GetExampleTextsForStringSorter( string_processing_step )
panel = EditStringJoinerPanel( dlg, string_processing_step, test_data = test_data )
elif isinstance( string_processing_step, ClientStrings.StringTagFilter ):
test_data = ClientParsing.ParsingTestData( {}, ( example_text, ) )

1
hydrus/client/gui/canvas/ClientGUICanvas.py
View File

@ -1322,6 +1322,7 @@ class Canvas( CAC.ApplicationCommandProcessorMixin, QW.QWidget ):
class MediaContainerDragClickReportingFilter( QC.QObject ):
def __init__( self, parent: Canvas ):

4
hydrus/client/gui/pages/ClientGUIManagementPanels.py
View File

@ -1268,6 +1268,8 @@ class ManagementPanelImporterMultipleGallery( ManagementPanelImporter ):
self._loading_highlight_job_key = ClientThreading.JobKey( cancellable = True )
self._loading_highlight_job_key.Finish()
#
self._gallery_downloader_panel = ClientGUICommon.StaticBox( self, 'gallery downloader' )
@ -2200,6 +2202,8 @@ class ManagementPanelImporterMultipleWatcher( ManagementPanelImporter ):
self._loading_highlight_job_key = ClientThreading.JobKey( cancellable = True )
self._loading_highlight_job_key.Finish()
checker_options = self._multiple_watcher_import.GetCheckerOptions()
file_import_options = self._multiple_watcher_import.GetFileImportOptions()
tag_import_options = self._multiple_watcher_import.GetTagImportOptions()

22
hydrus/client/media/ClientMedia.py
View File

@ -1811,8 +1811,15 @@ class MediaSingleton( Media ):
import_timestamp = timestamps_manager.GetImportedTimestamp( CC.COMBINED_LOCAL_FILE_SERVICE_KEY )
# if we haven't already printed this timestamp somewhere
line_is_interesting = False not in ( timestamp_is_interesting( t, import_timestamp ) for t in seen_local_file_service_timestamps )
if HG.client_controller.new_options.GetBoolean( 'hide_uninteresting_local_import_time' ):
# if we haven't already printed this timestamp somewhere
line_is_interesting = False not in ( timestamp_is_interesting( t, import_timestamp ) for t in seen_local_file_service_timestamps )
else:
line_is_interesting = True
lines.append( ( line_is_interesting, 'imported: {}'.format( ClientTime.TimestampToPrettyTimeDelta( import_timestamp ) ) ) )
@ -1868,8 +1875,15 @@ class MediaSingleton( Media ):
if file_modified_timestamp is not None:
# if we haven't already printed this timestamp somewhere
line_is_interesting = False not in ( timestamp_is_interesting( timestamp, file_modified_timestamp ) for timestamp in seen_local_file_service_timestamps )
if HG.client_controller.new_options.GetBoolean( 'hide_uninteresting_modified_time' ):
# if we haven't already printed this timestamp somewhere
line_is_interesting = False not in ( timestamp_is_interesting( timestamp, file_modified_timestamp ) for timestamp in seen_local_file_service_timestamps )
else:
line_is_interesting = True
lines.append( ( line_is_interesting, 'modified: {}'.format( ClientTime.TimestampToPrettyTimeDelta( file_modified_timestamp ) ) ) )

68
hydrus/client/networking/ClientLocalServerResources.py
View File

@ -167,6 +167,26 @@ def CheckTagService( tag_service_key: bytes ):
return service
def CheckTags( tags: typing.Collection[ str ] ):
for tag in tags:
try:
clean_tag = HydrusTags.CleanTag( tag )
except Exception as e:
raise HydrusExceptions.BadRequestException( 'Could not parse tag "{}"!'.format( tag ) )
if clean_tag == '':
raise HydrusExceptions.BadRequestException( 'Tag "{}" was empty!'.format( tag ) )
def GetServicesDict():
service_types = [
@ -2121,6 +2141,8 @@ class HydrusResourceClientAPIRestrictedAddTagsSearchTags( HydrusResourceClientAP
def _CheckAPIPermissions( self, request: HydrusServerRequest.HydrusRequest ):
# this doesn't need 'add tags' atm. I was going to add it, but I'm not sure it is actually appropriate
# this thing probably should have been in search files space, but whatever
request.client_api_permissions.CheckPermission( ClientAPI.CLIENT_API_PERMISSION_SEARCH_FILES )
@ -2199,40 +2221,50 @@ class HydrusResourceClientAPIRestrictedAddTagsSearchTags( HydrusResourceClientAP
return response_context
class HydrusResourceClientAPIRestrictedAddTagsGetTagSiblingsParents( HydrusResourceClientAPIRestrictedAddTags ):
def _CheckAPIPermissions( self, request: HydrusServerRequest.HydrusRequest ):
request.client_api_permissions.CheckPermission( ClientAPI.CLIENT_API_PERMISSION_SEARCH_FILES )
class HydrusResourceClientAPIRestrictedAddTagsGetTagSiblingsParents( HydrusResourceClientAPIRestrictedAddTags ):
def _threadDoGETJob( self, request: HydrusServerRequest.HydrusRequest ):
tags = request.parsed_request_args.GetValue( 'tags', list, expected_list_type = str )
CheckTags( tags )
tags_to_service_keys_to_siblings_and_parents = HG.client_controller.Read( 'tag_siblings_and_parents_lookup', tags )
body_dict = { }
for tag, service_keys_to_siblings_parents in tags_to_service_keys_to_siblings_and_parents.items():
body_dict[tag] = {}
for service_key, siblings_parents in service_keys_to_siblings_parents.items():
body_dict[tag][service_key.hex()] = {
'sibling_chain_members': list(siblings_parents[0]),
tags_dict = {}
for ( tag, service_keys_to_siblings_parents ) in tags_to_service_keys_to_siblings_and_parents.items():
tag_dict = {}
for ( service_key, siblings_parents ) in service_keys_to_siblings_parents.items():
tag_dict[ service_key.hex() ] = {
'siblings': list( siblings_parents[0] ),
'ideal_tag': siblings_parents[1],
'descendants': list(siblings_parents[2]),
'ancestors': list(siblings_parents[3])
'descendants': list( siblings_parents[2] ),
'ancestors': list( siblings_parents[3] )
}
tags_dict[ tag ] = tag_dict
body_dict = {
'tags' : tags_dict,
'services' : GetServicesDict()
}
body = Dumps( body_dict, request.preferred_mime )
response_context = HydrusServerResources.ResponseContext( 200, mime = request.preferred_mime, body = body )
return response_context
class HydrusResourceClientAPIRestrictedAddTagsCleanTags( HydrusResourceClientAPIRestrictedAddTags ):
def _threadDoGETJob( self, request: HydrusServerRequest.HydrusRequest ):

43
hydrus/core/HydrusAnimationHandling.py
View File

@ -1,3 +1,4 @@
import io
import typing
import struct
@ -41,11 +42,48 @@ def GetAPNGChunks( file_header_bytes: bytes ) ->list:
# n bytes of data
# 4 bytes of CRC
# lop off 8 bytes of 'this is a PNG' at the top
remaining_chunk_bytes = file_header_bytes[8:]
# ok this method went super slow when given a 200MB giga png
# it turns out list slicing on a very large bytes object is extremely slow
# so we'll move to a file-like BytesIO and just read the stream like a file
# note lol that if you debug this you'll still get the mega slowdown as your IDE copies the giganto bytes around over and over for variable inspection
chunks = []
buffer = io.BytesIO( file_header_bytes )
# lop off 8 bytes of 'this is a PNG' at the top
buffer.read( 8 )
while True:
chunk_num_bytes = buffer.read( 4 )
chunk_name = buffer.read( 4 )
if len( chunk_num_bytes ) < 4 or len( chunk_name ) < 4:
break
( num_data_bytes, ) = struct.unpack( '>I', chunk_num_bytes )
chunk_data = buffer.read( num_data_bytes )
if len( chunk_data ) < num_data_bytes:
break
buffer.read( 4 )
chunks.append( ( chunk_name, chunk_data ) )
# old solution
'''
remaining_chunk_bytes = file_header_bytes[8:]
while len( remaining_chunk_bytes ) > 12:
( num_data_bytes, ) = struct.unpack( '>I', remaining_chunk_bytes[ : 4 ] )
@ -58,6 +96,7 @@ def GetAPNGChunks( file_header_bytes: bytes ) ->list:
remaining_chunk_bytes = remaining_chunk_bytes[ 8 + num_data_bytes + 4 : ]
'''
return chunks

12
hydrus/core/HydrusConstants.py
View File

@ -100,8 +100,8 @@ options = {}
# Misc
NETWORK_VERSION = 20
SOFTWARE_VERSION = 540
CLIENT_API_VERSION = 50
SOFTWARE_VERSION = 541
CLIENT_API_VERSION = 51
SERVER_THUMBNAIL_DIMENSIONS = ( 200, 200 )
@ -866,13 +866,13 @@ APPLICATIONS = [
]
IMAGE_PROJECT_FILES = [
APPLICATION_PSD,
APPLICATION_CLIP,
APPLICATION_SAI2,
APPLICATION_KRITA,
APPLICATION_PROCREATE,
APPLICATION_PSD,
APPLICATION_SAI2,
IMAGE_SVG,
APPLICATION_XCF,
APPLICATION_PROCREATE
APPLICATION_XCF
]
ARCHIVES = [

1
hydrus/core/HydrusExceptions.py
View File

@ -69,6 +69,7 @@ class TagSizeException( VetoException ): pass
class ParseException( HydrusException ): pass
class StringConvertException( ParseException ): pass
class StringJoinerException( ParseException ): pass
class StringMatchException( ParseException ): pass
class StringSplitterException( ParseException ): pass
class StringSortException( ParseException ): pass

60
hydrus/core/HydrusProcreateHandling.py
View File

@ -18,69 +18,75 @@ def ExtractZippedThumbnailToPath( path_to_zip, temp_path_file ):
except KeyError:
raise HydrusExceptions.DamagedOrUnusualFileException( f'This procreate file had no thumbnail file!' )
def GetProcreatePlist( path ):
plist_file = HydrusArchiveHandling.GetZipAsPath( path, PROCREATE_DOCUMENT_ARCHIVE )
if not plist_file.exists():
raise HydrusExceptions.DamagedOrUnusualFileException('Procreate file has no plist!')
raise HydrusExceptions.DamagedOrUnusualFileException('Procreate file has no plist!')
with HydrusArchiveHandling.GetZipAsPath( path, PROCREATE_DOCUMENT_ARCHIVE ).open('rb') as document:
return plistlib.load(document)
def ZipLooksLikeProcreate( path ) -> bool:
try:
document = GetProcreatePlist( path )
objects = document['$objects']
class_pointer = objects[PROCREATE_PROJECT_KEY]['$class']
class_name = objects[class_pointer]['$classname']
return class_name == 'SilicaDocument'
except:
return False
def GetProcreateResolution( path ):
# TODO: animation stuff from plist
document = GetProcreatePlist( path )
objects = document['$objects']
dimension_pointer = objects[PROCREATE_PROJECT_KEY]['size'].data
# eg '{2894, 4093}'
size_string = objects[dimension_pointer]
size = size_string.strip('{').strip('}').split(', ')
orientation = objects[PROCREATE_PROJECT_KEY]['orientation']
if orientation in [3,4]:
# canvas is rotated 90 or -90 degrees
height = size[1]
width = size[0]
else:
height = size[0]
width = size[1]
return int(width), int(height)

1
hydrus/core/HydrusSerialisable.py
View File

@ -140,6 +140,7 @@ SERIALISABLE_TYPE_TIMESTAMP_DATA = 121
SERIALISABLE_TYPE_METADATA_SINGLE_FILE_EXPORTER_MEDIA_TIMESTAMPS = 122
SERIALISABLE_TYPE_METADATA_SINGLE_FILE_IMPORTER_MEDIA_TIMESTAMPS = 123
SERIALISABLE_TYPE_PETITION_HEADER = 124
SERIALISABLE_TYPE_STRING_JOINER = 125
SERIALISABLE_TYPES_TO_OBJECT_TYPES = {}

126
hydrus/test/TestClientAPI.py
View File

@ -2175,6 +2175,131 @@ class TestClientAPI( unittest.TestCase ):
self._compare_content_updates( service_keys_to_content_updates, expected_service_keys_to_content_updates )
def _test_add_tags_get_tag_siblings_and_parents( self, connection, set_up_permissions ):
db_data = {}
db_data[ 'blue eyes' ] = {
CC.DEFAULT_LOCAL_TAG_SERVICE_KEY : [
{
"blue eyes",
"blue_eyes",
"blue eye",
"blue_eye"
},
'blue eyes',
set(),
set()
],
HG.test_controller.example_tag_repo_service_key : [
{ 'blue eyes' },
'blue eyes',
set(),
set()
]
}
db_data[ 'samus aran' ] = {
CC.DEFAULT_LOCAL_TAG_SERVICE_KEY : [
{
"samus aran",
"samus_aran",
"character:samus aran"
},
'character:samus aran',
{
"character:samus aran (zero suit)"
"cosplay:samus aran"
},
{
"series:metroid",
"studio:nintendo"
}
],
HG.test_controller.example_tag_repo_service_key : [
{ 'samus aran' },
'samus aran',
{
"zero suit samus",
"samus_aran_(cosplay)"
},
set()
]
}
HG.test_controller.SetRead( 'tag_siblings_and_parents_lookup', db_data )
#
api_permissions = set_up_permissions[ 'add_urls' ]
access_key_hex = api_permissions.GetAccessKey().hex()
headers = { 'Hydrus-Client-API-Access-Key' : access_key_hex }
#
path = '/add_tags/get_siblings_and_parents?tags={}'.format( urllib.parse.quote( json.dumps( [ 'blue eyes', 'samus aran' ] ) ) )
connection.request( 'GET', path, headers = headers )
response = connection.getresponse()
data = response.read()
text = str( data, 'utf-8' )
self.assertEqual( response.status, 403 )
#
api_permissions = set_up_permissions[ 'everything' ]
access_key_hex = api_permissions.GetAccessKey().hex()
headers = { 'Hydrus-Client-API-Access-Key' : access_key_hex }
#
path = '/add_tags/get_siblings_and_parents?tags={}'.format( urllib.parse.quote( json.dumps( [ 'blue eyes', 'samus aran' ] ) ) )
connection.request( 'GET', path, headers = headers )
response = connection.getresponse()
data = response.read()
text = str( data, 'utf-8' )
self.assertEqual( response.status, 200 )
d = json.loads( text )
expected_answer = {
'services' : GetExampleServicesDict(),
'tags' : {}
}
for ( tag, data ) in db_data.items():
tag_dict = {}
for ( service_key, ( siblings, ideal_tag, descendants, ancestors ) ) in data.items():
tag_dict[ service_key.hex() ] = {
'siblings' : list( siblings ),
'ideal_tag' : ideal_tag,
'descendants' : list( descendants ),
'ancestors' : list( ancestors )
}
expected_answer[ 'tags' ][ tag ] = tag_dict
self.assertEqual( expected_answer, d )
def _test_add_tags_search_tags( self, connection, set_up_permissions ):
predicates = [
@ -5594,6 +5719,7 @@ class TestClientAPI( unittest.TestCase ):
self._test_add_ratings( connection, set_up_permissions )
self._test_add_tags( connection, set_up_permissions )
self._test_add_tags_search_tags( connection, set_up_permissions )
self._test_add_tags_get_tag_siblings_and_parents( connection, set_up_permissions )
self._test_add_urls( connection, set_up_permissions )
self._test_manage_duplicates( connection, set_up_permissions )
self._test_manage_cookies( connection, set_up_permissions )

124
hydrus/test/TestClientFileStorage.py Normal file
View File

@ -0,0 +1,124 @@
import itertools
import os
import shutil
import unittest
from hydrus.core import HydrusConstants as HC
from hydrus.core import HydrusData
from hydrus.core import HydrusExceptions
from hydrus.core import HydrusGlobals as HG
from hydrus.client import ClientConstants as CC
from hydrus.client import ClientFilesPhysical
def get_good_prefixes():
good_prefixes = [ f'f{prefix}' for prefix in HydrusData.IterateHexPrefixes() ]
good_prefixes.extend( [ f't{prefix}' for prefix in HydrusData.IterateHexPrefixes() ] )
return good_prefixes
class TestClientFileStorage( unittest.TestCase ):
def test_functions( self ):
hex_chars = '0123456789abcdef'
good_prefixes = get_good_prefixes()
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 'f', good_prefixes ), [] )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 't', good_prefixes ), [] )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 'f1', good_prefixes ), [] )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 't6', good_prefixes ), [] )
#
# testing here that the implicit 'report two char length at least' thing works, so if we remove a whole contiguous segment, it reports the longer result and not 'f1'
f1_series = [ 'f1' + c for c in hex_chars ]
good_prefixes = get_good_prefixes()
for prefix in f1_series:
good_prefixes.remove( prefix )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 'f', good_prefixes ), f1_series )
#
hex_chars = '0123456789abcdef'
good_prefixes = get_good_prefixes()
good_prefixes.append( 'f14' )
good_prefixes.append( 't63' )
good_prefixes.append( 'f145' )
good_prefixes.append( 't634' )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 'f', good_prefixes ), [] )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 't', good_prefixes ), [] )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 'f1', good_prefixes ), [] )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 't6', good_prefixes ), [] )
#
# same deal but missing everything
good_prefixes = [ prefix for prefix in get_good_prefixes() if prefix.startswith( 'f' ) ]
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 'f', [] ), good_prefixes )
#
good_prefixes = get_good_prefixes()
good_prefixes.remove( 'f53' )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 'f', good_prefixes ), [ 'f53' ] )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 't', good_prefixes ), [] )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 'f5', good_prefixes ), [ 'f53' ] )
#
good_prefixes = get_good_prefixes()
good_prefixes.remove( 't11' )
good_prefixes.extend( [ f't11{i}' for i in hex_chars ] )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 't', good_prefixes ), [] )
good_prefixes.remove( 't46' )
good_prefixes.remove( 't115' )
self.assertEqual( ClientFilesPhysical.GetMissingPrefixes( 't', good_prefixes ), [ 't115', 't46' ] )
#
good_prefixes = get_good_prefixes()
good_prefixes = [ f'f{prefix}' for prefix in HydrusData.IterateHexPrefixes() ]
good_prefixes.extend( [ f't{prefix}' for prefix in HydrusData.IterateHexPrefixes() ] )
ClientFilesPhysical.CheckFullPrefixCoverage( 'f', good_prefixes )
ClientFilesPhysical.CheckFullPrefixCoverage( 't', good_prefixes )
good_prefixes.remove( 'f00' )
good_prefixes.remove( 't06' )
with self.assertRaises( HydrusExceptions.DataMissing ):
ClientFilesPhysical.CheckFullPrefixCoverage( 'f', good_prefixes )
with self.assertRaises( HydrusExceptions.DataMissing ):
ClientFilesPhysical.CheckFullPrefixCoverage( 't', good_prefixes )

27
hydrus/test/TestClientParsing.py
View File

@ -309,6 +309,33 @@ class TestStringConverter( unittest.TestCase ):
self.assertEqual( string_converter.Convert( '0123456789' ), 'z xddddddcba' )
class TestStringJoiner( unittest.TestCase ):
def test_basics( self ):
texts = [
'ab',
'cd',
'ef',
'gh',
'ij'
]
#
joiner = ClientStrings.StringJoiner( joiner = '', join_tuple_size = None )
self.assertEqual( joiner.Join( texts ), [ 'abcdefghij' ] )
self.assertEqual( joiner.ToString(), 'joining all strings using ""' )
joiner = ClientStrings.StringJoiner( joiner = ',', join_tuple_size = None )
self.assertEqual( joiner.Join( texts ), [ 'ab,cd,ef,gh,ij' ] )
self.assertEqual( joiner.ToString(), 'joining all strings using ","' )
joiner = ClientStrings.StringJoiner( joiner = '--', join_tuple_size = 2 )
self.assertEqual( joiner.Join( texts ), [ 'ab--cd', 'ef--gh' ] )
self.assertEqual( joiner.ToString(), 'joining every 2 strings using "--"' )
class TestStringMatch( unittest.TestCase ):
def test_basics( self ):

10
hydrus/test/TestController.py
View File

@ -23,6 +23,7 @@ from hydrus.client import ClientAPI
from hydrus.client import ClientConstants as CC
from hydrus.client import ClientDefaults
from hydrus.client import ClientFiles
from hydrus.client import ClientFilesPhysical
from hydrus.client import ClientOptions
from hydrus.client import ClientManagers
from hydrus.client import ClientServices
@ -47,6 +48,7 @@ from hydrus.test import TestClientData
from hydrus.test import TestClientDB
from hydrus.test import TestClientDBDuplicates
from hydrus.test import TestClientDBTags
from hydrus.test import TestClientFileStorage
from hydrus.test import TestClientImageHandling
from hydrus.test import TestClientImportOptions
from hydrus.test import TestClientImportSubscriptions
@ -261,17 +263,17 @@ class Controller( object ):
self._name_read_responses[ 'services' ] = services
client_files_locations = {}
client_files_subfolders = []
for prefix in HydrusData.IterateHexPrefixes():
for c in ( 'f', 't' ):
client_files_locations[ c + prefix ] = client_files_default
client_files_subfolders.append( ClientFilesPhysical.FilesStorageSubfolder( c + prefix, client_files_default, False ) )
self._name_read_responses[ 'client_files_locations' ] = client_files_locations
self._name_read_responses[ 'client_files_subfolders' ] = client_files_subfolders
self._name_read_responses[ 'sessions' ] = []
self._name_read_responses[ 'tag_parents' ] = {}
@ -775,6 +777,7 @@ class Controller( object ):
TestClientDaemons,
TestClientConstants,
TestClientData,
TestClientFileStorage,
TestClientImportOptions,
TestClientParsing,
TestClientTags,
@ -815,6 +818,7 @@ class Controller( object ):
module_lookup[ 'data' ] = [
TestClientConstants,
TestClientData,
TestClientFileStorage,
TestClientImportOptions,
TestClientParsing,
TestClientTags,

4
hydrus_client.bat
View File

@ -4,7 +4,7 @@ pushd "%~dp0"
IF NOT EXIST "venv\" (
SET /P gumpf=You need to set up a venv! Check the running from source help for more info!
SET /P gumpf="You need to set up a venv! Check the running from source help for more info!"
popd
@ -16,7 +16,7 @@ CALL venv\Scripts\activate.bat
IF ERRORLEVEL 1 (
SET /P gumpf=The venv failed to activate, stopping now!
SET /P gumpf="The venv failed to activate, stopping now!"
popd

2
open_venv.bat
View File

@ -4,7 +4,7 @@ pushd "%~dp0"
IF NOT EXIST "venv\" (
SET /P gumpf=Sorry, you do not seem to have a venv!
SET /P gumpf="Sorry, you do not seem to have a venv!"
popd

22
setup_help.bat
View File

@ -4,22 +4,22 @@ pushd "%~dp0"
IF NOT EXIST "venv\" (
SET /P gumpf=You need to set up a venv! Check the running from source help for more info!
popd
EXIT /B 1
SET /P gumpf="You need to set up a venv! Check the running from source help for more info!"
popd
EXIT /B 1
)
:delete
IF EXIST "help\" (
echo Deleting old help...
rmdir /s /q help
echo Deleting old help...
rmdir /s /q help
)
:create
@ -34,6 +34,6 @@ mkdocs build -d help
CALL venv\Scripts\deactivate.bat
SET /P done=Done!
SET /P done="Done!"
popd

120
setup_venv.bat
View File

@ -4,27 +4,27 @@ pushd "%~dp0"
where /q python
IF ERRORLEVEL 1 (
SET /P gumpf=You do not seem to have python installed. Please check the 'running from source' help.
popd
EXIT /B 1
SET /P gumpf="You do not seem to have python installed. Please check the 'running from source' help."
popd
EXIT /B 1
)
IF EXIST "venv\" (
SET /P ready=Virtual environment will be reinstalled. Hit Enter to start.
echo Deleting old venv...
rmdir /s /q venv
SET /P ready="Virtual environment will be reinstalled. Hit Enter to start."
echo Deleting old venv...
rmdir /s /q venv
) ELSE (
SET /P ready=If you do not know what this is, check the 'running from source' help. Hit Enter to start.
SET /P ready="If you do not know what this is, check the 'running from source' help. Hit Enter to start."
)
:questions
@ -35,7 +35,7 @@ ECHO:
ECHO Your Python version is:
python --version
ECHO:
SET /P install_type=Do you want the (s)imple or (a)dvanced install?
SET /P install_type="Do you want the (s)imple or (a)dvanced install? "
IF "%install_type%" == "s" goto :create
IF "%install_type%" == "a" goto :question_qt
@ -47,7 +47,7 @@ goto :parse_fail
ECHO:
ECHO Qt is the User Interface library. We are now on Qt6.
ECHO If you are on Windows ^<=8.1, choose 5. If 6 gives you trouble, fall back to o.
SET /P qt=Do you want Qt(5), Qt(6), Qt6 (o)lder, or (t)est?
SET /P qt="Do you want Qt(5), Qt(6), Qt6 (o)lder, or (t)est? "
IF "%qt%" == "5" goto :question_mpv
IF "%qt%" == "6" goto :question_mpv
@ -60,7 +60,7 @@ goto :parse_fail
ECHO:
ECHO mpv is the main way to play audio and video. We need to tell hydrus how to talk to your mpv dll.
ECHO Try the n first. If it doesn't work, fall back to o.
SET /P mpv=Do you want (o)ld mpv or (n)ew mpv?
SET /P mpv="Do you want (o)ld mpv or (n)ew mpv? "
IF "%mpv%" == "o" goto :question_opencv
IF "%mpv%" == "n" goto :question_opencv
@ -71,7 +71,7 @@ goto :parse_fail
ECHO:
ECHO OpenCV is the main image processing library.
ECHO Try the n first. If it doesn't work, fall back to o. Very new python versions might need t.
SET /P opencv=Do you want (o)ld OpenCV, (n)ew OpenCV, or (t)est OpenCV?
SET /P opencv="Do you want (o)ld OpenCV, (n)ew OpenCV, or (t)est OpenCV? "
IF "%opencv%" == "o" goto :create
IF "%opencv%" == "n" goto :create
@ -87,13 +87,13 @@ python -m venv venv
CALL venv\Scripts\activate.bat
IF ERRORLEVEL 1 (
SET /P gumpf=The venv failed to activate, stopping now!
popd
EXIT /B 1
SET /P gumpf="The venv failed to activate, stopping now!"
popd
EXIT /B 1
)
python -m pip install --upgrade pip
@ -101,48 +101,48 @@ python -m pip install --upgrade pip
python -m pip install --upgrade wheel
IF "%install_type%" == "s" (
python -m pip install -r requirements.txt
python -m pip install -r requirements.txt
)
IF "%install_type%" == "d" (
python -m pip install -r static\requirements\advanced\requirements_core.txt
python -m pip install -r static\requirements\advanced\requirements_windows.txt
python -m pip install -r static\requirements\advanced\requirements_qt6_test.txt
python -m pip install pyside2
python -m pip install PyQtChart PyQt5
python -m pip install PyQt6-Charts PyQt6
python -m pip install -r static\requirements\advanced\requirements_mpv_new.txt
python -m pip install -r static\requirements\advanced\requirements_opencv_test.txt
python -m pip install -r static\requirements\hydev\requirements_windows_build.txt
python -m pip install -r static\requirements\advanced\requirements_core.txt
python -m pip install -r static\requirements\advanced\requirements_windows.txt
python -m pip install -r static\requirements\advanced\requirements_qt6_test.txt
python -m pip install pyside2
python -m pip install PyQtChart PyQt5
python -m pip install PyQt6-Charts PyQt6
python -m pip install -r static\requirements\advanced\requirements_mpv_new.txt
python -m pip install -r static\requirements\advanced\requirements_opencv_test.txt
python -m pip install -r static\requirements\hydev\requirements_windows_build.txt
)
IF "%install_type%" == "a" (
python -m pip install -r static\requirements\advanced\requirements_core.txt
python -m pip install -r static\requirements\advanced\requirements_windows.txt
IF "%qt%" == "5" python -m pip install -r static\requirements\advanced\requirements_qt5.txt
IF "%qt%" == "6" python -m pip install -r static\requirements\advanced\requirements_qt6.txt
IF "%qt%" == "o" python -m pip install -r static\requirements\advanced\requirements_qt6_older.txt
IF "%qt%" == "t" python -m pip install -r static\requirements\advanced\requirements_qt6_test.txt
IF "%mpv%" == "o" python -m pip install -r static\requirements\advanced\requirements_mpv_old.txt
IF "%mpv%" == "n" python -m pip install -r static\requirements\advanced\requirements_mpv_new.txt
IF "%opencv%" == "o" python -m pip install -r static\requirements\advanced\requirements_opencv_old.txt
IF "%opencv%" == "n" python -m pip install -r static\requirements\advanced\requirements_opencv_new.txt
IF "%opencv%" == "t" python -m pip install -r static\requirements\advanced\requirements_opencv_test.txt
python -m pip install -r static\requirements\advanced\requirements_core.txt
python -m pip install -r static\requirements\advanced\requirements_windows.txt
IF "%qt%" == "5" python -m pip install -r static\requirements\advanced\requirements_qt5.txt
IF "%qt%" == "6" python -m pip install -r static\requirements\advanced\requirements_qt6.txt
IF "%qt%" == "o" python -m pip install -r static\requirements\advanced\requirements_qt6_older.txt
IF "%qt%" == "t" python -m pip install -r static\requirements\advanced\requirements_qt6_test.txt
IF "%mpv%" == "o" python -m pip install -r static\requirements\advanced\requirements_mpv_old.txt
IF "%mpv%" == "n" python -m pip install -r static\requirements\advanced\requirements_mpv_new.txt
IF "%opencv%" == "o" python -m pip install -r static\requirements\advanced\requirements_opencv_old.txt
IF "%opencv%" == "n" python -m pip install -r static\requirements\advanced\requirements_opencv_new.txt
IF "%opencv%" == "t" python -m pip install -r static\requirements\advanced\requirements_opencv_test.txt
)
CALL venv\Scripts\deactivate.bat
SET /P done=Done!
SET /P done="Done!"
popd
@ -150,7 +150,7 @@ EXIT /B 0
:parse_fail
SET /P done=Sorry, did not understand that input!
SET /P done="Sorry, did not understand that input!"
popd

438
static/qss/CutieDuck_(darkorange)-alternate-tooltip-colour.qss Normal file
View File

@ -0,0 +1,438 @@
/* CutieDuck Hydrus.qss based on QTOrange */
QToolTip
{
border: 1px solid black;
background-color: white;
padding: 1px;
}
QWidget
{
color: white;
background-color: #323232;
alternate-background-color: #323232;
}
QWidget:disabled
{
background-color: #404040;
}
QWidget:item:hover
{
background-color: #d7801a;
color: black;
}
QWidget:item:selected
{
background-color: #ffa02f;
}
QMenuBar::item
{
background: transparent;
}
QMenuBar::item:selected
{
background: transparent;
border: 1px solid #ffa02f;
}
QMenu
{
border: 1px solid #000;
}
QMenu::item
{
padding: 2px 20px 2px 20px;
}
QMenu::item:selected
{
color: black;
}
QAbstractItemView
{
background-color: #4d4d4d;
}
QLineEdit
{
color: white;
background-color: #4d4d4d;
padding: 1px;
border-style: solid;
border: 1px solid #000000;
}
QPushButton
{
color: white;
background-color: #565656;
border-width: 1px;
border-color: #000000;
border-style: solid;
padding: 3px;
padding-left: 5px;
padding-right: 5px;
}
QPushButton:pressed
{
background-color: #323232;
}
QComboBox
{
selection-background-color: #ffa02f;
background-color: #565656;
border-style: solid;
border: 1.0px solid #000000;
}
QComboBox:hover,QPushButton:hover
{
border: 1px solid #d7801a;
}
QComboBox:on
{
padding-top: 3px;
padding-left: 4px;
background-color: #2d2d2d;
selection-background-color: #ffa02f;
}
QComboBox QAbstractItemView
{
border: 1px solid darkgray;
selection-background-color: #ffa02f;
}
QComboBox::drop-down
{
subcontrol-origin: padding;
subcontrol-position: top right;
width: 15px;
border-left-width: 0px;
border-left-color: darkgray;
border-left-style: solid; /* just a single line */
}
QComboBox::down-arrow
{
image: url(:/down_arrow.png);
}
QGroupBox:focus
{
border: 1px solid #d7801a;
}
QTextEdit:focus
{
border: 1px solid #2d2d2d;
}
QScrollBar:horizontal {
border: 1px solid #000000;
background: black;
height: 7px;
margin: 0px 16px 0 16px;
}
QScrollBar::handle:horizontal
{
background: #ffa02f;
min-height: 20px;
border-radius: 2px;
}
QScrollBar::add-line:horizontal {
border: 1px solid #000000;
border-radius: 2px;
background: #d7801a;
width: 14px;
subcontrol-position: right;
subcontrol-origin: margin;
}
QScrollBar::sub-line:horizontal {
border: 1px solid #000000;
border-radius: 2px;
background: #d7801a;
width: 14px;
subcontrol-position: left;
subcontrol-origin: margin;
}
QScrollBar::right-arrow:horizontal, QScrollBar::left-arrow:horizontal
{
border: 1px solid black;
width: 1px;
height: 1px;
background: white;
}
QScrollBar::add-page:horizontal, QScrollBar::sub-page:horizontal
{
background: none;
}
QScrollBar:vertical
{
background: #484848;
width: 7px;
margin: 16px 0 16px 0;
border: 1px solid #000000;
}
QScrollBar::handle:vertical
{
background: #ffa02f;
min-height: 20px;
}
QScrollBar::add-line:vertical
{
border: 1px solid #000000;
background: #ffa02f;
height: 14px;
subcontrol-position: bottom;
subcontrol-origin: margin;
}
QScrollBar::sub-line:vertical
{
border: 1px solid #000000;
background: #ffa02f;
height: 14px;
subcontrol-position: top;
subcontrol-origin: margin;
}
QScrollBar::up-arrow:vertical, QScrollBar::down-arrow:vertical
{
border: 1px solid black;
width: 1px;
height: 1px;
background: white;
}
QScrollBar::add-page:vertical, QScrollBar::sub-page:vertical
{
background: none;
}
QTextEdit
{
background-color: #000000;
}
QPlainTextEdit
{
background-color: #000000;
}
QHeaderView::section
{
background-color: #616161;
color: white;
padding-left: 4px;
border: 1px solid #6c6c6c;
}
QProgressBar
{
border: 1px solid grey;
text-align: center;
}
QProgressBar::chunk
{
background-color: #d7801a;
}
QTabBar::tab {
color: #b1b1b1;
background-color: #323232;
padding-left: 10px;
padding-right: 10px;
padding-top: 3px;
padding-bottom: 2px;
}
QTabWidget::pane {
border: 1px solid #444;
top: 1px;
}
QTabBar::tab:last
{
margin-right: 0; /* the last selected tab has nothing to overlap with on the right */
border-top-right-radius: 3px;
}
QTabBar::tab:first:!selected
{
margin-left: 0px; /* the last selected tab has nothing to overlap with on the right */
border-top-left-radius: 3px;
}
QTabBar::tab:selected
{
color: black;
border-bottom-style: solid;
background-color: #ffa02f;
}
QTabBar::tab:hover:!selected
{
color: black;
background-color: #d7801a;
}
/*
DUCK
*/
/*
Default QSS for hydrus. This is prepended to any stylesheet loaded in hydrus.
Copying these entries in your own stylesheets should override these settings.
This will get more work in future.
*/
/*
Here are some text and background colours
*/
/* Example: This regex is valid */
QLabel#HydrusValid
{
color: #2ed42e;
}
QLineEdit#HydrusValid
{
background-color: #80ff80;
}
QTextEdit#HydrusValid
{
background-color: #80ff80;
}
QPlainTextEdit#HydrusValid
{
background-color: #80ff80;
}
/* Duplicates 'middle' text colour */
QLabel#HydrusIndeterminate
{
color: #8080ff;
}
QLineEdit#HydrusIndeterminate
{
background-color: #8080ff;
}
QTextEdit#HydrusIndeterminate
{
background-color: #8080ff;
}
QPlainTextEdit#HydrusIndeterminate
{
background-color: #8080ff;
}
/* Example: This regex is invalid */
QLabel#HydrusInvalid
{
color: #ff7171;
}
QLineEdit#HydrusInvalid
{
background-color: #ff8080;
}
QTextEdit#HydrusInvalid
{
background-color: #ff8080;
}
QPlainTextEdit#HydrusInvalid
{
background-color: #ff8080;
}
/* Example: Your files are going to be deleted! */
QLabel#HydrusWarning
{
color: #ff7171;
}
QCheckBox#HydrusWarning
{
color: #ff7171;
}
/*
Buttons on dialogs
*/
QPushButton#HydrusAccept
{
color: #2ed42e;
}
QPushButton#HydrusCancel
{
color: #ff7171;
}
/*
This is the green/red button that switches 'include current tags' and similar states on/off
*/
QPushButton#HydrusOnOffButton[hydrus_on=true]
{
color: #2ed42e;
}
QPushButton#HydrusOnOffButton[hydrus_on=false]
{
color: #ff7171;
}
/*
Extra, hydev added this
*/
QLabel#HydrusHyperlink
{
qproperty-link_color: #ffa02f;
}

441
static/qss/Dark_Blue-alternate-tooltip-colour.qss Normal file
View File

@ -0,0 +1,441 @@
/*
Dark Blue: A Dark Blue theme for Hydrus Network by B1N4RYJ4N
Version..: 1.0
To achieve the intended results you must:
1. Activate dark mode
2. adjust the Qt style to Fusion
3. adjust the Qt stylesheet to Dark_Blue
4. adjust the current colourset under files > options > colors > current colourset to darkmode
5. adjust your color values under files > options > colors > darkmode like so:
thumbnail background normal..: #1e1e1e
thumbnail background selected: #007acc
thumbnail border normal......: #569cd6
thumbnail border selected....: #cccccc
thumbnail grid background....: #1e1e1e
autocomplete background......: #536267
media viewer background......: #1e1e1e
media viewer text............: #708090
tag box background...........: #1e1e1e
6. adjust your tag presentation color values under files > options > tag presentation > (On thumbnail top, On thumbnail bottom-right, On media viewer top) like so:
background colour............: #007acc
text colour..................: #ffffff
*/
/*
___ _
/ _ \___ _ __ ___ _ __ __ _| |
/ /_\/ _ \ '_ \ / _ \ '__/ _` | |
/ /_\\ __/ | | | __/ | | (_| | |
\____/\___|_| |_|\___|_| \__,_|_|
*/
QAbstractItemView {
background-color: #252526;
}
/*
____ __ __ _ _ _
/___ \/ / /\ \ (_) __| | __ _ ___| |_
// / /\ \/ \/ / |/ _` |/ _` |/ _ \ __|
/ \_/ / \ /\ /| | (_| | (_| | __/ |_
\___,_\ \/ \/ |_|\__,_|\__, |\___|\__|
|___/
*/
QWidget {
color: #CCCCCC;
background-color: #252526;
alternate-background-color: #252526;
}
QWidget::disabled {
background-color: #252526;
}
QWidget::item::selected {
color: #FFF;
background-color: #569cd6;
}
QWidget::item:hover {
color: #FFF;
background-color: #569cd6;
}
/*
____ _____ _ _____ _
/___ \/__ \___ ___ | /__ (_)_ __
// / / / /\/ _ \ / _ \| | / /\/ | '_ \
/ \_/ / / / | (_) | (_) | |/ / | | |_) |
\___,_\ \/ \___/ \___/|_|\/ |_| .__/
|_|
*/
QToolTip {
border: 1px solid black;
background-color: #CCCCCC;
padding: 1px;
}
/*
____
/___ \/\/\ ___ _ __ _ _
// / / \ / _ \ '_ \| | | |
/ \_/ / /\/\ \ __/ | | | |_| |
\___,_\/ \/\___|_| |_|\__,_|
*/
QMenu {
color: #CCCCCC;
background: #252526;
}
QMenu::item {
padding: 2px 20px 2px 20px;
}
QMenu::item:selected {
color: #FFF;
background: #569cd6;
}
/*
____ ___
/___ \/\/\ ___ _ __ _ _ / __\ __ _ _ __
// / / \ / _ \ '_ \| | | |/__\/// _` | '__|
/ \_/ / /\/\ \ __/ | | | |_| / \/ \ (_| | |
\___,_\/ \/\___|_| |_|\__,_\_____/\__,_|_|
*/
QMenuBar::item {
background: transparent;
}
QMenuBar::item:selected {
color: #FFF;
background: #569cd6;
}
/*
____ ___ _ ___ _ _
/___ \/ _ \_ _ ___| |__ / __\_ _| |_| |_ ___ _ __
// / / /_)/ | | / __| '_ \ /__\// | | | __| __/ _ \| '_ \
/ \_/ / ___/| |_| \__ \ | | / \/ \ |_| | |_| || (_) | | | |
\___,_\/ \__,_|___/_| |_\_____/\__,_|\__|\__\___/|_| |_|
*/
QPushButton {
color: #CCCCCC;
background-color: #252526;
}
QPushButton::hover {
color: #FFF;
background-color: #252526;
}
QPushButton#HydrusAccept {
color: #50fa7b;
}
QPushButton#HydrusCancel {
color: #ff5555;
}
QPushButton#HydrusOnOffButton[hydrus_on=true] {
color: #50fa7b;
}
QPushButton#HydrusOnOffButton[hydrus_on=false] {
color: #ff5555;
}
/*
____ _____ _ ___
/___ \/__ \__ _| |__ / __\ __ _ _ __
// / / / /\/ _` | '_ \ /__\/// _` | '__|
/ \_/ / / / | (_| | |_) / \/ \ (_| | |
\___,_\ \/ \__,_|_.__/\_____/\__,_|_|
*/
QTabBar::tab {
color: #8be9fd;
background-color: #1a1a1a;
padding-left: 10px;
padding-right: 10px;
padding-top: 3px;
padding-bottom: 2px;
}
QTabBar::tab:last {
border-top-right-radius: 3px;
}
QTabBar::tab:selected {
color: #FFF;
background-color: #007ACC;
}
QTabBar::tab:hover:!selected {
color: #FFF;
background-color: #007ACC;
}
/*
____ __ _ __ _ _ _
/___ \/ /(_)_ __ ___ /__\_| (_) |_
// / / / | | '_ \ / _ \/_\/ _` | | __|
/ \_/ / /__| | | | | __//_| (_| | | |_
\___,_\____/_|_| |_|\___\__/\__,_|_|\__|
*/
QLineEdit {
border: 1px solid #569cd6;
border-radius: 1px;
background-color: #383B3D;
padding: 1px;
}
QLineEdit:focus{
color: #FFF;
border: 1px solid #CCC;
}
/*
____ ___ ___
/___ \/ _ \_ __ ___ __ _ _ __ ___ ___ ___ / __\ __ _ _ __
// / / /_)/ '__/ _ \ / _` | '__/ _ \/ __/ __| /__\/// _` | '__|
/ \_/ / ___/| | | (_) | (_| | | | __/\__ \__ \/ \/ \ (_| | |
\___,_\/ |_| \___/ \__, |_| \___||___/___/\_____/\__,_|_|
|___/
*/
QProgressBar {
color: #FFF;
border: 1px solid #569cd6;
text-align: center;
padding: 1px;
border-radius: 0px;
background-color: #383B3D;
width: 15px;
}
QProgressBar::chunk {
color: #FFF;
background: QLinearGradient( x1: 0, y1: 0, x2: 1, y2: 0,
stop: 0 #78d,
stop: 0.4999 #46a,
stop: 0.5 #45a,
stop: 1 #238 );
border-radius: 0px;
border: 0px;
}
/*
____ _ _
/___ \/\ /\___ __ _ __| | ___ _ __/\ /(_) _____ __
// / / /_/ / _ \/ _` |/ _` |/ _ \ '__\ \ / / |/ _ \ \ /\ / /
/ \_/ / __ / __/ (_| | (_| | __/ | \ V /| | __/\ V V /
\___,_\/ /_/ \___|\__,_|\__,_|\___|_| \_/ |_|\___| \_/\_/
*/
QHeaderView::section {
background-color: #007ACC;
color: #f8f8f2;
padding-left: 4px;
border: 1px solid #569cd6;
}
/*
____ __ _ _ ___
/___ \/ _\ ___ _ __ ___ | | | / __\ __ _ _ __
// / /\ \ / __| '__/ _ \| | |/__\/// _` | '__|
/ \_/ / _\ \ (__| | | (_) | | / \/ \ (_| | |
\___,_\ \__/\___|_| \___/|_|_\_____/\__,_|_|
From Quassel Wiki: http://sprunge.us/iZGB
*/
QScrollBar {
background: #1A1A1A;
margin: 0;
}
QScrollBar:hover {
background: #1A1A1A;
}
QScrollBar:vertical {
width: 8px;
}
QScrollBar:horizontal {
height: 8px;
}
QScrollBar::handle {
padding: 0;
margin: 2px;
border-radius: 2px;
border: 2px solid #569cd6;
background: #1E1E1E;
}
QScrollBar::handle:vertical {
min-height: 20px;
min-width: 0px;
}
QScrollBar::handle:horizontal {
min-width: 20px;
min-height: 0px;
}
QScrollBar::handle:hover {
border-color: #007ACC;
background: #1E1E1E;
}
QScrollBar::handle:pressed {
background: #1E1E1E;
border-color: #007ACC;
}
QScrollBar::add-line , QScrollBar::sub-line {
height: 0px;
border: 0px;
}
QScrollBar::up-arrow, QScrollBar::down-arrow {
border: 0px;
width: 0px;
height: 0px;
}
QScrollBar::add-page, QScrollBar::sub-page {
background: none;
}
/*
____ _____ _ __ _ _ _
/___ \/__ \_____ _| |_ /__\_| (_) |_
// / / / /\/ _ \ \/ / __|/_\/ _` | | __|
/ \_/ / / / | __/> <| |_//_| (_| | | |_
\___,_\ \/ \___/_/\_\\__\__/\__,_|_|\__|
*/
QTextEdit {
background-color: #383B3D;
}
QTextEdit#HydrusValid {
background-color: #80ff80;
}
QTextEdit#HydrusIndeterminate {
background-color: #8080ff;
}
QTextEdit#HydrusInvalid {
background-color: #ff8080;
}
/*
____ ___ _ _ _____ _ __ _ _ _
/___ \/ _ \ | __ _(_)_ __/__ \_____ _| |_ /__\_| (_) |_
// / / /_)/ |/ _` | | '_ \ / /\/ _ \ \/ / __|/_\/ _` | | __|
/ \_/ / ___/| | (_| | | | | / / | __/> <| |_//_| (_| | | |_
\___,_\/ |_|\__,_|_|_| |_\/ \___/_/\_\\__\__/\__,_|_|\__|
*/
QPlainTextEdit {
background-color: #383B3D;
}
/*
____ __ _ _
/___ \/ / __ _| |__ ___| |
// / / / / _` | '_ \ / _ \ |
/ \_/ / /__| (_| | |_) | __/ |
\___,_\____/\__,_|_.__/ \___|_|
*/
QLabel#HydrusValid {
color: #50fa7b;
}
QLabel#HydrusIndeterminate {
color: #8080ff;
}
QLabel#HydrusInvalid {
color: #ff5555;
}
QLabel#HydrusWarning {
color: #ff5555;
}
/*
____ __ _ __ _ _ _
/___ \/ /(_)_ __ ___ /__\_| (_) |_
// / / / | | '_ \ / _ \/_\/ _` | | __|
/ \_/ / /__| | | | | __//_| (_| | | |_
\___,_\____/_|_| |_|\___\__/\__,_|_|\__|
*/
QLineEdit#HydrusValid {
background-color: #80ff80;
}
QLineEdit#HydrusIndeterminateValid {
background-color: #8080ff;
}
QLineEdit#HydrusInvalid {
background-color: #ff8080;
}
/*
____ ___ _ _ ___
/___ \/ __\ |__ ___ ___| | __ / __\ _____ __
// / / / | '_ \ / _ \/ __| |/ //__\/// _ \ \/ /
/ \_/ / /___| | | | __/ (__| </ \/ \ (_) > <
\___,_\____/|_| |_|\___|\___|_|\_\_____/\___/_/\_\
*/
QCheckBox#HydrusWarning {
color: #ff5555;
}
/*
Extra, hydev added this
*/
QLabel#HydrusHyperlink
{
qproperty-link_color: #8be9fd;
}

397
static/qss/DarkerDuck_(darkorange)-alternate-tooltip-colour.qss Normal file
View File

@ -0,0 +1,397 @@
/* CutieDuck Hydrus.qss based on QTOrange */
QToolTip
{
border: 1px solid black;
background-color: white;
padding: 1px;
}
QWidget
{
color: white;
background-color: #323232;
alternate-background-color: #323232;
}
QWidget:disabled
{
background-color: #404040;
}
QWidget:item:hover
{
background-color: #d7801a;
color: black;
}
QWidget:item:selected
{
background-color: #ffa02f;
}
QMenuBar::item
{
background: transparent;
}
QMenuBar::item:selected
{
background: transparent;
border: 1px solid #ffa02f;
}
QMenu
{
border: 1px solid #000;
}
QMenu::item
{
padding: 2px 20px 2px 20px;
}
QMenu::item:selected
{
color: black;
}
QAbstractItemView
{
background-color: #4d4d4d;
}
QLineEdit
{
color: white;
background-color: #4d4d4d;
padding: 1px;
border-style: solid;
border: 1px solid #1e1e1e;
}
QPushButton
{
color: white;
background-color: #000000;
border-width: 1px;
border-color: #bcbcbc;
border-style: solid;
padding: 3px;
padding-left: 5px;
padding-right: 5px;
}
QPushButton:pressed
{
background-color: #323232;
}
QComboBox
{
selection-background-color: #ffa02f;
background-color: #565656;
border-style: solid;
border: 1.0px solid #1e1e1e;
}
QComboBox:hover,QPushButton:hover
{
border: 1px solid #d7801a;
}
QComboBox:on
{
padding-top: 3px;
padding-left: 4px;
background-color: #2d2d2d;
selection-background-color: #ffa02f;
}
QComboBox QAbstractItemView
{
border: 1px solid darkgray;
selection-background-color: #ffa02f;
}
QComboBox::drop-down
{
subcontrol-origin: padding;
subcontrol-position: top right;
width: 15px;
border-left-width: 0px;
border-left-color: darkgray;
border-left-style: solid; /* just a single line */
}
QComboBox::down-arrow
{
image: url(:/down_arrow.png);
}
QGroupBox:focus
{
border: 1px solid #d7801a;
}
QTextEdit:focus
{
border: 1px solid #ffa02f;
}
QScrollBar:horizontal {
border: 1px solid #222222;
background: #121212;
height: 7px;
margin: 0px 16px 0 16px;
}
QScrollBar::handle:horizontal
{
background: #ffa02f;
min-height: 20px;
border-radius: 2px;
}
QScrollBar::add-line:horizontal {
border: 1px solid #1b1b19;
border-radius: 2px;
background: #d7801a;
width: 14px;
subcontrol-position: right;
subcontrol-origin: margin;
}
QScrollBar::sub-line:horizontal {
border: 1px solid #1b1b19;
border-radius: 2px;
background: #d7801a;
width: 14px;
subcontrol-position: left;
subcontrol-origin: margin;
}
QScrollBar::right-arrow:horizontal, QScrollBar::left-arrow:horizontal
{
border: 1px solid black;
width: 1px;
height: 1px;
background: white;
}
QScrollBar::add-page:horizontal, QScrollBar::sub-page:horizontal
{
background: none;
}
QScrollBar:vertical
{
background: #484848;
width: 7px;
margin: 16px 0 16px 0;
border: 1px solid #222222;
}
QScrollBar::handle:vertical
{
background: #ffa02f;
min-height: 20px;
}
QScrollBar::add-line:vertical
{
border: 1px solid #1b1b19;
background: #ffa02f;
height: 14px;
subcontrol-position: bottom;
subcontrol-origin: margin;
}
QScrollBar::sub-line:vertical
{
border: 1px solid #1b1b19;
background: #ffa02f;
height: 14px;
subcontrol-position: top;
subcontrol-origin: margin;
}
QScrollBar::up-arrow:vertical, QScrollBar::down-arrow:vertical
{
border: 1px solid black;
width: 1px;
height: 1px;
background: white;
}
QScrollBar::add-page:vertical, QScrollBar::sub-page:vertical
{
background: none;
}
QTextEdit
{
background-color: #242424;
}
QPlainTextEdit
{
background-color: #242424;
}
QHeaderView::section
{
background-color: #616161;
color: white;
padding-left: 4px;
border: 1px solid #6c6c6c;
}
QProgressBar
{
border: 1px solid grey;
text-align: center;
}
QProgressBar::chunk
{
background-color: #d7801a;
}
QTabBar::tab {
color: #b1b1b1;
background-color: #323232;
padding-left: 10px;
padding-right: 10px;
padding-top: 3px;
padding-bottom: 2px;
}
QTabWidget::pane {
border: 1px solid #444;
top: 1px;
}
QTabBar::tab:last
{
margin-right: 0; /* the last selected tab has nothing to overlap with on the right */
border-top-right-radius: 3px;
}
QTabBar::tab:first:!selected
{
margin-left: 0px; /* the last selected tab has nothing to overlap with on the right */
border-top-left-radius: 3px;
}
QTabBar::tab:selected
{
color: black;
border-bottom-style: solid;
background-color: #ffa02f;
}
QTabBar::tab:hover:!selected
{
color: black;
background-color: #d7801a;
}
/* Hydev gonking in here with some late entries */
/* Example: This regex is valid */
QLabel#HydrusValid
{
color: #2ed42e;
}
QLineEdit#HydrusValid
{
background-color: #80ff80;
}
QTextEdit#HydrusValid
{
background-color: #80ff80;
}
QPlainTextEdit#HydrusValid
{
background-color: #80ff80;
}
/* Duplicates 'middle' text colour */
QLabel#HydrusIndeterminate
{
color: #8080ff;
}
QLineEdit#HydrusIndeterminate
{
background-color: #8080ff;
}
QTextEdit#HydrusIndeterminate
{
background-color: #8080ff;
}
QPlainTextEdit#HydrusIndeterminate
{
background-color: #8080ff;
}
/* Example: This regex is invalid */
QLabel#HydrusInvalid
{
color: #ff7171;
}
QLineEdit#HydrusInvalid
{
background-color: #ff8080;
}
QTextEdit#HydrusInvalid
{
background-color: #ff8080;
}
QPlainTextEdit#HydrusInvalid
{
background-color: #ff8080;
}
/*
Buttons on dialogs
*/
QPushButton#HydrusAccept
{
color: #2ed42e;
}
QPushButton#HydrusCancel
{
color: #ff7171;
}
/*
Extra, hydev added this
*/
QLabel#HydrusHyperlink
{
qproperty-link_color: #ffa02f;
}

489
static/qss/Hydracula-alternate-tooltip-colour.qss Normal file
View File

@ -0,0 +1,489 @@
/* Hydracula: Dracula theme for Hydrus by deDUCKted aka purple_azurite*/
/* To achieve the intended results you must: 1. Activate dark mode, 2. adjust your color values under files > options > colors > darkmode like so: */
/* thumbnail grid background: #282a36
/* media viewer background: #282a36
/*color: usually refers to foreground color */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
/* Tooltips that appear on mouse hover */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
QToolTip
{
border: 1px solid black;
background-color: #f8f8f2;
padding: 1px;
}
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
/* Widgets are UI elements that are appended on the main GUI. For instance, */
/* the tag-selection submenu on the left is a widget. */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
QWidget
{
color: #f8f8f2;
background-color: #282a36;
alternate-background-color: #44475a;
}
/*DEV: Could you create an entirely new class for the drop-down menus (ie. for the application menu) so they don't share with Qwidget:item:selected?*/
QWidget:disabled
{
background-color: #44475a;
}
QWidget:item:hover
{
background-color: #50fa7b;
color: black;
}
QWidget:item:selected
{
color: black;
background-color: #50fa7b;
}
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
/* The application menu stuff (file, undo, page etc.) */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
QMenuBar::item
{
background: transparent;
}
QMenuBar::item:selected
{
color: #50fa7b;
background: transparent;
}
QMenu
{
color: #f8f8f2;
background: #44475a;
border: 1px solid #000;
}
QMenu::item
{
padding: 2px 20px 2px 20px;
}
QMenu::item:selected
{
color: black;
}
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
/* This is hard to explain. Just trial and error. Mostly stuff appended on */
/* top of widgets. */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
QAbstractItemView
{
background-color: #282a36;
}
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
/* Text input fields */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
QLineEdit
{
color: #f8f8f2;
background-color: #6272a4;
padding: 1px;
border-style: solid;
border: 1px solid #000000;
}
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
/* BUTTONS! */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
QPushButton
{
color: #f8f8f2;
background-color: transparent;
border-width: 1px;
border-color: #44475a;
border-style: solid;
padding: 3px;
font-size: 12px;
padding-left: 5px;
padding-right: 5px;
}
QPushButton:hover
{
border-color: #50fa7b;
}
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
/* May the Omnissiah guide your hand. */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
QComboBox
{
selection-background-color: #bd93f9;
background-color: #6272a4;
border-style: solid;
border: 1.0px solid #000000;
}
QComboBox:hover,QPushButton:hover
{
border: 1px solid #50fa7b;
}
QComboBox:on
{
padding-top: 3px;
padding-left: 4px;
background-color: #44475a;
selection-background-color: #bd93f9;
}
QComboBox QAbstractItemView
{
border: 1px solid darkgray;
selection-background-color: #bd93f9;
}
QComboBox::drop-down
{
subcontrol-origin: padding;
subcontrol-position: top right;
width: 15px;
border-left-width: 0px;
border-left-color: darkgray;
border-left-style: solid; /* just a single line */
}
QComboBox::down-arrow
{
image: url(:/down_arrow.png);
}
QGroupBox:focus
{
border: 1px solid #50fa7b;
}
QTextEdit:focus
{
border: 1px solid #44475a;
}
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
/* Scrollbar */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
QScrollBar:horizontal {
border: 1px solid #000000;
background: black;
height: 7px;
margin: 0px 16px 0 16px;
}
QScrollBar::handle:horizontal
{
background: #bd93f9;
min-height: 20px;
border-radius: 2px;
}
QScrollBar::add-line:horizontal {
border: 1px solid #000000;
border-radius: 2px;
background: #50fa7b;
width: 14px;
subcontrol-position: right;
subcontrol-origin: margin;
}
QScrollBar::sub-line:horizontal {
border: 1px solid #000000;
border-radius: 2px;
background: #50fa7b;
width: 14px;
subcontrol-position: left;
subcontrol-origin: margin;
}
QScrollBar::right-arrow:horizontal, QScrollBar::left-arrow:horizontal
{
border: 1px solid black;
width: 1px;
height: 1px;
background: #f8f8f2;
}
QScrollBar::add-page:horizontal, QScrollBar::sub-page:horizontal
{
background: none;
}
QScrollBar:vertical
{
background: transparent;
width: 7px;
margin: 16px 0 16px 0;
border: 1px solid #000000;
}
QScrollBar::handle:vertical
{
background: #bd93f9;
min-height: 20px;
}
QScrollBar::add-line:vertical
{
border: 1px solid #000000;
background: #bd93f9;
height: 14px;
subcontrol-position: bottom;
subcontrol-origin: margin;
}
QScrollBar::sub-line:vertical
{
border: 1px solid #000000;
background: #bd93f9;
height: 14px;
subcontrol-position: top;
subcontrol-origin: margin;
}
QScrollBar::up-arrow:vertical, QScrollBar::down-arrow:vertical
{
border: 1px solid black;
width: 1px;
height: 1px;
background: #f8f8f2;
}
QScrollBar::add-page:vertical, QScrollBar::sub-page:vertical
{
background: none;
}
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
/* Progressbar for downloaders etc. */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
QTextEdit
{
background-color: #000000;
}
QPlainTextEdit
{
background-color: #000000;
}
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
/* Headers */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
QHeaderView::section
{
background-color: #6272a4;
color: #f8f8f2;
padding-left: 4px;
border: 1px solid #6272a4;
}
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
/* Progressbar for downloaders etc. */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
QProgressBar
{
color: black;
border: 1px solid grey;
text-align: center;
}
QProgressBar::chunk
{
background-color: #ffb86c;
}
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
/* Tabs menu (ie. pages) */
/*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*/
QTabBar::tab {
color: #8be9fd;
background-color: #282a36;
padding-left: 10px;
padding-right: 10px;
padding-top: 3px;
padding-bottom: 2px;
}
QTabWidget::pane {
border: 1px solid #444;
top: 1px;
}
QTabBar::tab:last
{
margin-right: 0; /* the last selected tab has nothing to overlap with on the right */
border-top-right-radius: 3px;
}
QTabBar::tab:first:!selected
{
margin-left: 0px; /* the last selected tab has nothing to overlap with on the right */
border-top-left-radius: 3px;
}
QTabBar::tab:selected
{
color: #50fa7b;
border-bottom-style: solid;
background-color: #282a36;
}
QTabBar::tab:hover:!selected
{
color: #bd93f9;
background-color: #282a36;
}
/*
DUCK
*/
/*
Default QSS for hydrus. This is prepended to any stylesheet loaded in hydrus.
Copying these entries in your own stylesheets should override these settings.
This will get more work in future.
*/
/*
Here are some text and background colours
*/
/* Example: This regex is valid */
QLabel#HydrusValid
{
color: #50fa7b;
}
QLineEdit#HydrusValid
{
background-color: #80ff80;
}
QTextEdit#HydrusValid
{
background-color: #80ff80;
}
QPlainTextEdit#HydrusValid
{
background-color: #80ff80;
}
/* Duplicates 'middle' text colour */
QLabel#HydrusIndeterminate
{
color: #8080ff;
}
QLineEdit#HydrusIndeterminate
{
background-color: #8080ff;
}
QTextEdit#HydrusIndeterminate
{
background-color: #8080ff;
}
QPlainTextEdit#HydrusIndeterminate
{
background-color: #8080ff;
}
/* Example: This regex is invalid */
QLabel#HydrusInvalid
{
color: #ff5555;
}
QLineEdit#HydrusInvalid
{
background-color: #ff8080;
}
QTextEdit#HydrusInvalid
{
background-color: #ff8080;
}
QPlainTextEdit#HydrusInvalid
{
background-color: #ff8080;
}
/* Example: Your files are going to be deleted! */
QLabel#HydrusWarning
{
color: #ff5555;
}
QCheckBox#HydrusWarning
{
color: #ff5555;
}
/*
Buttons on dialogs
*/
QPushButton#HydrusAccept
{
color: #50fa7b;
}
QPushButton#HydrusCancel
{
color: #ff5555;
}
/*
This is the green/red button that switches 'include current tags' and similar states on/off
*/
QPushButton#HydrusOnOffButton[hydrus_on=true]
{
color: #50fa7b;
}
QPushButton#HydrusOnOffButton[hydrus_on=false]
{
color: #ff5555;
}
/* You are permitted to use this theme however you want for the low, low price of your soul. :3 */
/*
Extra, hydev added this
*/
QLabel#HydrusHyperlink
{
qproperty-link_color: #50fa7b;
}

439
static/qss/OledBlack-alternate-tooltip-colour.qss Normal file
View File

@ -0,0 +1,439 @@
/* OledBlack.qss based on CutieDuck Hydrus.qss */
QToolTip
{
border: 1px solid black;
background-color: white;
padding: 1px;
}
QWidget
{
color: white;
background-color: #101010;
alternate-background-color: #101010;
}
QWidget:disabled
{
background-color: #101010;
}
QWidget:item:hover
{
background-color: #FF2800;
color: black;
}
QWidget:item:selected
{
background-color: #9B1800;
}
QMenuBar::item
{
background: transparent;
}
QMenuBar::item:selected
{
background: transparent;
border: 1px solid #9B1800;
}
QMenu
{
border: 1px solid #000;
}
QMenu::item
{
padding: 2px 20px 2px 20px;
}
QMenu::item:selected
{
color: black;
}
QAbstractItemView
{
background-color: #242424;
}
QLineEdit
{
color: white;
background-color: #242424;
padding: 1px;
border-style: solid;
border: 1px solid #000000;
}
QPushButton
{
color: white;
background-color: #242424;
border-width: 1px;
border-color: #000000;
border-style: solid;
padding: 3px;
padding-left: 5px;
padding-right: 5px;
}
QPushButton:pressed
{
background-color: #323232;
}
QComboBox
{
selection-background-color: #9B1800;
background-color: #242424;
border-style: solid;
border: 1.0px solid #000000;
}
QComboBox:hover,QPushButton:hover
{
border: 1px solid #FF2800;
}
QComboBox:on
{
padding-top: 3px;
padding-left: 4px;
background-color: #2d2d2d;
selection-background-color: #9B1800;
}
QComboBox QAbstractItemView
{
border: 1px solid darkgray;
selection-background-color: #9B1800;
}
QComboBox::drop-down
{
subcontrol-origin: padding;
subcontrol-position: top right;
width: 15px;
border-left-width: 0px;
border-left-color: darkgray;
border-left-style: solid; /* just a single line */
}
QComboBox::down-arrow
{
image: url(:/down_arrow.png);
}
QGroupBox:focus
{
border: 1px solid #FF2800;
}
QTextEdit:focus
{
border: 1px solid #2d2d2d;
}
QScrollBar:horizontal {
border: 1px solid #000000;
background: black;
height: 7px;
margin: 0px 16px 0 16px;
}
QScrollBar::handle:horizontal
{
background: #9B1800;
min-height: 20px;
border-radius: 2px;
}
QScrollBar::add-line:horizontal {
border: 1px solid #000000;
border-radius: 2px;
background: #FF2800;
width: 14px;
subcontrol-position: right;
subcontrol-origin: margin;
}
QScrollBar::sub-line:horizontal {
border: 1px solid #000000;
border-radius: 2px;
background: #FF2800;
width: 14px;
subcontrol-position: left;
subcontrol-origin: margin;
}
QScrollBar::right-arrow:horizontal, QScrollBar::left-arrow:horizontal
{
border: 1px solid black;
width: 1px;
height: 1px;
background: white;
}
QScrollBar::add-page:horizontal, QScrollBar::sub-page:horizontal
{
background: none;
}
QScrollBar:vertical
{
background: #484848;
width: 7px;
margin: 16px 0 16px 0;
border: 1px solid #000000;
}
QScrollBar::handle:vertical
{
background: #9B1800;
min-height: 20px;
}
QScrollBar::add-line:vertical
{
border: 1px solid #000000;
background: #9B1800;
height: 14px;
subcontrol-position: bottom;
subcontrol-origin: margin;
}
QScrollBar::sub-line:vertical
{
border: 1px solid #000000;
background: #9B1800;
height: 14px;
subcontrol-position: top;
subcontrol-origin: margin;
}
QScrollBar::up-arrow:vertical, QScrollBar::down-arrow:vertical
{
border: 1px solid black;
width: 1px;
height: 1px;
background: white;
}
QScrollBar::add-page:vertical, QScrollBar::sub-page:vertical
{
background: none;
}
QTextEdit
{
background-color: #000000;
}
QPlainTextEdit
{
background-color: #000000;
}
QHeaderView::section
{
background-color: #616161;
color: white;
padding-left: 4px;
border: 1px solid #6c6c6c;
}
QProgressBar
{
border: 1px solid grey;
text-align: center;
}
QProgressBar::chunk
{
background-color: #FF2800;
}
QTabBar::tab {
color: #b1b1b1;
background-color: #323232;
padding-left: 10px;
padding-right: 10px;
padding-top: 3px;
padding-bottom: 2px;
}
QTabWidget::pane {
border: 1px solid #444;
top: 1px;
}
QTabBar::tab:last
{
margin-right: 0; /* the last selected tab has nothing to overlap with on the right */
border-top-right-radius: 3px;
}
QTabBar::tab:first:!selected
{
margin-left: 0px; /* the last selected tab has nothing to overlap with on the right */
border-top-left-radius: 3px;
}
QTabBar::tab:selected
{
color: black;
border-bottom-style: solid;
background-color: #9B1800;
}
QTabBar::tab:hover:!selected
{
color: black;
background-color: #FF2800;
}
/*
DUCK
*/
/*
Default QSS for hydrus. This is prepended to any stylesheet loaded in hydrus.
Copying these entries in your own stylesheets should override these settings.
This will get more work in future.
*/
/*
Here are some text and background colours
*/
/* Example: This regex is valid */
QLabel#HydrusValid
{
color: #2ed42e;
}
QLineEdit#HydrusValid
{
background-color: #80ff80;
}
QTextEdit#HydrusValid
{
background-color: #80ff80;
}
QPlainTextEdit#HydrusValid
{
background-color: #80ff80;
}
/* Duplicates 'middle' text colour */
QLabel#HydrusIndeterminate
{
color: #8080ff;
}
QLineEdit#HydrusIndeterminate
{
background-color: #8080ff;
}
QTextEdit#HydrusIndeterminate
{
background-color: #8080ff;
}
QPlainTextEdit#HydrusIndeterminate
{
background-color: #8080ff;
}
/* Example: This regex is invalid */
QLabel#HydrusInvalid
{
color: #ff7171;
}
QLineEdit#HydrusInvalid
{
background-color: #ff8080;
}
QTextEdit#HydrusInvalid
{
background-color: #ff8080;
}
QPlainTextEdit#HydrusInvalid
{
background-color: #ff8080;
}
/* Example: Your files are going to be deleted! */
QLabel#HydrusWarning
{
color: #ff7171;
}
QCheckBox#HydrusWarning
{
color: #ff7171;
}
/*
Buttons on dialogs
*/
QPushButton#HydrusAccept
{
color: #2ed42e;
}
QPushButton#HydrusCancel
{
color: #ff7171;
}
/*
This is the green/red button that switches 'include current tags' and similar states on/off
*/
QPushButton#HydrusOnOffButton[hydrus_on=true]
{
color: #2ed42e;
}
QPushButton#HydrusOnOffButton[hydrus_on=false]
{
color: #ff7171;
}
/*
Extra, hydev added this
*/
QLabel#HydrusHyperlink
{
qproperty-link_color: white;
}