stash/pkg/database/migrations
gitgiggety 3e526a49a4
Add indexes for path and checksum to images (#1740)
* Add indexes for path and checksum to images

The scenes table has unique indexes/constraints on path and checksum
colums. The images table doesn't, which doesn't really make sense, as
scanning uses these colums extensively which warrents an index, and both
should be unique as well.

Adding these indexes thus heavily improves the scanning tasks
performance. On a database containing 4700 images a (re)scan of those
4700 files, which thus shouldn't do anything, took 1.2 seconds, with the
indexes added this only takes 0.4 seconds. Taking the same test on a
generated database containing 4M images + the actual 4700 images took 26
minutes for a rescan, and with the index existing also only takes 0.4
seconds.

* Add images.checksum unique constraint in code with fallback

Work around the issue where in some cases duplicate images (/checksums
on images) might exist. This as discussed in #1740 by creating the index
on startup and in case of an error logging the duplicates. This so the
users where this scenario exists can correct the database (by searching
on the logged checksum(s) and removing the duplicates) and after a
restart the unique index / constraint will still be created. In case
when creating the unique index fails a "normal" / non-unique index is
created as surrogate so the user will still get the performance benefit
(for example during scanning) without being forced to remove the
duplicates and restart beforehand. This surrogate is also automatically
cleaned up after the unique index is succesfully created.
2021-09-21 11:48:52 +10:00
..
1_initial.down.sql
1_initial.up.sql
2_cover_image.up.sql Added support for import / export of scene cover images 2019-12-31 14:38:49 -08:00
3_o_counter.up.sql Add O-counter (#334) 2020-02-02 19:17:28 -05:00
4_movie.up.sql Movies Section (#338) 2020-03-10 14:28:15 +11:00
5_performer_gender.down.sql Add gender support for performer (#371) 2020-04-01 09:36:38 +11:00
5_performer_gender.up.sql Add gender support for performer (#371) 2020-04-01 09:36:38 +11:00
6_scenes_format.up.sql Add detection of container/video_codec/audio_codec compatibility for live file streaming or transcoding (#384) 2020-04-10 08:38:34 +10:00
7_performer_optimization.up.sql Add slim endpoints for entities to speed up filters (#460) 2020-04-19 12:03:51 +10:00
8_movie_fix.up.sql Add Studio to movie and fix movie schema (#458) 2020-04-22 11:22:14 +10:00
9_studios_parent_studio.up.sql Parent studios (#595) 2020-06-15 21:34:39 +10:00
10_image_tables.up.sql Move image blobs into separate tables (#618) 2020-06-23 09:19:19 +10:00
11_tag_image.up.sql Add grid view, image to tag (#641) 2020-07-07 10:35:43 +10:00
12_oshash.up.sql Add oshash support (#667) 2020-08-06 11:21:14 +10:00
13_images.up.sql Images section (#813) 2020-10-13 10:12:46 +11:00
14_stash_box_ids.up.sql Stash-box tagger integration (#454) 2020-10-24 14:31:39 +11:00
15_file_mod_time.up.sql Handle zip file modification (#877) 2020-11-05 10:26:51 +11:00
16_organized_flag.up.sql Organised flag (#988) 2020-12-18 08:06:49 +11:00
17_reset_scene_size.up.sql Fix integer overflow for scene size on 32bit systems (#994) 2020-12-22 10:29:53 +11:00
18_scene_galleries.up.sql Decouple galleries from scenes (#1057) 2021-02-02 07:56:54 +11:00
19_performer_tags.up.sql Performer tags (#1132) 2021-03-10 12:25:51 +11:00
20_phash.up.sql Add phash generation and dupe checking (#1158) 2021-04-12 09:04:40 +10:00
21_performers_studios_details.up.sql added details, deathdate, hair color, weight to performers and added details to studios (#1274) 2021-04-16 16:06:35 +10:00
22_performers_studios_rating.up.sql Added rating to performers and studios (#1308) 2021-04-26 13:48:32 +10:00
23_scenes_interactive.up.sql Add Handy / Funscript support (#1377) 2021-05-24 13:34:28 +10:00
24_tag_aliases.up.sql Tag aliases (#1412) 2021-05-26 14:36:05 +10:00
25_saved_filters.up.sql Saved filters (#1474) 2021-06-16 14:53:32 +10:00
26_tag_hierarchy.up.sql Tag hierarchy (#1519) 2021-09-09 14:58:43 +10:00
27_studio_aliases.up.sql Studio aliases (#1660) 2021-09-09 18:13:42 +10:00
28_images_indexes.up.sql Add indexes for path and checksum to images (#1740) 2021-09-21 11:48:52 +10:00