Commit Graph

4 Commits

Author SHA1 Message Date
WithoutPants 2b9215702e
Refactor xpath scraper code. Add fixed and map (#616)
* Refactor xpath scraper code
* Make post-process a list
* Add map post-process action
* Add fixed xpath values
* Refactor scrapers into cache
* Refactor into mapped config
* Trim test html
2020-07-21 14:06:25 +10:00
WithoutPants 92837fe1f7 Add scene metadata scraping functionality (#236)
* Add scene scraping functionality

* Adapt to changed scraper config
2019-12-15 20:35:34 -05:00
WithoutPants 50784025f2 Change scraper config to yaml (#256) 2019-12-12 14:27:44 -05:00
WithoutPants 17247060b6 Generic performer scrapers (#203)
* Generalise scraper API

* Add script performer scraper

* Fixes from testing

* Add context to scrapers and generalise

* Add scraping performer from URL

* Add error handling

* Move log to debug

* Add supported scrape types
2019-11-18 21:49:05 -05:00