Commit Graph

7095 Commits (066b81fb747a1e49940183c5da67f50340956bc1)

Author SHA1 Message Date
eric 066b81fb74 add digitalis harvest 2020-07-28 20:58:25 -04:00
eric 19d39cf4a6 add code to deal with ebooks already harvested from different source 2020-07-28 20:57:46 -04:00
eric b799b2a4c9 increase harvest limit to 500 2020-07-28 09:27:40 -04:00
eric bf73124250 deal with no head 2020-07-27 20:39:45 -04:00
eric 74005584d0 add kit.edu 2020-07-27 19:06:15 -04:00
eric a14a94aba1 fix jbe condition 2020-07-27 17:54:16 -04:00
eric e306f319ce add transcript verlag harvest 2020-07-27 17:53:59 -04:00
eric 5932bc09ed allow harvest to harvest multiple ebooks 2020-07-27 17:53:32 -04:00
eric 26e32a4738 sometimes there's no contenttype header! 2020-07-27 17:50:21 -04:00
eric 2f28b32fbf also use disposition from contenttyper 2020-07-27 17:49:04 -04:00
eric dd76c112e9 fix degruyter harvest 2020-07-27 17:47:53 -04:00
eric 15424eaf4d add error handling for doab 404s 2020-07-26 16:06:33 -04:00
eric d86ce969b8 improve handling when G doesn't return an item with same isbn 2020-07-24 13:00:08 -04:00
eric 1a8813832e bugfix 2020-07-23 15:48:11 -04:00
eric eb6ca2d570 refactor harvest.py
also don't remake ebooks
2020-07-23 10:42:34 -04:00
eric 28a49a5e11 add some providers 2020-07-22 19:28:23 -04:00
eric 79aa49a1f1 one more thing for doi 2020-07-22 19:28:02 -04:00
eric 42899559e2 enrich management command
can now harvest doab from a date  or  starting at an doab_id
2020-07-22 19:27:45 -04:00
eric e036570068 document puzzling method 2020-07-22 19:16:03 -04:00
eric 737d40593b add OBP harvest
also add support for harvesting books via post
2020-07-22 19:15:34 -04:00
eric 5882c07854 add dois from doab 2020-07-22 19:10:05 -04:00
eric 961da4f081 improved content typing
ContentTyper now
-follows head redirects
-considers content-disposition header
- checks to see if we already know format
- tries get if head not allowed (405)
2020-07-22 19:04:48 -04:00
eric 61d0c80b12 remove facebook 2020-07-20 13:29:47 -04:00
eric 7e83509174 add dump.rdb to gitignore 2020-07-20 12:03:02 -04:00
eric 0da0731662 fix pprevious 2020-07-20 11:05:38 -04:00
eric a7b8d93a18 minor optimizations 2020-07-20 11:02:37 -04:00
eric b50c79baa6 hide learn-more by default 2020-07-20 10:41:31 -04:00
eric 6114483d88 update description
we're only showing free books now
2020-07-20 10:31:52 -04:00
eric ff165fdd6e optimatize empty tests 2020-07-19 21:54:42 -04:00
eric b3e81b105b remove tablsfrom work lists 2020-07-19 21:54:11 -04:00
eric daab584336 add emit notices task 2020-07-01 18:38:25 -04:00
eric fad442b16f switch to direct auth urls for reset password
in dj111, a self-redirect was breaking the django-registration url shims, causing a bad success url.
2020-06-27 19:27:23 -04:00
eric 3ebccdbe88 update ubiquity sites 2020-06-25 14:21:37 -04:00
eric 72876831b2 wrong questionaire version 2020-06-05 17:00:30 -04:00
eric 95653e87f2 one more time 2020-06-03 17:48:12 -04:00
eric adbc15c150 update dependencies 2020-06-03 17:01:51 -04:00
eric c1f67e8b36 add amazon note 2020-06-03 15:18:11 -04:00
eric fbfeccbb60 update dependencies 2020-06-03 15:13:06 -04:00
eric 9c231bc401 no work if no title
Amazon does this to robots
2020-06-01 18:54:01 -04:00
eric df67d1d36f update sorl thumbnail 2020-04-27 16:47:46 -04:00
eric 9a411f1906 fic empty new edition
fix error when user click new edition without entering anything
2020-04-27 13:10:32 -04:00
Eric Hellman 194950e2a5
Merge pull request #867 from Gluejar/stream-api
Stream api
2020-04-06 15:14:21 -04:00
eric 8efe0b012e instantiate soup inside methods 2020-04-02 14:09:34 -04:00
eric 3bb738dbf3 Revert "Revert "maybe its the xml parser???""
This reverts commit 9d7c780488.
2020-04-02 13:47:50 -04:00
eric 5579308e4d Revert "flailing here"
This reverts commit 6d083cea83.
2020-04-02 12:42:17 -04:00
eric 9d7c780488 Revert "maybe its the xml parser???"
This reverts commit bfa957429f.
2020-04-02 12:42:10 -04:00
eric bfa957429f maybe its the xml parser??? 2020-04-02 12:34:06 -04:00
eric 6d083cea83 flailing here
Trying to isolate deployment problem
2020-04-02 10:27:08 -04:00
eric f77f8b4006 try being careful instantiating soup 2020-04-01 20:24:36 -04:00
eric f4a9697971 fix some problems 2020-04-01 17:18:37 -04:00