add googlebooks virtual urls to disallow

after we added author links, robots started creating db entries by
following links in search page
pull/1/head
eric 2016-06-28 14:56:59 -04:00
parent a9152bdbe3
commit ca08c5caef
1 changed files with 1 additions and 0 deletions

View File

@ -4,6 +4,7 @@ User-agent: *
Disallow: /accounts/
Disallow: /feedback/
Disallow: /socialauth/
Disallow: /googlebooks/
{% else %}
User-agent: *
Disallow: /