added robot section; updated new_website.md with known issues

master
Gutenberg back end service account 2020-08-27 08:24:25 -04:00
parent aa1bfb4d75
commit daa9a14736
4 changed files with 70 additions and 68 deletions

View File

@ -49,4 +49,4 @@ destination: /public/vhost/g/gutenberg/dev/html/
timezone: America/New_York
encoding: utf-8 # default
incremental: TRUE # doesn't seem to work... all files are being built
keep_files: [.htaccess,cache,robots.txt,google3c00b5891470e2e3.html,catalog,browse,dirs,files,.acl.gutenwebdev]
keep_files: [.htaccess,cache,robots.txt,google3c00b5891470e2e3.html,catalog,browse,dirs,files,.acl.gutenwebdev,.acl.gutenwebprod,MIRRORS.ALL.utf8]

View File

@ -4,37 +4,40 @@
<!--<head>
<title>Cover Flow</title>
</head>-->
<!--<body><div>--><a href="/ebooks/63037" title="Die Abenteuer des Fliegers von Tsingtau: Meine Erlebnisse in drei Erdteilen" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63037/pg63037.cover.small.jpg" alt="Die Abenteuer des Fliegers von Tsingtau: Meine Erlebnisse in drei Erdteilen" title="Die Abenteuer des Fliegers von Tsingtau: Meine Erlebnisse in drei Erdteilen" draggable="false">
</div><div class="cover_title"><h5>Die Abenteuer des Fliegers von Tsingtau: Meine Erlebnisse in drei Erdteilen</h5></div></div></a>
<a href="/ebooks/63036" title="Geography of the Land" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63036/pg63036.cover.small.jpg" alt="Geography of the Land" title="Geography of the Land" draggable="false">
</div><div class="cover_title"><h5>Geography of the Land</h5></div></div></a>
<a href="/ebooks/63035" title="Eleonora Lubomirski
Historiallis-romantillinen kertoelma Kaarle X ja Kaarle XI ajoilta" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63035/pg63035.cover.small.jpg" alt="Eleonora Lubomirski
Historiallis-romantillinen kertoelma Kaarle X ja Kaarle XI ajoilta" title="Eleonora Lubomirski
Historiallis-romantillinen kertoelma Kaarle X ja Kaarle XI ajoilta" draggable="false">
</div><div class="cover_title"><h5>Eleonora Lubomirski</h5></div></div></a>
<a href="/ebooks/63034" title="Comedia do Campo volume III (Scenas do Minho)" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63034/pg63034.cover.small.jpg" alt="Comedia do Campo volume III (Scenas do Minho)" title="Comedia do Campo volume III (Scenas do Minho)" draggable="false">
</div><div class="cover_title"><h5>Comedia do Campo volume III (Scenas do Minho)</h5></div></div></a>
<a href="/ebooks/63033" title="Emanuel Swedenborg&apos;s Investigations in Natural Science and the Basis for His Statements Concerning the Functions of the Brain" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63033/pg63033.cover.small.jpg" alt="Emanuel Swedenborg&apos;s Investigations in Natural Science and the Basis for His Statements Concerning the Functions of the Brain" title="Emanuel Swedenborg&apos;s Investigations in Natural Science and the Basis for His Statements Concerning the Functions of the Brain" draggable="false">
</div><div class="cover_title"><h5>Emanuel Swedenborg&apos;s Investigations in Natural Science and the Basis for</h5></div></div></a>
<a href="/ebooks/63032" title="One Against the Stars" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63032/pg63032.cover.small.jpg" alt="One Against the Stars" title="One Against the Stars" draggable="false">
</div><div class="cover_title"><h5>One Against the Stars</h5></div></div></a>
<a href="/ebooks/63031" title="Chambers&apos;s Journal of Popular Literature, Science, and Art, No. 745, April 6, 1878" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63031/pg63031.cover.small.jpg" alt="Chambers&apos;s Journal of Popular Literature, Science, and Art, No. 745, April 6, 1878" title="Chambers&apos;s Journal of Popular Literature, Science, and Art, No. 745, April 6, 1878" draggable="false">
</div><div class="cover_title"><h5>Chambers&apos;s Journal of Popular Literature, Science, and Art, No. 745, April</h5></div></div></a>
<a href="/ebooks/63030" title="Tauromaquia completa, ó sea, El arte de torear en plaza" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63030/pg63030.cover.small.jpg" alt="Tauromaquia completa, ó sea, El arte de torear en plaza" title="Tauromaquia completa, ó sea, El arte de torear en plaza" draggable="false">
</div><div class="cover_title"><h5>Tauromaquia completa, ó sea, El arte de torear en plaza</h5></div></div></a>
<a href="/ebooks/63029" title="Shaggo, the Mighty Buffalo: His Many Adventures" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63029/pg63029.cover.small.jpg" alt="Shaggo, the Mighty Buffalo: His Many Adventures" title="Shaggo, the Mighty Buffalo: His Many Adventures" draggable="false">
</div><div class="cover_title"><h5>Shaggo, the Mighty Buffalo: His Many Adventures</h5></div></div></a>
<a href="/ebooks/63028" title="Reinstern" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63028/pg63028.cover.small.jpg" alt="Reinstern" title="Reinstern" draggable="false">
</div><div class="cover_title"><h5>Reinstern</h5></div></div></a>
<!--<body><div>--><a href="/ebooks/63052" title="Knox Eat and Reduce Plan
Including choice-of-foods Chart" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63052/pg63052.cover.small.jpg" alt="Knox Eat and Reduce Plan
Including choice-of-foods Chart" title="Knox Eat and Reduce Plan
Including choice-of-foods Chart" draggable="false">
</div><div class="cover_title"><h5>Knox Eat and Reduce Plan</h5></div></div></a>
<a href="/ebooks/63051" title="Oliver Cromwell" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63051/pg63051.cover.small.jpg" alt="Oliver Cromwell" title="Oliver Cromwell" draggable="false">
</div><div class="cover_title"><h5>Oliver Cromwell</h5></div></div></a>
<a href="/ebooks/63050" title="Be Polite to All" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63050/pg63050.cover.small.jpg" alt="Be Polite to All" title="Be Polite to All" draggable="false">
</div><div class="cover_title"><h5>Be Polite to All</h5></div></div></a>
<a href="/ebooks/63049" title="Up in the garret" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63049/pg63049.cover.small.jpg" alt="Up in the garret" title="Up in the garret" draggable="false">
</div><div class="cover_title"><h5>Up in the garret</h5></div></div></a>
<a href="/ebooks/63048" title="Wanderers of the Wolf-Moon" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63048/pg63048.cover.small.jpg" alt="Wanderers of the Wolf-Moon" title="Wanderers of the Wolf-Moon" draggable="false">
</div><div class="cover_title"><h5>Wanderers of the Wolf Moon</h5></div></div></a>
<a href="/ebooks/63047" title="Stories of the Bible, Volume 1
The People of the Chosen Land" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63047/pg63047.cover.small.jpg" alt="Stories of the Bible, Volume 1
The People of the Chosen Land" title="Stories of the Bible, Volume 1
The People of the Chosen Land" draggable="false">
</div><div class="cover_title"><h5>Stories of the Bible, Volume 1</h5></div></div></a>
<a href="/ebooks/63046" title="Mr. Meek—Musketeer" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63046/pg63046.cover.small.jpg" alt="Mr. Meek—Musketeer" title="Mr. Meek—Musketeer" draggable="false">
</div><div class="cover_title"><h5>Mr. Meek—Musketeer</h5></div></div></a>
<a href="/ebooks/63045" title="Aunt Olive in Bohemia" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63045/pg63045.cover.small.jpg" alt="Aunt Olive in Bohemia" title="Aunt Olive in Bohemia" draggable="false">
</div><div class="cover_title"><h5>Aunt Olive in Bohemia</h5></div></div></a>
<a href="/ebooks/63044" title="A Girl of the Plains Country" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63044/pg63044.cover.small.jpg" alt="A Girl of the Plains Country" title="A Girl of the Plains Country" draggable="false">
</div><div class="cover_title"><h5>A Girl of the Plains Country</h5></div></div></a>
<a href="/ebooks/63043" title="Valdemar Seier. Finnish" target="_top"><div class="cover_image">
<div class="cover_img"><img src="/cache/epub/63043/pg63043.cover.small.jpg" alt="Valdemar Seier. Finnish" title="Valdemar Seier. Finnish" draggable="false">
</div><div class="cover_title"><h5>Valdemar Seier. Finnish</h5></div></div></a>
<!--</div></body></html>-->

View File

@ -12,12 +12,12 @@ Sorry, but our website automatically denied you access to that page.
Maybe you have just a wrong url. Go to [www.gutenberg.org](/) to see whether the error persists.
If you get the error again, this might explain what is happening:
- Visiting from a German Internet address? As of February 27 2018, addresses in Germany are blocked. For more information, see <a href="https://block.pglaf.org/germany.shtml">block.pglaf.org/germany.shtml</a>.</li>
- Anonymized IP addresses might trigger automated anti-abuse blocks. This can happen with anonymizers, open proxies, VPNs, or TOR, and includes the Google proxies that are used by Chrome.</li>
- Don't access Project Gutenberg from hosted servers.</li>
- Don't use automated software to download lots of books. We have a limit on how fast you can go while using this site. If you surpass this limit you get blocked automatically, and eventually blocks expire automatically.</li>
- Kindle users, and people using shared IP addresses (such as some schools) might trigger a block due to too many requests coming from the same address. </li>
- If you use the RSS feed, set your update interval to 24 hours.</li>
- Visiting from a German Internet address? As of February 27 2018, addresses in Germany are blocked. For more information, see <a href="https://block.pglaf.org/germany.shtml">block.pglaf.org/germany.shtml</a>.
- Anonymized IP addresses might trigger automated anti-abuse blocks. This can happen with anonymizers, open proxies, VPNs, or TOR, and includes the Google proxies that are used by Chrome.
- Don't access Project Gutenberg from hosted servers.
- Don't use automated software to download lots of books. We have a limit on how fast you can go while using this site. If you surpass this limit you get blocked automatically, and eventually blocks expire automatically.
- Kindle users, and people using shared IP addresses (such as some schools) might trigger a block due to too many requests coming from the same address.
- If you use the RSS feed, set your update interval to 24 hours.
## What to do if I'm blocked inappropriately
If you are sure that none of the above applies to you,

View File

@ -7,9 +7,29 @@ permalink: /help/new_website.html
New Website 2020
================
The Project Gutenberg website, along with some of its back-end functionality, has been undergoing major updates. The new website is available for public beta testing. Input, fixes and suggestions are welcome.
## Known issues and "TO DO" items
### Content issues
1. Revise the Volunteer's FAQ (currently in "the attic" since it was outdated). **Status: The Whitewashers team is looking into this.**
2. Add these links to the DP HTML documentation, to the Volunteer's FAQ. **Status: awaiting the Volunteer's FAQ mentioned just above.**
The Post-Processing FAQ --
https://www.pgdp.net/wiki/DP_Official_Documentation:PP_and_PPV/Post-Processing_FAQ
Easy Epub -- https://www.pgdp.net/wiki/DP_Official_Documentation:PP_and_PPV/Easy_Epub (It's a guide to how best to handle the HTML that goes through epubmaker to lead to passable epubs/mobis)
HTML Best Practices -- https://www.pgdp.org/~jana/best-practices/ (this was written a while back but DP tries to keep it up-to-date)
### User interface and user experience issues
1. Logo improvements. Some changes to the size and layout of the logos, which generally appear on the top left of the new website's pages. **Status: being worked on.**
### Search-related issues
1. UI/Search: Search output order seems random.**Status: Defer. Search is handled by PostgreSQL, and unchanged from the current site. We will look into a different search implementation, for the future.**
# New Website Overview
The Project Gutenberg website, along with some of its back-end functionality, has undergone major updates in 2019 and 2020. On August 26 2020, the legacy site was retired and the new site went into service - this followed over six months of public "beta" testing, when members of the public were invited to visit the new site and give input (including a brief anonymous survey).
Input, fixes and suggestions are welcome. The website is still undergoing development, and the basic look-and-feel (menus, colors, fonts, etc.) has been updated with the new design.
You can find the development site at [dev.gutenberg.org](https://dev.gutenberg.org) (http or https). The main production site remains at [www.gutenberg.org](https://www.gutenberg.org).
## Goals
@ -17,11 +37,12 @@ Goals for the website redesign and back-end updates include:
* Modernize the website, including a responsive design and updated content. Deprecation or removal of outdated content.
* Abandonment of non-English website pages, which generally have been unmaintained for years. (Interest in non-English eBooks remains strong, and there has been no decrease in the enthusiasm for non-English eBooks.)
* Utilize HTML version 5 and Cascading Style Sheets for the entire experience. We will no longer require Javascript, and will only use dynamically-generated pages where they truly make sense (such as for searching).
* Utilize HTML version 5 and Cascading Style Sheets for the entire experience. The site no longer requires Javascript, and uses dynamically-generated pages only for eBook landing pages and search results.
* Discontinue the mobile site (previously at http://m.gutenberg.org), which was unmaintained and utilized older-style PHP that was vulnerable to mis-use.
* Stop utilizing MediaWiki for the main pages. Instead, write page content in Markdown. Propagate the content to the website automatically, with HTML5+CSS for layout, menus, etc.
* Make the whole website content publicly available for edit suggestions. We are using github for this.
* Make the metadata and site structure more easily mirrorable, whether in bulk or for subsets. This is partially done by using github for website content. In the future, we plan to make landing pages use static HTML5+CSS, instead of via autocat3 (below).
* In the future, we hope to make the eBooks themselves available similarly. This will facilitate reader-contributed formats, as well as errata reporting and fixes.
* In the future, we hope to make the eBooks themselves available similarly (via github). This will facilitate reader-contributed formats, as well as new methods for errata reporting and fixes.
## How to provide input, fixes and suggestions
@ -29,7 +50,7 @@ Please first check below in case your suggestion is already being worked on.
### General reporting of errors
First, if you find a error, typo, etc. within an eBook you can report it using the [regular mechanisms](/help/errata.html). The eBooks themselves are exactly the same (i.e., the exact same files), whether you are looking at the development site or the main site.
First, if you find a error, typo, etc. within an eBook you can report it using the [regular mechanisms](/help/errata.html). The eBooks themselves are exactly the same on this new site as on the old (i.e., the exact same files).
If you are not sure how to report a problem you find, it's fine to just email help2020@pglaf.org. It will get to the right person.
@ -66,27 +87,5 @@ If you encounter something that doesn't work for you, or could be better, please
It is especially important that we can understand what you are experiencing, and what could be improved. To help with this, please provide a description of how you accessed the website, and the problems encountered. If you know of external testing sites, or standards, or software that we should look into, please let us know.
Eventually, we hope to retire much or all of the current Project Gutenberg [mobile site](http://m.gutenberg.org), because it does not have anyone maintaining it, and it lacks much of the functionality and content of the main site.
## Known issues and "TO DO" items
### Content issues
1. Revise the Volunteer's FAQ (currently in "the attic" since it was outdated). **Status: The Whitewashers team is looking into this.**
2. Add these links to the DP HTML documentation, to the Volunteer's FAQ. **Status: awaiting the Volunteer's FAQ mentioned just above.**
The Post-Processing FAQ --
https://www.pgdp.net/wiki/DP_Official_Documentation:PP_and_PPV/Post-Processing_FAQ
Easy Epub -- https://www.pgdp.net/wiki/DP_Official_Documentation:PP_and_PPV/Easy_Epub (It's a guide to how best to handle the HTML that goes through epubmaker to lead to passable epubs/mobis)
HTML Best Practices -- https://www.pgdp.org/~jana/best-practices/ (this was written a while back but DP tries to keep it up-to-date)
### User interface and user experience issues
1. Logo improvements. Some changes to the size and layout of the logos, which generally appear on the top left of the new website's pages. **Status: being worked on.**
### Search-related issues
The [autocat3](https://github.com/gutenbergtools/autocat3) program handles search, and also generates landing pages. There are probably some remaining problems with how results are displayed, or possibly with wrong links in bibrec tabs. If you find a problem, please send the exact link or search you used, so we can replicate and fix.
1. UI/Search: Search output order seems random.**Status: Defer. Search is handled by PostgreSQL, and unchanged from the current site. We will look into a different search implementation, for the future.**
*Most recently updated: August 25, 2020*
*Most recently updated: August 27, 2020*