EbookFoundation
/
oapen-suggestion-service
mirror of https://github.com/EbookFoundation/oapen-suggestion-service.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
OAPEN Suggestion Service Web Application - SIT Senior Capstone
35 Commits
31 Branches
0 Tags
3.6 MiB
Python 68.1%
JavaScript 12.7%
TypeScript 11.4%
Dockerfile 2.7%
CSS 2.5%
Other 2.6%
 
 
 
 
 
 
Go to file
Celina Peralta 4333d4fcc3
[Draft] OAP-32 Ngram Caching (#18) 
* start caching ngrams

* fix build warnings

* add timestamp

* resolve comments

* pull out mogrify

* remove pytest from hook for now
 		2022-11-02 23:07:56 -04:00
.github/workflows [Draft] OAP-32 Ngram Caching (#18) 2022-11-02 23:07:56 -04:00
api OAP-17: PostgreSQL integration into API with pg-promise, data function to read from DB, dotenv to read DB credentials from environment variables (#9) 2022-10-26 03:07:10 +00:00
oapen-engine [Draft] OAP-32 Ngram Caching (#18) 2022-11-02 23:07:56 -04:00
web OAP-35 Connect `api/` and `web/`, fix querying between them, add running documentation & make dev environment easier to use (#13) 2022-10-18 08:10:11 -04:00
.flake8 [Draft] OAP-32 Ngram Caching (#18) 2022-11-02 23:07:56 -04:00
.gitignore celinanperalta/OAP 33 (#16) 2022-10-24 19:35:43 -04:00
.isort.cfg Fix pre-commit hook + linting jobs for OAPEN engine (#14) 2022-10-23 19:51:47 -04:00
.pre-commit-config.yaml Fix pre-commit hook + linting jobs for OAPEN engine (#14) 2022-10-23 19:51:47 -04:00
LICENSE.md Create LICENSE.md 2022-09-27 14:00:13 -04:00
README.md OAP-17: PostgreSQL integration into API with pg-promise, data function to read from DB, dotenv to read DB credentials from environment variables (#9) 2022-10-26 03:07:10 +00:00
all-dev.sh OAP-17: PostgreSQL integration into API with pg-promise, data function to read from DB, dotenv to read DB credentials from environment variables (#9) 2022-10-26 03:07:10 +00:00
pyproject.toml Fix pre-commit hook + linting jobs for OAPEN engine (#14) 2022-10-23 19:51:47 -04:00
run-api.sh OAP-17: PostgreSQL integration into API with pg-promise, data function to read from DB, dotenv to read DB credentials from environment variables (#9) 2022-10-26 03:07:10 +00:00
run-web.sh OAP-17: PostgreSQL integration into API with pg-promise, data function to read from DB, dotenv to read DB credentials from environment variables (#9) 2022-10-26 03:07:10 +00:00
setup.cfg [Draft] OAP-32 Ngram Caching (#18) 2022-11-02 23:07:56 -04:00
setup.sh OAP-17: PostgreSQL integration into API with pg-promise, data function to read from DB, dotenv to read DB credentials from environment variables (#9) 2022-10-26 03:07:10 +00:00

README.md

OAPEN Suggestion Engine

The OAPEN Suggestion Engine will suggest ebooks based on other books with similar content.

Running server

You can run all the servers together with ./all-dev.sh -- after installing dependencies with ./setup.sh

Monorepo components

This project is a monorepo, with multiple pieces that can be added or removed as neccessary for deployment.

Mining Engine (Core)

This engine is written in Python, and generates the recommendation data for users. Our suggestion service is centered around the trigram semantic inferencing algorithm. This script should be run as a job on a cron schedule to periodically ingest new texts added to the OAPEN catalog through their API. It will populate the Database (see Database section) with pre-processed lists of suggestions for each entry in the catalog.

You can find the code for the mining engine in oapen-engine/.

Base dependencies:

  • Python v3.10
  • PIP package manager
  • make

Automatically-installed dependencies:

  • nltk -- Natural language toolkit.
    • Maintained on GitHub by 300+ contributors.
    • Most recent update: 8 days ago
  • requests -- HTTP request library
    • Maintained on GitHub by 600+ conributors, and backed by sponsors.
    • Most recent update: 1 month ago.
  • psycopg2 -- PostgreSQL Database Adapter
    • Maintained on GitHub by 100+ contributors, and used by 480,000+ packages.
    • Most popular PostgreSQL database adapter for Python
  • pandas -- data analysis library
    • Maintained by PYData with large amounts of sponsors. 2,700+ contributors.
  • sklearn -- Scikit Learn

API Engine (Core)

This API server returns a list of recommended books from the database.

You can find the code for the API engine in api/.

Base dependencies:

  • NodeJS 14.x+
  • NPM package manager

Automatically-installed dependencies:

  • express - Basic HTTP server
  • pg-promise -- basic PostgreSQL driver
  • dotenv -- loads environment variables from .env

Web Demo (Optional)

This is a web-app demo that can be used to query the API engine and see suggested books. This does not have to be maintained if the API is used on another site, but is useful for development and a tech demo.

You can find the code for the web demo in web/.

Base dependencies:

  • NodeJS 14.x+
  • NPM package manager

Automatically-installed dependencies:

  • next -- Framework for production-driven web apps
    • Maintained by Vercel and the open source community
  • react -- Frontend design framework
    • Maintained by Meta.
    • Largest frontend web UI library.
    • (Alternative considered: Angular -- however, was recently deprecated by Google)
  • pg -- basic PostgreSQL driver
  • typescript -- Types for JavaScript
    • Maintained by Microsoft and the open source community.