EbookFoundation
/
oapen-suggestion-service
mirror of https://github.com/EbookFoundation/oapen-suggestion-service.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
OAPEN Suggestion Service Web Application - SIT Senior Capstone
51 Commits
31 Branches
0 Tags
3.6 MiB
Python 68.1%
JavaScript 12.7%
TypeScript 11.4%
Dockerfile 2.7%
CSS 2.5%
Other 2.6%
 
 
 
 
 
 
Go to file
Celina Peralta 6d256249db Move daemon env vars to docker-compose.yml 2023-02-23 16:04:43 -05:00
.github/workflows Setup docker (#26) 2022-12-13 07:46:08 -05:00
api Fix harvest synchronization, add threshold parameter 2023-02-23 14:27:51 -05:00
oapen-engine Move daemon env vars to docker-compose.yml 2023-02-23 16:04:43 -05:00
web Setup docker (#26) 2022-12-13 07:46:08 -05:00
.flake8 [Draft] OAP-32 Ngram Caching (#18) 2022-11-02 23:07:56 -04:00
.gitignore Fixed minor error with handle validation (#33) 2023-02-09 16:46:22 +00:00
.isort.cfg Fix pre-commit hook + linting jobs for OAPEN engine (#14) 2022-10-23 19:51:47 -04:00
.pre-commit-config.yaml Fix pre-commit hook + linting jobs for OAPEN engine (#14) 2022-10-23 19:51:47 -04:00
LICENSE.md Create LICENSE.md 2022-09-27 14:00:13 -04:00
README.md Setup docker (#26) 2022-12-13 07:46:08 -05:00
all-dev.sh OAP-17: PostgreSQL integration into API with pg-promise, data function to read from DB, dotenv to read DB credentials from environment variables (#9) 2022-10-26 03:07:10 +00:00
docker-compose.yml Move daemon env vars to docker-compose.yml 2023-02-23 16:04:43 -05:00
pyproject.toml Fix pre-commit hook + linting jobs for OAPEN engine (#14) 2022-10-23 19:51:47 -04:00
run-api.sh OAP-17: PostgreSQL integration into API with pg-promise, data function to read from DB, dotenv to read DB credentials from environment variables (#9) 2022-10-26 03:07:10 +00:00
run-web.sh OAP-17: PostgreSQL integration into API with pg-promise, data function to read from DB, dotenv to read DB credentials from environment variables (#9) 2022-10-26 03:07:10 +00:00
setup.cfg [Draft] OAP-32 Ngram Caching (#18) 2022-11-02 23:07:56 -04:00
setup.sh OAP-17: PostgreSQL integration into API with pg-promise, data function to read from DB, dotenv to read DB credentials from environment variables (#9) 2022-10-26 03:07:10 +00:00

README.md

OAPEN Suggestion Engine

The OAPEN Suggestion Engine will suggest ebooks based on other books with similar content.

Running server

You can run all the servers together with ./all-dev.sh -- after installing dependencies with ./setup.sh

Monorepo components

This project is a monorepo, with multiple pieces that can be added or removed as neccessary for deployment.

Mining Engine (Core)

This engine is written in Python, and generates the recommendation data for users. Our suggestion service is centered around the trigram semantic inferencing algorithm. This script should be run as a job on a cron schedule to periodically ingest new texts added to the OAPEN catalog through their API. It will populate the Database (see Database section) with pre-processed lists of suggestions for each entry in the catalog.

You can find the code for the mining engine in oapen-engine/.

Base dependencies:

  • Python v3.10
  • PIP package manager
  • make

Automatically-installed dependencies:

  • nltk -- Natural language toolkit.
    • Maintained on GitHub by 300+ contributors.
    • Most recent update: 8 days ago
  • requests -- HTTP request library
    • Maintained on GitHub by 600+ conributors, and backed by sponsors.
    • Most recent update: 1 month ago.
  • psycopg2 -- PostgreSQL Database Adapter
    • Maintained on GitHub by 100+ contributors, and used by 480,000+ packages.
    • Most popular PostgreSQL database adapter for Python
  • pandas -- data analysis library
    • Maintained by PYData with large amounts of sponsors. 2,700+ contributors.
  • scikit-learn -- Scikit Learn

API Engine (Core)

This API server returns a list of recommended books from the database.

You can find the code for the API engine in api/.

Base dependencies:

  • NodeJS 14.x+
  • NPM package manager

Automatically-installed dependencies:

  • express - Basic HTTP server
  • pg-promise -- basic PostgreSQL driver
  • dotenv -- loads environment variables from .env

Web Demo (Optional)

This is a web-app demo that can be used to query the API engine and see suggested books. This does not have to be maintained if the API is used on another site, but is useful for development and a tech demo.

You can find the code for the web demo in web/.

Base dependencies:

  • NodeJS 14.x+
  • NPM package manager

Automatically-installed dependencies:

  • next -- Framework for production-driven web apps
    • Maintained by Vercel and the open source community
  • react -- Frontend design framework
    • Maintained by Meta.
    • Largest frontend web UI library.
    • (Alternative considered: Angular -- however, was recently deprecated by Google)
  • pg -- basic PostgreSQL driver
  • typescript -- Types for JavaScript
    • Maintained by Microsoft and the open source community.