Dockerized Scraper

- Implements Dockerized Version of Scraper
- Atomized tags and categories columns
This commit is contained in:
quorploop 2025-12-20 20:55:04 +01:00
parent 7cc3d1b7e4
commit bcd210ce01
5 changed files with 201 additions and 42 deletions

View file

@ -0,0 +1,18 @@
Knack-Scraper does exacly what its name suggests it does.
Knack-Scraper scrapes knack.news and writes to an sqlite
database for later usage.
## Example for .env
```
NUM_THREADS=8
NUM_SCRAPES=100
DATABASE_LOCATION='./data/knack.sqlite'
```
## Run once
```
python main.py
```