mirror of
https://github.com/mblanke/Gov_Travel_App.git
synced 2026-03-01 06:00:21 -05:00
mblanke
ae1f13d69e
- Implemented multiple scripts to check and inspect meal plans, scraped data, and accommodations for Munich and Riga. - Added a migration script to convert scraped data into the application's database format. - Introduced new database service methods for querying and updating travel rates. - Enhanced server configuration for serving static files in production. - Updated PostCSS configuration for consistency.
Gov_Travel_App
Overview
This repository contains a Python scraper that collects travel rate tables from the NJC and accommodation listings, then stores the raw tables and normalized entries in a SQLite database.
Setup
python -m venv .venv
source .venv/bin/activate
pip install -e .
Run the scraper
python -m gov_travel.main --db data/travel_rates.sqlite3
The database includes:
raw_tablesfor every scraped HTML table.rate_entriesfor parsed rate rows (country/city/province + rate fields).exchange_ratesfor parsed currency rates.accommodationsfor parsed lodging listings.
If a field is not detected by the heuristics, the full row is still preserved in raw_tables and the raw_json columns for deeper post-processing.