mirror of
https://github.com/mblanke/Gov_Travel_App.git
synced 2026-03-01 14:10:22 -05:00
mblanke
15094ac94b
- Implemented Python scraper using BeautifulSoup and pandas to automatically collect travel rates from official NJC website - Added currency extraction from table titles (supports EUR, USD, AUD, CAD, ARS, etc.) - Added country extraction from table titles for international rates - Flatten pandas MultiIndex columns for cleaner data structure - Default to CAD for domestic Canadian sources (accommodations and domestic tables) - Created SQLite database schema (raw_tables, rate_entries, exchange_rates, accommodations) - Successfully scraped 92 tables with 17,205 rate entries covering 25 international cities - Added migration script to convert scraped data to Node.js database format - Updated .gitignore for Python files (.venv/, __pycache__, *.pyc, *.sqlite3) - Fixed city validation and currency conversion in main app - Added comprehensive debug and verification scripts This replaces manual JSON maintenance with automated data collection from official government source.
29 lines
603 B
Docker
29 lines
603 B
Docker
# Use Node.js LTS version (slim Debian-based image for better compatibility)
|
|
FROM node:18-slim
|
|
|
|
# Install SQLite
|
|
RUN apt-get update && apt-get install -y \
|
|
sqlite3 \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
WORKDIR /app
|
|
|
|
# Copy package files
|
|
COPY package*.json ./
|
|
|
|
# Install dependencies
|
|
RUN npm install --production
|
|
|
|
# Copy application files
|
|
COPY . .
|
|
|
|
# Run complete database migration on container start
|
|
RUN mkdir -p database && \
|
|
node scripts/migrateCompleteTravelRates.js || echo "Migration will run on first request"
|
|
|
|
# Expose port 5001
|
|
EXPOSE 5001
|
|
|
|
# Start the server
|
|
CMD ["node", "server.js"]
|