diff --git a/updates.md b/updates.md new file mode 100644 index 0000000..063b42a --- /dev/null +++ b/updates.md @@ -0,0 +1,203 @@ +# Lottery Tracker — Update Log + +## Overview + +Complete codebase overhaul: cleanup, modernization, new features, and full Next.js frontend build. + +--- + +## Phase 1: Code Review & Cleanup + +### Files Deleted +- `email_sender.py` — Email sending logic (removed entire email system) +- `send_email_now.py` — Manual email trigger +- `test_email.py` — Email tests +- `analyze_excel.py` — Excel analysis utility +- `read_excel.py` — Excel reader utility +- `import requests.py` — Misnamed utility script +- `powerball_numbers.html` — Debug HTML file +- `megamillions_debug.html` — Debug HTML file +- `Dockerfile.email` — Email service Docker image +- `EMAIL_SETUP.md` — Email configuration docs + +### Issues Fixed +- **Hardcoded credentials** removed (Gmail password in docker docs) +- **`verify=False`** on all HTTP requests replaced with default SSL verification +- **Code duplication** — scraping logic was copy-pasted across 3 files, consolidated into one +- **No tests** — full test suite added +- **No linting** — ruff configured +- **Flask `debug=True`** hardcoded — replaced with env var +- **Docker healthcheck** used `curl` (not installed in slim image) — switched to Python urllib +- **`.dockerignore`** missing `.venv/` — build context was 456 MB, now much smaller + +--- + +## Phase 2: Backend Rewrite + +### New Modules + +#### `config.py` +- `AppConfig` dataclass with nested `ScraperURLs`, `TaxConfig`, `InvestmentDefaults` +- `STATE_TAX_RATES` — all 51 US states + DC with 2024-2025 tax rates +- `LOTTERY_ODDS` — odds and ticket costs for all 4 lotteries +- `ANNUITY_YEARS`, `ANNUITY_ANNUAL_INCREASE` constants +- `load_config()` reads from environment variables with sensible defaults + +#### `scrapers.py` +- Unified scraping with TTL cache (6-hour default via `cachetools`) +- `scrape_powerball()` / `scrape_mega_millions()` — requests + BeautifulSoup from lotto.net +- `scrape_canadian_lotteries()` — Playwright sync API from olg.ca +- `get_all_jackpots()` — returns all 4 jackpots with cache +- `clear_cache()` — force refresh + +#### `app.py` (Rewritten) +Flask app factory pattern (`create_app()`) with endpoints: +| Endpoint | Method | Description | +|----------|--------|-------------| +| `/api/jackpots` | GET | Current jackpots (cached) | +| `/api/jackpots/refresh` | POST | Force-refresh jackpots | +| `/api/calculate` | POST | Full investment calculation with state tax | +| `/api/states` | GET | All US states with tax rates | +| `/api/states/` | GET | Single state info | +| `/api/compare` | GET | Side-by-side all 4 lotteries | +| `/api/calculate/breakeven` | POST | Break-even jackpot analysis | +| `/api/calculate/annuity` | POST | Annuity payment schedule | +| `/api/calculate/group` | POST | Group play split | +| `/api/odds` | GET | Probability data for all lotteries | +| `/api/health` | GET | Health check | + +#### `lottery_calculator.py` (Rewritten) +Pure calculation functions with explicit parameters: +- `_run_investment_cycles()` — shared 90-day reinvestment cycle logic +- `calculate_us_lottery()` — lump sum + federal/state tax + CAD conversion + cycles +- `calculate_canadian_lottery()` — tax-free + investment cycles +- `calculate_break_even()` — EV formula: required jackpot for positive expected value +- `calculate_annuity()` — 30-year schedule with configurable annual increase +- `calculate_group_split()` — N-way split with custom share ratios + +### Configuration Files +- `requirements.txt` — pinned versions, removed openpyxl/pandas/schedule +- `requirements-dev.txt` — pytest, pytest-mock, pytest-cov, httpx, ruff +- `pyproject.toml` — ruff config (line-length=100, py311 target), pytest config +- `.env.example` — all environment variables documented + +### Test Suite (64 tests, all passing) +- `tests/conftest.py` — Flask test fixtures +- `tests/test_lottery_calculator.py` — ~25 tests (US calc, Canadian calc, break-even, annuity, group split) +- `tests/test_api.py` — ~15 endpoint integration tests with mocked scrapers +- `tests/test_config.py` — config loading, env override, state validation +- `tests/test_scrapers.py` — parser tests, mocked HTTP, cache behavior + +--- + +## Phase 3: Frontend (Next.js 15 + MUI 6) + +### Scaffold +- `frontend/package.json` — Next.js 15.1.4, MUI 6.3.1, Recharts 2.15, Axios +- `frontend/tsconfig.json` — standard Next.js TypeScript config with `@/*` paths +- `frontend/next.config.js` — standalone output, `/api/*` rewrite to backend +- `frontend/.env.local` — `NEXT_PUBLIC_API_URL=http://localhost:5000` + +### Shared Libraries +- `frontend/lib/types.ts` — Full TypeScript interfaces (JackpotData, Calculation, StateInfo, BreakEvenResult, AnnuityResult, GroupResult, OddsInfo, etc.) +- `frontend/lib/api.ts` — Typed Axios client with all API endpoint functions +- `frontend/lib/format.ts` — `formatCurrency`, `formatCurrencyFull`, `formatPercent`, `formatNumber` +- `frontend/lib/theme.ts` — Dark MUI theme (primary: `#4fc3f7`, background: `#0a1929`) + +### Components +- `frontend/components/ThemeRegistry.tsx` — MUI ThemeProvider wrapper with CssBaseline +- `frontend/components/NavBar.tsx` — Sticky app bar with navigation links + +### Pages (7 routes) + +| Route | File | Description | +|-------|------|-------------| +| `/` | `app/page.tsx` | Home — jackpot cards grid (4 lotteries) + tool cards grid (6 tools) | +| `/calculator` | `app/calculator/page.tsx` | Investment calculator — jackpot input, lottery type, state selector with tax rates, invest % slider, annual return slider, cycles, results summary, 90-day cycle table | +| `/compare` | `app/compare/page.tsx` | Side-by-side comparison — state selector, 4 lottery cards with jackpot, odds, after-tax take-home, daily/annual income | +| `/breakeven` | `app/breakeven/page.tsx` | Break-even analysis — lottery/state selectors, required jackpot, odds, probability, take-home fraction, expected value | +| `/annuity` | `app/annuity/page.tsx` | Annuity calculator — jackpot, type, years, annual increase slider, summary cards, year-by-year payment schedule table | +| `/group` | `app/group/page.tsx` | Group play — member count, custom share weights, per-member breakdown table (share %, jackpot share, after-tax, daily/annual income) | +| `/odds` | `app/odds/page.tsx` | Probability display — odds, percentage, ticket cost, log-scale relative bar, "50% chance" ticket count | + +--- + +## Phase 4: Docker & Infrastructure + +### Docker Images Built +| Image | Size | Description | +|-------|------|-------------| +| `lottery-tracker-backend` | 2.28 GB | Python 3.13-slim + Playwright Chromium + Gunicorn | +| `lottery-tracker-frontend` | 285 MB | Node.js 20-alpine + Next.js standalone | + +### Docker Fixes Applied +1. **Removed `playwright install-deps chromium`** from Dockerfile.backend — system deps already installed manually; the auto-installer fails on newer Debian due to renamed font packages (`ttf-unifont` → `fonts-unifont`) +2. **Generated `package-lock.json`** — required by `npm ci` in frontend Dockerfile +3. **Created `frontend/public/`** directory with `manifest.json` — Dockerfile COPY step requires it +4. **Added `.venv/` to `.dockerignore`** — was inflating build context +5. **Removed `version: '3.8'`** warning — obsolete in modern Docker Compose +6. **Removed email-scheduler service** from `docker-compose.yml` +7. **Switched healthcheck** from `curl` to `python -c "import urllib.request; ..."` +8. **Backend CMD** changed to gunicorn (2 workers, 120s timeout) + +### MUI Grid Fix +All 7 frontend pages used `` syntax (Grid2 API), but imported `Grid` from `@mui/material` (legacy Grid v1). Fixed by changing imports to `Grid2 as Grid` in all pages. + +### Running Containers +| Container | Status | Port Mapping | +|-----------|--------|-------------| +| `lottery-backend` | Up (healthy) | `0.0.0.0:5000 → 5000` | +| `lottery-frontend` | Up | `0.0.0.0:3003 → 3000` | + +### Access URLs +- **Frontend**: http://localhost:3003 +- **Backend API**: http://localhost:5000/api/health + +--- + +## Phase 5: Documentation & Config + +### Updated Files +- **`.gitignore`** — added `.next/`, `*.xlsx`, `*.html`, `ssl/` certs, coverage outputs +- **`AGENTS.md`** — filled in repo facts (Python 3.13/Flask + Next.js 15/MUI 6, pytest, ruff, docker compose) +- **`DOCKER_README.md`** — rewritten: removed email references, removed hardcoded password, added `.env.example` workflow, config table, production deployment guide +- **`DOCKER_QUICKSTART.md`** — rewritten: removed email service, updated container descriptions, simplified quick start + +--- + +## Verification + +- **64/64 tests passing** (`pytest -q`) +- **ruff lint clean** (`ruff check .` — all checks passed) +- **Docker build successful** — both images built +- **Containers running** — backend healthy, frontend serving + +--- + +## Architecture Summary + +``` +┌─────────────────────────────────────────────────┐ +│ nginx (prod) │ +│ Rate limiting + SSL │ +├─────────────────┬───────────────────────────────┤ +│ Frontend │ Backend │ +│ Next.js 15 │ Flask 3.1 │ +│ MUI 6 │ Gunicorn │ +│ TypeScript │ Python 3.13 │ +│ Port 3000 │ Port 5000 │ +│ │ │ +│ 7 pages │ 11 API endpoints │ +│ Dark theme │ 4 lottery scrapers │ +│ Axios client │ TTL cache (6h) │ +│ │ 51 state tax rates │ +│ │ Playwright (Canadian) │ +└─────────────────┴───────────────────────────────┘ +``` + +### Tech Stack +- **Backend**: Python 3.13, Flask 3.1, Gunicorn, BeautifulSoup, Playwright +- **Frontend**: Next.js 15, TypeScript, Material-UI 6, Recharts, Axios +- **Testing**: pytest (64 tests), ruff (lint) +- **Docker**: Multi-container (backend + frontend + nginx for prod) +- **Config**: Environment variables via `.env` + `python-dotenv`