Add comprehensive documentation and API testing script

Co-authored-by: mblanke <9078342+mblanke@users.noreply.github.com>

ARCHITECTURE.md

@@ -0,0 +1,472 @@
+# Architecture Documentation
+
+This document describes the architecture and design decisions for VelociCompanion.
+
+## System Overview
+
+VelociCompanion is a multi-tenant, cloud-native threat hunting companion designed to work with Velociraptor. It provides secure authentication, data isolation, and role-based access control.
+
+```
+┌─────────────┐     ┌─────────────┐     ┌──────────────┐
+│             │     │             │     │              │
+│  Frontend   │────▶│   Backend   │────▶│  PostgreSQL  │
+│  (React)    │     │  (FastAPI)  │     │   Database   │
+│             │     │             │     │              │
+└─────────────┘     └─────────────┘     └──────────────┘
+                           │
+                           ▼
+                    ┌─────────────┐
+                    │             │
+                    │ Velociraptor│
+                    │   Servers   │
+                    │             │
+                    └─────────────┘
+```
+
+## Technology Stack
+
+### Backend
+- **FastAPI**: Modern, fast web framework for building APIs
+- **SQLAlchemy**: SQL toolkit and ORM
+- **PostgreSQL**: Relational database
+- **Alembic**: Database migration tool
+- **Python-Jose**: JWT token handling
+- **Passlib**: Password hashing with bcrypt
+
+### Frontend
+- **React**: UI library
+- **TypeScript**: Type-safe JavaScript
+- **Axios**: HTTP client
+- **React Router**: Client-side routing
+
+### Infrastructure
+- **Docker**: Containerization
+- **Docker Compose**: Multi-container orchestration
+
+## Core Components
+
+### 1. Authentication System
+
+#### JWT Token Flow
+```
+1. User submits credentials (username/password)
+2. Backend verifies credentials
+3. Backend generates JWT token with:
+   - user_id (sub)
+   - tenant_id
+   - role
+   - expiration time
+4. Frontend stores token in localStorage
+5. All subsequent requests include token in Authorization header
+6. Backend validates token and extracts user context
+```
+
+#### Password Security
+- Passwords are hashed using bcrypt with automatic salt generation
+- Password hashes are never exposed in API responses
+- Plaintext passwords are never logged or stored
+
+#### Token Security
+- Tokens expire after 30 minutes (configurable)
+- Tokens are signed with HS256 algorithm
+- Secret key must be at least 32 characters
+
+### 2. Multi-Tenancy
+
+#### Data Isolation
+Every database query is automatically scoped to the user's tenant:
+
+```python
+# Example: Listing hosts
+hosts = db.query(Host).filter(Host.tenant_id == current_user.tenant_id).all()
+```
+
+#### Tenant Creation
+- Default tenant is created automatically on first user registration
+- Admin users can create additional tenants
+- Users are assigned to exactly one tenant
+
+#### Cross-Tenant Access
+- Regular users: Can only access data in their tenant
+- Admin users: Can access all data in their tenant
+- Super-admin (future): Could access multiple tenants
+
+### 3. Role-Based Access Control (RBAC)
+
+#### Roles
+- **user**: Standard user with read/write access to their tenant's data
+- **admin**: Elevated privileges within their tenant
+  - Can manage users in their tenant
+  - Can create/modify/delete resources
+  - Can view all data in their tenant
+
+#### Permission Enforcement
+```python
+# Endpoint requiring admin role
+@router.get("/users")
+async def list_users(
+    current_user: User = Depends(require_role(["admin"]))
+):
+    # Only admins can access this
+    pass
+```
+
+### 4. Database Schema
+
+#### Core Tables
+
+**tenants**
+- id (PK)
+- name (unique)
+- description
+- created_at
+
+**users**
+- id (PK)
+- username (unique)
+- password_hash
+- role
+- tenant_id (FK → tenants)
+- is_active
+- created_at
+
+**hosts**
+- id (PK)
+- hostname
+- ip_address
+- os
+- tenant_id (FK → tenants)
+- host_metadata (JSON)
+- created_at
+- last_seen
+
+**cases**
+- id (PK)
+- title
+- description
+- status (open, closed, investigating)
+- severity (low, medium, high, critical)
+- tenant_id (FK → tenants)
+- created_at
+- updated_at
+
+**artifacts**
+- id (PK)
+- artifact_type (hash, ip, domain, email, etc.)
+- value
+- description
+- case_id (FK → cases)
+- artifact_metadata (JSON)
+- created_at
+
+#### Relationships
+```
+tenants (1) ──< (N) users
+tenants (1) ──< (N) hosts
+tenants (1) ──< (N) cases
+cases (1) ──< (N) artifacts
+```
+
+### 5. API Design
+
+#### RESTful Principles
+- Resources are nouns (users, hosts, cases)
+- HTTP methods represent actions (GET, POST, PUT, DELETE)
+- Proper status codes (200, 201, 401, 403, 404)
+
+#### Authentication
+All endpoints except `/auth/register` and `/auth/login` require authentication.
+
+```
+Authorization: Bearer <jwt_token>
+```
+
+#### Response Format
+Success:
+```json
+{
+  "id": 1,
+  "username": "john",
+  "role": "user",
+  "tenant_id": 1
+}
+```
+
+Error:
+```json
+{
+  "detail": "User not found"
+}
+```
+
+### 6. Frontend Architecture
+
+#### Component Structure
+```
+src/
+├── components/        # Reusable UI components
+│   └── PrivateRoute.tsx
+├── context/          # React Context providers
+│   └── AuthContext.tsx
+├── pages/           # Page components
+│   ├── Login.tsx
+│   └── Dashboard.tsx
+├── utils/           # Utilities
+│   └── api.ts       # API client
+├── App.tsx          # Main app component
+└── index.tsx        # Entry point
+```
+
+#### State Management
+- **AuthContext**: Global authentication state
+  - Current user
+  - Login/logout functions
+  - Loading state
+  - Authentication status
+
+#### Routing
+```
+/login        → Login page (public)
+/             → Dashboard (protected)
+/*            → Redirect to / (protected)
+```
+
+### 7. Security Architecture
+
+#### Authentication Flow
+1. Frontend sends credentials to `/api/auth/login`
+2. Backend validates and returns JWT token
+3. Frontend stores token in localStorage
+4. Token included in all API requests
+5. Backend validates token on each request
+
+#### Authorization Flow
+1. Extract JWT from Authorization header
+2. Verify token signature and expiration
+3. Extract user_id from token payload
+4. Load user from database
+5. Check user's role for endpoint access
+6. Apply tenant scoping to queries
+
+#### Security Headers
+```python
+# CORS configuration
+allow_origins=["http://localhost:3000"]
+allow_credentials=True
+allow_methods=["*"]
+allow_headers=["*"]
+```
+
+## Data Flow Examples
+
+### User Registration
+```
+1. POST /api/auth/register
+   {username: "john", password: "pass123"}
+2. Backend hashes password
+3. Create default tenant if needed
+4. Create user record
+5. Return user object (without password_hash)
+```
+
+### Host Ingestion
+```
+1. Velociraptor sends data to POST /api/ingestion/ingest
+   - Must include valid JWT token
+2. Extract tenant_id from current user
+3. Find or create host with hostname
+4. Update host metadata
+5. Return success response
+```
+
+### Listing Resources
+```
+1. GET /api/hosts with Authorization header
+2. Validate JWT token
+3. Extract tenant_id from user
+4. Query: SELECT * FROM hosts WHERE tenant_id = ?
+5. Return filtered results
+```
+
+## Deployment Architecture
+
+### Development
+```
+┌──────────────────────────────────────┐
+│           Docker Compose              │
+├──────────────────────────────────────┤
+│  Frontend:3000  Backend:8000  DB:5432│
+└──────────────────────────────────────┘
+```
+
+### Production (Recommended)
+```
+┌─────────────┐     ┌─────────────┐
+│   Nginx/    │     │  Frontend   │
+│   Traefik   │────▶│  (Static)   │
+│   (HTTPS)   │     └─────────────┘
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐     ┌──────────────┐
+│   Backend   │     │  PostgreSQL  │
+│  (Multiple  │────▶│  (Managed)   │
+│  instances) │     └──────────────┘
+└─────────────┘
+```
+
+## Performance Considerations
+
+### Database Indexing
+- Primary keys on all tables
+- Unique index on usernames
+- Index on tenant_id columns for fast filtering
+- Index on hostname for host lookups
+
+### Query Optimization
+- Always filter by tenant_id early in queries
+- Use pagination for large result sets (skip/limit)
+- Lazy load relationships when not needed
+
+### Caching (Future)
+- Cache tenant information
+- Cache user profiles
+- Cache frequently accessed hosts
+
+## Monitoring & Logging
+
+### Health Checks
+```
+GET /health → {"status": "healthy"}
+```
+
+### Logging
+- Request logging via Uvicorn
+- Error tracking in application logs
+- Database query logging (development only)
+
+### Metrics (Future)
+- Request count per endpoint
+- Authentication success/failure rate
+- Database query performance
+- Active user count
+
+## Migration Strategy
+
+### Database Migrations
+```bash
+# Create migration
+alembic revision --autogenerate -m "Description"
+
+# Apply migration
+alembic upgrade head
+
+# Rollback
+alembic downgrade -1
+```
+
+### Schema Evolution
+1. Create migration for schema changes
+2. Test migration in development
+3. Apply to staging environment
+4. Verify data integrity
+5. Apply to production during maintenance window
+
+## Testing Strategy
+
+### Unit Tests (Future)
+- Test individual functions
+- Mock database connections
+- Test password hashing
+- Test JWT token creation/verification
+
+### Integration Tests (Future)
+- Test API endpoints
+- Test authentication flow
+- Test multi-tenancy isolation
+- Test RBAC enforcement
+
+### Manual Testing
+- Use test_api.sh script
+- Use FastAPI's /docs interface
+- Test frontend authentication flow
+
+## Future Enhancements
+
+### Phase 2
+- Refresh tokens for longer sessions
+- Password reset functionality
+- Email verification
+- Two-factor authentication (2FA)
+
+### Phase 3
+- Audit logging
+- Advanced search and filtering
+- Real-time notifications
+- Velociraptor direct integration
+
+### Phase 4
+- Machine learning for threat detection
+- Automated playbooks
+- Integration with SIEM systems
+- Advanced reporting and analytics
+
+## Troubleshooting Guide
+
+### Common Issues
+
+**Token Expired**
+- Tokens expire after 30 minutes
+- User must login again
+- Consider implementing refresh tokens
+
+**Permission Denied**
+- User lacks required role
+- Check user's role in database
+- Verify endpoint requires correct role
+
+**Data Not Visible**
+- Check tenant_id of user
+- Verify data belongs to correct tenant
+- Ensure tenant_id is being applied to queries
+
+**Database Connection Failed**
+- Check DATABASE_URL environment variable
+- Verify PostgreSQL is running
+- Check network connectivity
+
+## Development Guidelines
+
+### Adding New Endpoints
+
+1. Create route in `app/api/routes/`
+2. Add authentication dependency
+3. Apply tenant scoping to queries
+4. Add role check if needed
+5. Create Pydantic schemas
+6. Update router registration in main.py
+7. Test with /docs interface
+
+### Adding New Models
+
+1. Create model in `app/models/`
+2. Add tenant_id foreign key
+3. Create migration
+4. Create Pydantic schemas
+5. Create CRUD routes
+6. Apply tenant scoping
+
+### Code Style
+
+- Follow PEP 8 for Python
+- Use type hints
+- Write docstrings for functions
+- Keep functions small and focused
+- Use meaningful variable names
+
+## References
+
+- [FastAPI Documentation](https://fastapi.tiangolo.com/)
+- [SQLAlchemy Documentation](https://docs.sqlalchemy.org/)
+- [JWT RFC](https://tools.ietf.org/html/rfc7519)
+- [OAuth 2.0 RFC](https://tools.ietf.org/html/rfc6749)

QUICKSTART.md

@@ -0,0 +1,263 @@
+# Quick Start Guide
+
+This guide will help you get VelociCompanion up and running in minutes.
+
+## Prerequisites
+
+- Docker and Docker Compose installed
+- 8GB RAM minimum
+- Ports 3000, 5432, and 8000 available
+
+## Step 1: Start the Application
+
+```bash
+# Clone the repository
+git clone https://github.com/mblanke/ThreatHunt.git
+cd ThreatHunt
+
+# Start all services
+docker-compose up -d
+
+# Check service status
+docker-compose ps
+```
+
+Expected output:
+```
+NAME                COMMAND                  SERVICE             STATUS              PORTS
+threathunt-backend-1   "sh -c 'alembic upgr…"   backend             running             0.0.0.0:8000->8000/tcp
+threathunt-db-1        "docker-entrypoint.s…"   db                  running             0.0.0.0:5432->5432/tcp
+threathunt-frontend-1  "docker-entrypoint.s…"   frontend            running             0.0.0.0:3000->3000/tcp
+```
+
+## Step 2: Verify Backend is Running
+
+```bash
+# Check backend health
+curl http://localhost:8000/health
+
+# Expected response:
+# {"status":"healthy"}
+
+# View API documentation
+open http://localhost:8000/docs
+```
+
+## Step 3: Access the Frontend
+
+Open your browser and navigate to:
+```
+http://localhost:3000
+```
+
+You should see the VelociCompanion login page.
+
+## Step 4: Create Your First User
+
+### Option A: Via API (using curl)
+
+```bash
+# Register a new user
+curl -X POST http://localhost:8000/api/auth/register \
+  -H "Content-Type: application/json" \
+  -d '{
+    "username": "admin",
+    "password": "admin123",
+    "role": "admin"
+  }'
+
+# Login to get a token
+curl -X POST http://localhost:8000/api/auth/login \
+  -H "Content-Type: application/x-www-form-urlencoded" \
+  -d "username=admin&password=admin123"
+```
+
+### Option B: Via Frontend
+
+1. The first time you access the app, you'll need to register via API first (as shown above)
+2. Then login through the frontend at http://localhost:3000/login
+
+## Step 5: Explore the API
+
+Use the interactive API documentation at:
+```
+http://localhost:8000/docs
+```
+
+Click "Authorize" and enter your token in the format:
+```
+Bearer YOUR_TOKEN_HERE
+```
+
+## Step 6: Test the API
+
+Run the test script to verify all endpoints:
+
+```bash
+./test_api.sh
+```
+
+Expected output:
+```
+===================================
+VelociCompanion API Test Script
+===================================
+
+1. Testing health endpoint...
+✓ Health check passed
+
+2. Registering a new user...
+✓ User registration successful
+
+3. Logging in...
+✓ Login successful
+
+4. Getting current user profile...
+✓ Profile retrieval successful
+
+5. Listing tenants...
+✓ Tenants list retrieved
+
+6. Listing hosts...
+Hosts: []
+
+7. Testing authentication protection...
+✓ Authentication protection working
+
+===================================
+API Testing Complete!
+===================================
+```
+
+## Common Operations
+
+### Create a Host
+
+```bash
+# Get your token from login
+TOKEN="your_token_here"
+
+# Create a host
+curl -X POST http://localhost:8000/api/hosts \
+  -H "Authorization: Bearer $TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "hostname": "workstation-01",
+    "ip_address": "192.168.1.100",
+    "os": "Windows 10"
+  }'
+```
+
+### List Hosts
+
+```bash
+curl -X GET http://localhost:8000/api/hosts \
+  -H "Authorization: Bearer $TOKEN"
+```
+
+### Ingest Data
+
+```bash
+curl -X POST http://localhost:8000/api/ingestion/ingest \
+  -H "Authorization: Bearer $TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "hostname": "server-01",
+    "data": {
+      "artifact": "Windows.System.TaskScheduler",
+      "results": [...]
+    }
+  }'
+```
+
+## Troubleshooting
+
+### Database Connection Issues
+
+```bash
+# Check if database is running
+docker-compose logs db
+
+# Restart database
+docker-compose restart db
+```
+
+### Backend Not Starting
+
+```bash
+# Check backend logs
+docker-compose logs backend
+
+# Common issues:
+# - Database not ready: Wait a few seconds and check logs
+# - Port 8000 in use: Stop other services using that port
+```
+
+### Frontend Not Loading
+
+```bash
+# Check frontend logs
+docker-compose logs frontend
+
+# Rebuild frontend if needed
+docker-compose build frontend
+docker-compose up -d frontend
+```
+
+### Reset Everything
+
+```bash
+# Stop and remove all containers and volumes
+docker-compose down -v
+
+# Start fresh
+docker-compose up -d
+```
+
+## Next Steps
+
+1. **Create Additional Users**: Use the `/api/auth/register` endpoint
+2. **Set Up Tenants**: Create tenants via `/api/tenants` (admin only)
+3. **Integrate with Velociraptor**: Configure Velociraptor to send data to `/api/ingestion/ingest`
+4. **Explore Cases**: Create and manage threat hunting cases
+5. **Configure VirusTotal**: Set up VirusTotal API integration for hash lookups
+
+## Security Considerations
+
+⚠️ **Before deploying to production:**
+
+1. Change the `SECRET_KEY` in docker-compose.yml or .env file
+   - Must be at least 32 characters
+   - Use a cryptographically random string
+
+2. Use strong passwords for the database
+
+3. Enable HTTPS/TLS for API and frontend
+
+4. Configure proper firewall rules
+
+5. Review and update CORS settings in `backend/app/main.py`
+
+## Development Mode
+
+To run in development mode with hot reload:
+
+```bash
+# Backend
+cd backend
+python -m venv venv
+source venv/bin/activate
+pip install -r requirements.txt
+uvicorn app.main:app --reload
+
+# Frontend (in another terminal)
+cd frontend
+npm install
+npm start
+```
+
+## Support
+
+- Documentation: See [README.md](README.md)
+- API Docs: http://localhost:8000/docs
+- Issues: GitHub Issues

test_api.sh

@@ -0,0 +1,116 @@
+#!/bin/bash
+
+# Test script for VelociCompanion API
+# This script demonstrates the authentication flow and basic API usage
+
+API_URL="http://localhost:8000"
+
+echo "==================================="
+echo "VelociCompanion API Test Script"
+echo "==================================="
+echo ""
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+# Test 1: Health Check
+echo -e "${YELLOW}1. Testing health endpoint...${NC}"
+response=$(curl -s -o /dev/null -w "%{http_code}" "$API_URL/health")
+if [ "$response" = "200" ]; then
+    echo -e "${GREEN}✓ Health check passed${NC}"
+else
+    echo -e "${RED}✗ Health check failed (HTTP $response)${NC}"
+    exit 1
+fi
+echo ""
+
+# Test 2: Register a new user
+echo -e "${YELLOW}2. Registering a new user...${NC}"
+register_response=$(curl -s -X POST "$API_URL/api/auth/register" \
+    -H "Content-Type: application/json" \
+    -d '{
+        "username": "testuser",
+        "password": "testpass123",
+        "role": "user"
+    }')
+
+if echo "$register_response" | grep -q "username"; then
+    echo -e "${GREEN}✓ User registration successful${NC}"
+    echo "Response: $register_response"
+else
+    echo -e "${YELLOW}⚠ User might already exist or registration failed${NC}"
+    echo "Response: $register_response"
+fi
+echo ""
+
+# Test 3: Login
+echo -e "${YELLOW}3. Logging in...${NC}"
+login_response=$(curl -s -X POST "$API_URL/api/auth/login" \
+    -H "Content-Type: application/x-www-form-urlencoded" \
+    -d "username=testuser&password=testpass123")
+
+TOKEN=$(echo "$login_response" | grep -o '"access_token":"[^"]*"' | cut -d'"' -f4)
+
+if [ -n "$TOKEN" ]; then
+    echo -e "${GREEN}✓ Login successful${NC}"
+    echo "Token: ${TOKEN:0:20}..."
+else
+    echo -e "${RED}✗ Login failed${NC}"
+    echo "Response: $login_response"
+    exit 1
+fi
+echo ""
+
+# Test 4: Get current user profile
+echo -e "${YELLOW}4. Getting current user profile...${NC}"
+profile_response=$(curl -s -X GET "$API_URL/api/auth/me" \
+    -H "Authorization: Bearer $TOKEN")
+
+if echo "$profile_response" | grep -q "username"; then
+    echo -e "${GREEN}✓ Profile retrieval successful${NC}"
+    echo "Profile: $profile_response"
+else
+    echo -e "${RED}✗ Profile retrieval failed${NC}"
+    echo "Response: $profile_response"
+fi
+echo ""
+
+# Test 5: List tenants
+echo -e "${YELLOW}5. Listing tenants...${NC}"
+tenants_response=$(curl -s -X GET "$API_URL/api/tenants" \
+    -H "Authorization: Bearer $TOKEN")
+
+if echo "$tenants_response" | grep -q "name"; then
+    echo -e "${GREEN}✓ Tenants list retrieved${NC}"
+    echo "Tenants: $tenants_response"
+else
+    echo -e "${YELLOW}⚠ No tenants found or access denied${NC}"
+    echo "Response: $tenants_response"
+fi
+echo ""
+
+# Test 6: List hosts
+echo -e "${YELLOW}6. Listing hosts...${NC}"
+hosts_response=$(curl -s -X GET "$API_URL/api/hosts" \
+    -H "Authorization: Bearer $TOKEN")
+
+echo "Hosts: $hosts_response"
+echo ""
+
+# Test 7: Test authentication protection (without token)
+echo -e "${YELLOW}7. Testing authentication protection...${NC}"
+unauth_response=$(curl -s -o /dev/null -w "%{http_code}" -X GET "$API_URL/api/hosts")
+
+if [ "$unauth_response" = "401" ]; then
+    echo -e "${GREEN}✓ Authentication protection working (got 401 as expected)${NC}"
+else
+    echo -e "${RED}✗ Authentication not properly enforced (got HTTP $unauth_response)${NC}"
+fi
+echo ""
+
+echo "==================================="
+echo -e "${GREEN}API Testing Complete!${NC}"
+echo "==================================="