Events Calendar, Pipeline Health Dashboard, Mass.gov Search Expansion, Security Hardening
New Features
- -Events calendar — Full month-view calendar UI at /<town>/events with colored event dots (purple = Town, blue = Library, green = Schools), day detail panel, source filters (All/Town/Library/Schools), Month/List view toggle, and month navigation (PR #141)
- -Calendar subscribe — "Subscribe to Calendar" button copies an iCal feed URL for Google Calendar, Apple Calendar, or Outlook. Per-event "Add to Calendar" dropdown with Google Calendar link and .ics download (PR #141)
- -ICS feed endpoint — /api/events/ics?town=needham returns a VCALENDAR feed with events for the past 7 days + next 90 days, 1-hour cache (PR #141)
- -Pipeline Health dashboard — New "Pipeline" tab in admin showing connector health (healthy/warning/error/disabled badges), last pipeline run, content freshness KPIs, and article generation stats (PR #140)
- -Mass.gov query expansion — 20+ new regex patterns in the query tier router so searches about building codes, septic, property assessment, conservation, firearms/liquor licenses, and special education now include relevant mass.gov results alongside local content (PR #137)
Bug Fixes
- -Duplicate search bars — Fixed homepage showing two search bars by adding case-insensitive regex and sticky positioning (PR #143)
- -iCal injection — Backslash escaping now runs before other iCal escape sequences to prevent content injection (PR #143)
Other Changes
- -12 CodeQL alerts resolved — SSRF re-parse in test-url endpoint, log injection prevention in connector runner, exact domain matching in geo-filter (PR #143)
- -~70 SonarCloud issues resolved — .replace(/g) → .replaceAll(), Readonly<> props, nested ternary extraction, globalThis usage, negated condition flips across 28 files (PR #143)
- -Event source configs — Seed script for 5 event sources: 3 town iCal feeds (enabled), library scraper + school calendar (disabled pending setup) (PR #141)
UX Polish, Search Quality, and Security Hardening
New Features
- -AI disclaimers — "AI answers may not be current" notice on search answer cards and first chat message (PR #130)
- -Source date labels — Source chips now show document date (e.g. "Verified Jan 2026") so users know how fresh the data is (PR #130, #133)
- -Search suggestions dropdown — Hero search input shows filtered suggestions on focus, populated from Popular Questions (PR #130)
- -Article filters — Keyword search and date range pills (Today / This Week / This Month / All Time) on the news page (PR #130)
- -Rotating search placeholder — Search bar cycles through 5 example queries every 3.5s, i18n in 3 languages, pauses when typing (PR #133)
- -Events empty state — Helpful prompt to ask Navigator about upcoming events when no calendar data is available (PR #129)
Bug Fixes
- -Search deduplication — Two-pass dedup (canonical URL + normalized title) eliminates duplicate results for common queries like "transfer station" and "building permits" (PR #131, #136)
- -Document-level dedup in RAG — hybridSearch now deduplicates at the document level, so the same page doesn't appear multiple times even when multiple chunks match (PR #136)
- -USED_SOURCES leak — Internal source metadata no longer appears in AI chat responses (PR #127, #130)
- -Geographic filtering — Off-topic articles from other states (e.g. Connecticut Eversource) filtered from homepage feed; article generation now applies geo-filter at output (PR #129, #131)
- -Expired transit alerts — Transit page now filters out expired MBTA alerts and auto-refreshes (PR #128)
- -Negative timestamps — Fixed edge case where malformed dates produced negative Unix timestamps (PR #127)
- -Chat bubble overflow — Long URLs in AI messages no longer break layout (PR #130)
- -Preview text quality — Sentence-boundary-aware truncation replaces raw substring for article previews (PR #131)
- -RAG fallback quality — Zero-result fallback now uses curated search tiers (excludes archive/irrelevant) (PR #131)
- -Header search bar — Hidden on pages that already have their own search input to avoid duplication (PR #131)
Other Changes
- -Ten PRs merged to staging since v0.16.0, covering UX improvements, search deduplication, security fixes, and code quality.
- -SSRF fix — Admin test-url endpoint now validates URLs against a domain allowlist (PR #133)
- -URL sanitization — Geo-filter rewritten with proper hostname parsing (PR #133)
- -106 SonarCloud issues resolved — Code quality sweep across 39 files (PR #134)
- -4 CodeQL alerts resolved — Addressed all open code scanning findings (PR #134)
- -28 nosemgrep suppressions — Reviewed and annotated false-positive Semgrep alerts (PR #133, #134)
- -Mobile tap targets — All interactive elements meet 44×44px minimum for accessibility (PR #133)
- -Staging/production split — develop branch deploys to staging.needhamnavigator.com; main deploys to production. PRs auto-merge to develop; production promotion requires manual merge (PR #125)
- -CI hardening — Updated CodeQL and Semgrep workflows, improved branch protection (PR #134, #135)
Parallel Feature Build: Chat UX, PWA, Admin Dashboard v2, Events Scraper
New Features
- -Chat UX improvements (PR #118) — Conversation history drawer (localStorage, max 20 per town, FIFO), Share button (copies Q&A to clipboard), follow-up suggestion chips (context-aware: permits, schools, taxes, etc.), New Chat button
- -Progressive Web App (PWA) (PR #119) — Web app manifest for "Add to Home Screen" installability, service worker with network-first navigation and cache-first static assets, branded offline fallback page, PWA icons (192px and 512px)
- -Admin Dashboard: Search Analytics & Content Quality (PR #120) — Two new admin tabs: Search Analytics (total queries, avg latency, zero-result rate, confidence distribution, top 20 queries, content gaps) and Content Quality (documents by domain, stale documents, freshness distribution). New /api/admin/stats endpoint.
- -Events Calendar scraper (PR #121) — Cheerio-based scraper for needhamma.gov calendar and CivicAlerts pages. Supports --dry-run flag. Upserts to content_items with category: 'events'. Includes SSRF protection (host allowlist) and Supabase migration for source config.
Other Changes
- -Four features built and merged in parallel from independent branches.
- -New: src/lib/chat-history.ts — localStorage conversation persistence
- -New: src/components/ChatHistory.tsx — Slide-in drawer component
- -New: public/manifest.json, public/sw.js, public/offline.html — PWA assets
- -New: src/app/api/admin/stats/route.ts — Admin stats API
- -New: scripts/scrape-events.ts — Events scraper (469 lines)
- -New: supabase/migrations/20260222_add_events_scraper_source.sql
- -Modified: src/components/TownChatPage.tsx — Chat UX features + DRY refactoring
- -Modified: src/app/admin/page.tsx — Search Analytics + Content Quality tabs
- -Modified: src/app/layout.tsx — PWA manifest link, meta tags, SW registration
- -Modified: src/lib/i18n.tsx — 13 new i18n keys (chat + events) in en/es/zh
- -Modified: next.config.mjs — Added events to rewrite path regex
- -All 4 PRs passed build, CodeQL, Semgrep, and 19 Playwright E2E tests
Infra: Migrate Vector Search from Pinecone to Upstash Vector
Other Changes
- -Pinecone vector count (~244K vectors) exceeded the free tier limit. Rather than pay $50/month for Pinecone, vector search is migrated to Upstash Vector — a serverless vector DB with pay-as-you-go pricing (~$0.50/month for our usage). Supabase pgvector was considered but rejected because Supabase is already near its 500 MB free tier limit and can't absorb ~1.46 GB of embeddings.
- -Same 2-query pattern — Query → OpenAI embed → Upstash Vector query (returns IDs + scores) → Supabase fetch (text + metadata) → merge results. Drop-in replacement for Pinecone with identical data flow.
- -Two namespaces: chunks (document_chunks) and content (content_items) — same as Pinecone
- -Minimal metadata in Upstash (for filtering only): { town_id, relevance_tier } for chunks, { town_id, category } for content
- -Supabase remains source of truth for all text, metadata, and structured data
- -Vector embeddings remain at 1536 dimensions using text-embedding-3-large
- -Upstash uses DiskANN algorithm (~0.95 recall, comparable to Pinecone's HNSW)
- -New: src/lib/upstash-vector.ts — Upstash Vector SDK wrapper with backward-compatible function names (queryPinecone, upsertToPinecone, deleteFromPinecone)
- -New: Filter builder converts Pinecone filter syntax ($eq, $in, $ne) to Upstash SQL-like syntax
- -Deleted: src/lib/pinecone.ts — Pinecone client wrapper
- -Deleted: scripts/migrate-to-pinecone.ts — obsolete migration script
- -Modified: src/lib/rag.ts — import swap only (1 line)
- -Modified: src/lib/connectors/runner.ts — import swap only
- -Modified: scripts/embed.ts, scripts/re-embed.ts, scripts/smoke-test.ts, scripts/cleanup-url-duplicates.ts — import swap only
- -Modified: scripts/re-embed-content.ts — updated to upsert to Upstash Vector instead of Supabase embedding column
- -Modified: scripts/classify-documents.ts — removed Pinecone metadata update (Supabase-only; re-embed refreshes Upstash metadata)
- -Modified: all test files — mock target changed from @/lib/pinecone to @/lib/upstash-vector
- -Swapped dependency: @pinecone-database/pinecone → @upstash/vector
- -New env vars: UPSTASH_VECTOR_REST_URL, UPSTASH_VECTOR_REST_TOKEN
- -Removed env var: PINECONE_API_KEY
- -Pinecone: $0–50/month → $0/month (eliminated)
- -Upstash Vector: ~$0.50/month (PAYG: $0.25/GB storage + $0.40/100K requests)
- -Re-embedding: ~$0.05 one-time cost via OpenAI API
- -1. Create Upstash Vector index at upstash.com (1536 dims, cosine metric)
- -2. Set UPSTASH_VECTOR_REST_URL and UPSTASH_VECTOR_REST_TOKEN in .env.local and Vercel Dashboard
- -3. Re-embed documents: npx tsx --env-file=.env.local scripts/re-embed.ts --town=needham
- -4. Re-embed content items: npx tsx --env-file=.env.local scripts/re-embed-content.ts
- -5. Remove PINECONE_API_KEY from Vercel Dashboard and .env.local
- -6. Delete the Pinecone index to stop any billing
Search Quality: Tiered Relevance Filtering + Document Deduplication
New Features
- -Relevance tier classification — All 244K+ documents are now classified into tiers: primary (Needham-specific), regional (Norfolk County/MBTA), state (mass.gov), supplementary (utilities/reviews), archive (historical meeting minutes), and irrelevant (wrong towns). Default search returns only primary + regional content.
- -Smart tier expansion — Queries about state-level topics (taxes, RMV, MassHealth, unemployment, building codes, etc.) automatically expand search to include mass.gov and state content.
- -Zero-result fallback — If no results match within the default tiers, search automatically expands to all tiers so users always get an answer.
- -Document-level deduplication — Search results now show each page only once (best-matching chunk), eliminating the "same page 5 times" problem caused by multiple chunks per document.
- -URL canonicalization — Shared utility normalizes URLs (force https, strip www, strip trailing slash, remove tracking params) to properly deduplicate pages ingested via http/https/www URL variants.
- -Source pill dedup — Chat source pills now deduplicate by canonical URL instead of title, preventing duplicate source links.
Other Changes
- -New relevance_tier column on documents table with CHECK constraint and index
- -New canonical_url column on documents table for deduplication
- -Vector metadata now includes relevance_tier for query-time filtering
- -Query tier routing: getSearchTiers(query) detects state-level queries via 30+ regex patterns
- -New: src/lib/url-canonicalize.ts — shared URL canonicalization
- -New: src/lib/relevance-classifier.ts — domain-to-tier mapping + archive detection
- -New: src/lib/query-tier-router.ts — query intent detection for tier expansion
- -New: scripts/classify-documents.ts — backfill script for relevance_tier (Supabase + Pinecone)
- -New: scripts/cleanup-url-duplicates.ts — URL duplicate cleanup script
- -New: supabase/migrations/20260222000000_add_canonical_url.sql
- -New: supabase/migrations/20260222000001_add_relevance_tier.sql
- -Modified: src/lib/rag.ts — tier filtering in vectorSearch, document-level dedup, fallback expansion
- -Modified: src/app/api/search/route.ts — uses shared canonicalizeUrl for dedup
- -Modified: scripts/embed.ts — includes relevance_tier in Pinecone metadata
- -Modified: src/lib/connectors/runner.ts — includes relevance_tier in Pinecone metadata
- -Modified: scripts/re-embed.ts — includes relevance_tier in Pinecone metadata
- -Modified: scripts/scraper-config.ts — removed wrong-town domains from allowedDomains
Infra: Migrate Vector Search from Supabase to Pinecone
New Features
- -Pinecone vector search — All vector similarity search now runs on Pinecone serverless free tier instead of Supabase pgvector. This resolves the storage crisis (5 GB / 0.5 GB free tier limit) that was causing disk IO throttling and extremely slow search/chat responses.
- -Migration script — scripts/migrate-to-pinecone.ts bulk-exports all 244K vectors from Supabase to Pinecone with progress logging, batch processing, and rate limiting.
Other Changes
- -Search flow: Query → OpenAI embed → Pinecone query (vector similarity) → Supabase fetch (chunk text + metadata) → results
- -Ingestion flow: Generate embeddings → upsert vectors to Pinecone → store text/metadata in Supabase (no embedding column)
- -Vector embeddings remain at 1536 dimensions using text-embedding-3-large — no quality reduction
- -Supabase retains all structured data (chunk_text, metadata, documents, content_items)
- -New: src/lib/pinecone.ts — Pinecone client with query, upsert, and delete helpers
- -New: scripts/migrate-to-pinecone.ts — bulk vector export script
- -New: supabase/migrations/20260221000002_drop_embeddings.sql — drops embedding columns/indexes after migration (run manually)
- -Modified: src/lib/rag.ts — vectorSearch() and vectorSearchContentItems() use Pinecone
- -Modified: scripts/embed.ts — writes vectors to Pinecone, stores null embedding in Supabase
- -Modified: src/lib/connectors/runner.ts — writes content_item vectors to Pinecone
- -Modified: scripts/re-embed.ts — re-embeds to Pinecone
- -Modified: scripts/smoke-test.ts — uses Pinecone for test queries
- -New dependency: @pinecone-database/pinecone v7
- -New env var: PINECONE_API_KEY
- -Supabase: ~5,010 MB → ~490 MB (under 500 MB free tier)
- -Pinecone: 0 → ~1.46 GB (under 2 GB free tier)
- -Combined cost: $0/month
Fix: Supabase Storage Bloat Prevention
Bug Fixes
- -Tighter data retention — reduced retention periods to prevent PostgreSQL dead-tuple bloat that caused the database to balloon from ~50 MB actual data to 5 GB on the free tier:
- -search_telemetry: 30 days → 7 days
- -api_costs: 30 days → 7 days
- -ingestion_log: 90 days → 14 days
- -conversations: added 30-day retention (was unbounded)
- -feedback: added 30-day retention (was unbounded)
- -Reduced telemetry write frequency — embedding cost sampling reduced from 20% to 5%, search telemetry from 20% to 10%, cutting database write IO by ~60%
Other Changes
- -New migration: supabase/migrations/20260221000001_tighten_retention.sql — updated cleanup_old_data() function
- -Modified: src/lib/cost-tracker.ts — embedding cost sampling 20% → 5%
- -Modified: src/lib/telemetry.ts — search telemetry sampling 20% → 10%
Fix: Supabase Disk IO Budget Protection + Content API Pagination
Bug Fixes
- -Content API pagination bug — /api/content was returning total: filtered.length (post-geo-filter count) instead of the true database count, causing the News page to report wrong totals and breaking "load more" pagination logic
- -Cron job disk IO protection — daily cron now wraps each step (monitor, ingest, generate) with per-step timeouts (90s/120s) and 3-second cooldown delays between steps to prevent exhausting the Supabase Nano tier's 30-minute daily burst IO budget
- -Connector IO throttling — 2-second delay between sequential connector runs to avoid disk IO spikes during ingestion
- -Keep-alive ping — lightweight SELECT query at the start of the daily cron prevents Supabase free-tier auto-pause (7-day inactivity threshold)
Other Changes
- -Modified: src/app/api/cron/daily/route.ts — added withTimeout() wrapper, IO cooldowns, keep-alive ping
- -Modified: src/lib/connectors/runner.ts — added inter-connector cooldown delay
- -Modified: src/app/api/content/route.ts — fixed total and hasMore in JSON response
Geographic Content Filter + Unified News Experience
New Features
- -Unified News page — Articles and external news are now combined into a single feed at /<town>/articles, replacing the separate News and Articles pages. Content is merged from both the AI article generator and external news scrapers, sorted by date.
- -Source filter — filter the unified feed by All Sources, AI Articles, Patch, Observer, Needham Local, or Town of Needham
- -Geographic relevance filter (src/lib/geo-filter.ts) — keyword-based pre-filter that blocks off-topic content (e.g. Connecticut articles) before it reaches the AI pipeline. Uses allowlists (Needham + 10 neighbors + Boston metro), blocklists (49 states, distant cities), and category-specific scope rules:
- -Needham only: government, schools, public safety, development
- -Metro area OK: community, events, dining, news
- -Locality boost in search — RAG re-ranking now includes a 0.15 weight locality factor, boosting Needham-specific content and demoting distant-location content
- -Pipeline health endpoint (/api/debug/pipeline) — diagnostic endpoint reporting source config status, content item counts by source, article counts by type, and last daily brief date (protected by CRON_SECRET)
Bug Fixes
- -Daily briefing no longer includes off-topic geographic content — geo-filter integrated into article generator prompts and daily brief generation
- -Header navigation simplified — separate "News" and "Articles" links replaced with a single "News" entry
- -/[town]/news redirects to /[town]/articles — old URL still works
Other Changes
- -Article generator now runs geographic pre-filter before LLM calls (saves API costs on obviously irrelevant content)
- -Content API (/api/content) post-filters geographically irrelevant items
- -Daily cron enhanced with per-connector diagnostic logging (items found, upserted, errors per connector)
- -URL pattern filters added to Observer and Needham Local scrapers to restrict to on-domain content
- -New file: src/lib/geo-filter.ts — geographic relevance filter
- -New file: src/app/api/debug/pipeline/route.ts — pipeline health endpoint
- -Modified: src/lib/article-generator.ts, src/lib/rag.ts, src/app/api/content/route.ts, src/app/api/cron/daily/route.ts, scripts/seed-sources.ts
- -Modified: src/components/Header.tsx, src/components/SearchHomePage.tsx, src/app/[town]/articles/page.tsx, src/app/[town]/news/page.tsx
Fix: Search & Chat Resilience — Cold Start Timeouts, Retry Logic, Non-Fatal Text Search
Bug Fixes
- -Search results now load reliably on cold starts — previously, Vercel serverless cold starts caused /api/search to return HTTP 500 (heavy dependency loading took ~38s, exceeding the default 10s timeout). Extended maxDuration to 60s on both search and chat API routes.
- -Text search failure no longer kills entire search — when Supabase full-text search hit a statement timeout, Promise.all rejected and discarded working semantic (vector) search results. Text search is now non-fatal — if it fails, semantic results still come through.
- -Client-side retry on failed API calls — both search and chat UI now retry once after a 2-second delay when the API returns a non-200 response, transparently handling cold-start transients instead of silently showing empty results.
Other Changes
- -Lazy Cohere SDK loading — CohereClient is now loaded on first use via require() instead of at module import time, reducing cold start payload
- -Health endpoint + keep-alive cron — new /api/health endpoint warms the Supabase connection; hourly Vercel cron keeps functions warm to prevent cold starts
- -src/app/api/search/route.ts — added maxDuration = 60
- -src/app/api/chat/route.ts — added maxDuration = 60
- -src/components/SearchHomePage.tsx — retry-once logic for /api/search and /api/chat fetches
- -src/components/TownChatPage.tsx — retry-once logic for /api/chat fetch
- -src/lib/rag.ts — lazy getCohereClient(), non-fatal textSearchChunks in hybridSearch()
- -src/app/api/health/route.ts — new lightweight health check endpoint
- -vercel.json — added hourly /api/health cron
Post-Deploy Polish — Source Display, Nav Cleanup, Homepage Live Widgets
Bug Fixes
- -Article source links now display human-readable names (e.g. "Needham Observer") instead of raw source type tags like needham:observer-news
- -Article sources dropdown shows all source URLs instead of limiting to 3; count no longer says "3 of 5"
- -Source type labels in article headers now show "News Sources", "Meeting Minutes", etc. instead of raw database values
- -Article generator skips non-article URLs (author pages, tag pages, category pages) to prevent broken source citations
- -Admin Sources tab — new POST /api/admin/sources/seed endpoint populates the empty sources table from the 140+ hardcoded crawl config entries
Other Changes
- -Language selector moved from header navigation to footer (standard placement, less nav clutter)
- -"Permits" nav label renamed to "Permits & Zoning" with i18n translations (en/es/zh)
- -"Right Now in Needham" live widgets — new homepage section between articles and Browse by Topic showing compact cards for Weather (NWS API), Transit (MBTA API), and Community safety status; each links to its full page; includes loading skeletons and graceful API failure fallbacks
- -Browse by Topic cards reordered by likely user frequency: Schools, Taxes, Permits, Trash, Transportation, Recreation
- -"Ask a Question" CTA button made more prominent with larger size and subtle shadow
- -Transportation topic card now shows "Live commuter rail, parking, roads" description
Community Feature Pages — Events, Weather, Safety, Transit, Dining, Zoning
New Features
- -Events page (/<town>/events) — Community events, meetings, and activities fetched from the content API with date/time/location display, pagination, and "Ask About Events" fallback
- -Weather page (/<town>/weather) — Live weather conditions and 7-day extended forecast from the National Weather Service API using the town's lat/lng coordinates; shows current temp, wind, humidity, and NWS icons
- -Safety page (/<town>/safety) — Emergency 911 banner, auto-detected Police & Fire quick-dial cards from town config departments, and safety updates from the content feed
- -Transit page (/<town>/transit) — Live MBTA API integration using transit_route from town config; shows service alerts and upcoming departure schedule with stop names and directions
- -Dining page (/<town>/dining) — Local restaurant/eatery listings from the content API in a 2-column grid with address, hours, and ratings metadata
- -Zoning page (/<town>/zoning-map) — Zoning district info with Google Maps embed (when API key configured), 4 interactive topic cards that open chat with pre-filled zoning questions
- -Header "More" dropdown — New dropdown menu in the navigation bar with icons for all 6 new feature pages, gated by existing feature flags (enableEvents, enableWeather, enableSafety, enableTransit, enableDining, enableZoningMap)
- -i18n support — All 6 new navigation labels translated in English, Spanish, and Chinese
Other Changes
- -All pages follow established patterns: gradient hero, content API integration, loading skeletons, error/retry states, empty states with chat fallback
- -Header refactored with shared NAV_LINK_CLASS constant and click-outside-to-close dropdown
- -Weather uses free NWS API (api.weather.gov) — no API key required
- -Transit uses free MBTA V3 API (api-v3.mbta.com) — no API key required
- -All 6 pages statically generated for both needham and mock-town
RSS/External News Integration — Unified Community Platform
New Features
- -External news ingestion — Needham Patch, Needham Observer, and Needham Local are scraped every 4 hours via the connector framework; content stored in content_items with vector embeddings
- -AI article summaries — summarizeExternalArticle() generates reader-friendly AI summaries from external news; inserted as ai_summary articles with source attribution
- -Automated article generation cron (/api/cron/generate) — daily cron at 10 AM UTC generates articles from new documents + summarizes external news + creates daily brief
- -Content items in RAG search — vectorSearchContentItems() searches content_items in parallel with document_chunks; external news surfaces in both search and chat with a 0.95x ranking penalty so official content ranks higher
- -Ingest cron — /api/cron/ingest runs every 4 hours on Vercel to keep external sources fresh
- -Patch RSS source config — added needham:patch-rss (disabled — feed returns 404; scrape connector handles Patch)
Other Changes
- -40 external content items ingested (15 Patch, 15 Observer, 10 Needham Local)
- -20 AI Summary articles generated from external news
Real Article Generation — Nuke Fake Seed Data
New Features
- -Real article generation (src/lib/article-generator.ts rewritten) — generates articles exclusively from ingested documents in the Supabase documents table; every article has a real source URL; low-confidence results are discarded
- -Nightly GitHub Action (.github/workflows/generate-articles.yml) — runs generate-articles.ts at 5 AM ET every day; can also be triggered manually from GitHub Actions UI
- -Admin API (POST /api/articles/generate) — admin-authenticated endpoint to trigger article generation on demand; accepts { type: 'meeting_minutes' | 'public_record' | 'external' | 'daily_brief' | 'all' }
- -CLI generation script (scripts/generate-articles.ts) — standalone script for manual runs or CI; exits 0 on success, 1 on any errors
- -Delete-seed script (scripts/delete-seed-articles.ts) — permanently removes all articles from the database; used once to purge fake data
Bug Fixes
- -Deleted all fake seed articles from production — fabricated articles about events that never happened, people that don't exist, and 404 source URLs are gone
- -Removed scripts/seed-articles.ts — the script that created fake data is deleted from the repo
- -Fixed feedback API bug — article detail page was sending POST with { feedback } but API expects PATCH with { type }; frontend now correctly sends PATCH { type }
Other Changes
- -Daily brief page — empty state now says "Today's brief hasn't been generated yet. Check back after 5 AM." and a Sources section with clickable links is shown below each brief when sources exist
- -Homepage articles section — entire "Latest Articles" section (header + grid) is now hidden when no articles exist, instead of showing a floating orphan header
- -Articles page — empty state updated to: "Articles are generated daily from Needham's public records, meeting minutes, and local news. Check back soon!"
- -Article detail page — source_type badge displayed alongside the "Sources (N)" count header; source links now have break-all for long URLs
AI Articles Hub Frontend
New Features
- -Articles list page (/<town>/articles) — Browse all AI-generated articles and summaries with category and content-type filters, 12-per-page load-more, and an empty state
- -Article detail page (/<town>/articles/[id]) — Full article view with proper markdown rendering (react-markdown), AI disclaimer banner, collapsible sources, thumbs up/down feedback, "Ask about this" button that pre-populates chat, and related articles sidebar
- -Daily Brief page (/<town>/daily-brief) — Today's digest rendered as formatted markdown, with 7-day accordion history of previous briefs
- -Homepage articles section — Below the search hero, the home page now shows a Daily Brief Banner (when a brief exists for today) and a "Latest Articles" grid of 6 featured articles, with a "View all articles" link
- -Header nav — Added "Articles" link (gated behind enableNews feature flag) pointing to /articles
Other Changes
- -ArticleCard — Card component with content-type badge, category tag, title, summary, source, and relative timestamp; supports grid and list variants
- -DailyBriefBanner — Gradient banner for the homepage showing today's brief summary with bullet points and a link to the full brief
- -ArticleFilters — Category dropdown + content-type pill filters for the articles list page
- -ArticleSkeleton / DailyBriefSkeleton — Animated loading placeholders
- -Installed react-markdown and @tailwindcss/typography for proper markdown rendering with prose styles
- -scripts/seed-articles.ts — Dev seed script for 8 realistic sample articles (run with npx tsx --env-file=.env.local scripts/seed-articles.ts)
Search-First UI Skin with Floating Chat
New Features
- -Search-first UI mode — Alternative interface optimized for document discovery and quick lookup
- -Large search hero with instant results from /api/search endpoint
- -AI answers load asynchronously above search results
- -Cached answers display instantly with green "Instant" badge
- -Browse by topic cards and popular questions for discovery
- -Floating chat panel — Persistent bottom-right chat bubble that opens to a side panel
- -"Ask about this" buttons on search results pre-populate the chat
- -"Ask follow-up" buttons on AI answers open chat with context
- -Suggestion chips before first message
- -Full conversation history maintained while panel is open
- -UI mode toggle — Controlled via uiMode: 'classic' | 'search' flag in town config
- -Classic mode (default): Chat-first interface (existing HomePage)
- -Search mode: Search-first interface (new SearchHomePage)
- -Skin router in [town]/page.tsx selects appropriate component
Other Changes
- -SearchHomePage — Main search-first homepage with hero, topic cards, results
- -SearchResultCard — Individual search result with similarity score, "Ask about this" button
- -AIAnswerCard — Four states: loading, cached (instant), loaded (streamed), prompt (opt-in)
- -FloatingChat — Bottom-right FAB + slide-up chat panel with streaming responses
- -src/types/search.ts — Shared TypeScript interfaces for search API
- -src/lib/stream-parser.ts — Reusable SSE stream parser utility
- -Refactored TownChatPage to use new parseStreamResponse() utility (cleaner, more maintainable)
- -Search results show department tags, dates, and similarity percentages
- -Mobile-responsive floating chat (full-width with margins on small screens)
- -Graceful fallback if /api/search endpoint doesn't exist yet
- -Skin routing logic in src/app/[town]/page.tsx checks townConfig.feature_flags.uiMode
- -SSE parsing logic extracted from TownChatPage into reusable utility
- -Both skins share ChatBubble, SourceChip, and other components
- -No changes to existing classic skin (HomePage) — fully backward compatible
Remove AI Disclaimer Preamble from Chat Responses
Other Changes
- -Chat responses no longer start with an AI accuracy disclaimer — the UI already displays a verification badge and disclaimer footer, so the redundant LLM-generated preamble ("This tool uses AI and may provide inaccurate information...") has been removed
- -System prompt now explicitly instructs the LLM to jump straight into answering the question
- -Removed FIRST-MESSAGE DISCLAIMER section from system prompt in src/lib/prompts.ts
- -Removed getFirstMessageDisclaimer() usage from buildChatSystemPrompt()
- -Added rule 7: "Do NOT start your response with a disclaimer or preamble"
- -Updated integration tests to verify new behavior
SEO Fundamentals & Branded 404 Page
New Features
- -robots.txt: Allows all crawlers, blocks /admin and /api/ paths, points to sitemap
- -Dynamic sitemap.xml: Auto-generates URLs from town configs — adding a new town to config/towns.ts automatically adds its routes to the sitemap. Feature-flagged routes (e.g., news) included only when enabled.
- -Open Graph & Twitter meta tags: Social media sharing now shows title, description, and site name instead of blank previews
- -Branded 404 page: Clean "Page not found" page with the Navigator logo and a link back to the homepage
Other Changes
- -src/app/robots.ts — Next.js Metadata API robots file
- -src/app/sitemap.ts — Dynamic sitemap from TOWN_CONFIGS, skips test towns
- -src/app/layout.tsx — Enhanced metadata with metadataBase, OG, Twitter, and robots tags
- -src/app/not-found.tsx — Styled 404 page matching site design language
Multi-Tenant System Prompt — Remove Hardcoded Needham Data
Bug Fixes
- -System prompt no longer hardcodes Needham-specific facts (Transfer Station address, MBTA stations, sticker prices, Town Hall phone). All town-specific facts now come from RAG retrieval, making the prompt fully multi-tenant.
- -Fallback "call Town Hall" message now uses the correct phone number from the town's config instead of a hardcoded (781) 455-7500
- -Synonym expansions no longer embed factual data (addresses, dates) — only genuine informal→formal query mappings remain
Other Changes
- -buildChatSystemPrompt() now accepts townName and townHallPhone parameters
- -FIRST_MESSAGE_DISCLAIMER constant replaced with getFirstMessageDisclaimer(townHallPhone) function
- -Chat route looks up town config via getTownById() and passes dynamic values to prompt builder
- -Removed "1421 Central Avenue" and "first Monday in May" from synonym expansions in synonyms.ts
OpenAI Cost Tracking & Admin Dashboard
New Features
- -Automatic cost logging: Every chat API call now logs token usage (input/output tokens) and estimated USD cost to the api_costs table — fully async, does not slow down responses
- -Cost monitor dashboard: New "Costs" tab in /admin with:
- -Today / This Week / This Month summary cards
- -Projected monthly cost (extrapolated from daily average)
- -Average cost per query, total requests, total tokens
- -Daily cost bar chart (last 30 days) with hover tooltips
- -Cost-by-model breakdown with progress bars
- -Friendly empty state when no data exists yet
- -Cost API endpoint: GET /api/admin/costs returns cost summary JSON (admin-protected)
- -Model pricing constants: MODEL_COSTS in src/lib/cost-tracker.ts maps all supported models to per-token pricing for easy updates
Other Changes
- -Migration: 002_cost_tracking.sql — creates api_costs table with indexes on created_at, endpoint, and town_id
- -Uses Vercel AI SDK v6 result.usage promise for non-blocking token tracking
- -Gracefully handles missing api_costs table (shows empty state, no errors)
- -Fire-and-forget logging pattern — cost tracking failures never affect chat responses
Embedding Model Migration: text-embedding-3-small → text-embedding-3-large
Bug Fixes
- -Fixed broken RAG retrieval: OpenAI changed text-embedding-3-small to return 384 dimensions instead of 1536 after a service incident, causing "different vector dimensions" errors in pgvector search
- -Switched to text-embedding-3-large (with dimensions: 1536) which correctly respects the dimensions parameter — same vector size, more capable model
- -Re-embedded all 2,068 chunks in Supabase with the new model — full RAG pipeline restored with high-confidence answers and source citations
Configurable Chat Model + Upgrade to GPT-5 Nano
New Features
- -Admin model selector: New Settings tab in /admin dashboard lets you swap the chat model without redeploying. Choose from GPT-5 Nano, GPT-5 Mini, GPT-4o Mini, or GPT-4.1 Mini with pricing info displayed inline.
- -Default model → GPT-5 Nano: 3x cheaper than GPT-4o Mini ($0.05 vs $0.15/M input), newer model, 400K context window. Perfect for municipal Q&A.
- -Model stored in Supabase: Chat model preference persisted in towns.config JSONB column — changes take effect immediately, no migration needed.
API Error Handling Hardening
Bug Fixes
- -Chat no longer returns 500 when OpenAI embedding API fails: Wrapped RAG retrieval in its own try/catch so embedding or Supabase failures gracefully degrade to the "call Town Hall" fallback response instead of crashing
- -Content API handles missing content_items table: If the migration hasn't been applied, /api/content returns an empty result set (200) instead of a 500 error
- -Added console.error logging to both /api/chat and /api/content: Future errors will now appear in Vercel runtime logs for faster diagnosis
Data Quality Cleanup
Bug Fixes
- -Department classification 4x improvement: Title-based fallback patterns classify 47% of pages (236/500) into 24 departments — up from 12% (58/500) using URL patterns alone
- -Duplicate URL elimination: Trailing-slash normalization in scraper prevents duplicate pages (e.g., /page vs /page/)
- -Stale date prevention: System prompt now includes today's date so the AI won't present past events as upcoming
- -Build fix: Resolved pre-existing type errors in connector framework (runner.ts, scraper.ts)
Other Changes
- -Full re-scrape and re-ingest with improved department metadata
- -500 documents, 2,068 chunks, 0 errors
Community Platform: Connector Framework + Local News Aggregation
New Features
- -Pluggable Connector Framework: Config-driven data source architecture — add new content sources without code changes
- -Pluggable Connector Framework: Generic connectors for RSS feeds, iCal calendars, and web scraping
- -Pluggable Connector Framework: Connector registry with factory pattern for extensibility
- -Pluggable Connector Framework: Automated ingestion runner with schedule tracking, error handling, and deduplication
- -Pluggable Connector Framework: Content normalization into unified content_items table with pgvector embeddings
- -Pluggable Connector Framework: source_configs table stores per-town connector configuration (URL, selectors, schedule)
- -Pluggable Connector Framework: Cron API endpoint (/api/cron/ingest) for automated daily/hourly ingestion
- -Local News Feed (/<town>/news)
- -Pluggable Connector Framework: Aggregates articles from 3 local news sources:
- -Pluggable Connector Framework: Needham Patch — 15 articles ingested
- -Pluggable Connector Framework: Needham Observer — 15 articles ingested
- -Pluggable Connector Framework: Needham Local — 10 articles ingested
- -Pluggable Connector Framework: Filter by source with one-click filter chips
- -Pluggable Connector Framework: Article cards with source badge, relative time, summary, and external link
- -Pluggable Connector Framework: Load more pagination for browsing older articles
- -Pluggable Connector Framework: News link in header navigation (feature-flag gated)
- -Content API (/api/content)
- -Pluggable Connector Framework: Generic content query endpoint: filter by town, category, source
- -Pluggable Connector Framework: Pagination with offset/limit and total count
- -Pluggable Connector Framework: Excludes expired content automatically
- -AI Content Generator (src/lib/ai/content-generator.ts)
- -Pluggable Connector Framework: Article summarization via GPT-4o-mini (2-sentence summaries)
- -Pluggable Connector Framework: Daily digest generation from recent content items
- -Pluggable Connector Framework: Weekend event preview generation (for future events integration)
- -News Widget (src/components/dashboard/NewsFeedWidget.tsx)
- -Pluggable Connector Framework: Compact 5-headline widget for future dashboard integration
- -Pluggable Connector Framework: "View all" link to full news page
Other Changes
- -Migration: 004_content_platform.sql — creates content_items, source_configs, and generated_content tables with pgvector HNSW indexing, RLS policies, and semantic search function
- -40 news articles ingested from initial scrape of 3 Needham news sources
- -Expanded TownConfig with new feature flags: enableNews, enableEvents, enableDining, enableSafety, enableTransit, enableWeather, enableDashboard
- -Added location (lat/lng) and transit_route fields to town config
- -New scripts: scripts/seed-sources.ts (source config seeder)
- -Connector source files: src/lib/connectors/ (types, registry, runner, rss, ical, scraper)
- -/<town>/news — local news feed
- -/api/content — content query API
- -/api/cron/ingest — automated ingestion endpoint
- -2 additional static generation pages
Full Re-Scrape & Data Quality Validation
Other Changes
- -500 pages scraped (up from 198 with Firecrawl — 152% increase)
- -2,068 chunks ingested (up from 1,023 — 102% increase)
- -47 PDFs discovered for future processing
- -Zero boilerplate across all 500 pages — no [Loading], CivicPlus footers, language pickers, or navigation menus
- -All old Firecrawl data cleared and replaced with clean custom scraper output
- -9 of 10 test questions returned meaningful answers
- -6 High confidence, 3 Medium confidence (vs mostly Low/Low-Medium before)
- -All citation titles are clean (no "Default" or "CivicPlus.CMS.FAQ" metadata)
- -All responses are conversational and human-sounding
- -Ingestion validation passed with zero errors
- -Zero orphaned chunks, zero documents without chunks
- -Department coverage across 13+ municipal departments
Custom Scraper — Firecrawl Replacement
New Features
- -Custom Web Scraper (scripts/scraper.ts)
- -Replaces Firecrawl API with a zero-cost custom scraper built on cheerio + @mozilla/readability + turndown
- -Purpose-built for CivicPlus municipal sites (needhamma.gov page structure)
- -Saves $16/month in Firecrawl API costs
- -Parallel page fetching with configurable concurrency and rate limiting
- -Content-hash based change detection for incremental scraping
- -Extracts clean markdown from HTML with boilerplate removal
- -Scraper Configuration (scripts/scraper-config.ts)
- -Centralized URL patterns, selectors, and exclusion rules for CivicPlus sites
- -Configurable content selectors, navigation exclusions, and PDF link discovery
- -Easy to extend for additional municipal site platforms
- -Clean Re-ingestion Script (scripts/reingest-clean.ts)
- -One-command pipeline to scrape → chunk → embed → upsert fresh data
- -Designed for full data refresh after switching scraper backends
- -Smoke Test (scripts/smoke-test.ts)
- -Quick validation that the scraper can reach the site, fetch pages, and extract content
- -Useful for CI and pre-deploy checks
Other Changes
- -scripts/crawl.ts — now a thin backward-compatible wrapper that delegates to scripts/scraper.ts
- -scripts/ingest.ts — updated to work with the new scraper output format
- -__tests__/scripts/crawl.test.ts — rewritten for the new scraper API
- -.env.example — removed FIRECRAWL_API_KEY, added scraper config vars
- -.gitignore — added scraper output/cache directories
- -Removed @mendable/firecrawl-js dependency
- -Added cheerio, @mozilla/readability, turndown, jsdom dependencies
AI Polish & Municipal Intelligence
New Features
- -Conversational AI Personality: Rewritten system prompt: warm, conversational tone like a friendly town clerk
- -Conversational AI Personality: Leads with direct answers, ends with natural follow-up questions
- -Conversational AI Personality: Understands informal language ("the dump", "cops", "can I build a deck")
- -Conversational AI Personality: Phone numbers are clickable (tel: links) in chat responses
- -Conversational AI Personality: No bracket citations or metadata in AI text — clean, natural prose
- -Smart Query Expansion (src/lib/synonyms.ts)
- -Conversational AI Personality: Two-tier synonym dictionary: 30+ universal entries + 9 Needham-specific entries
- -Conversational AI Personality: Automatic query expansion: "dump" → also searches "Transfer Station, solid waste, recycling"
- -Conversational AI Personality: Intent detection: "when is X open?" adds schedule/hours keywords to search
- -Conversational AI Personality: Department routing: matches queries to relevant town departments for reranking
- -Conversational AI Personality: Word-boundary matching prevents false positives (e.g., "street" won't match "tree")
- -Improved Source Citations: Clean citation pills: CivicPlus CMS metadata stripped from document titles
- -Improved Source Citations: Clickable source links open official town pages in a new tab
- -Improved Source Citations: Generic/untitled sources filtered out automatically
- -Improved Source Citations: Deduplication by URL, max 4 source pills per response
- -Improved Source Citations: Responsive overflow handling on mobile
- -Better Confidence Scoring: Now based on top similarity score (not average), more accurate for mixed-quality results
- -Better Confidence Scoring: Configurable thresholds via environment variables (CONFIDENCE_HIGH_THRESHOLD, CONFIDENCE_MEDIUM_THRESHOLD)
- -Better Confidence Scoring: New labels: "Verified from official sources" (high), "Based on town documents — verify for important decisions" (medium), "Limited information — contact the department directly" (low)
- -Better Confidence Scoring: Updated labels in all 3 languages (en/es/zh)
- -Smarter Retrieval Pipeline: Multi-query vector search: parallel searches with original + expanded queries
- -Smarter Retrieval Pipeline: Similarity floor (0.35) filters out noise chunks
- -Smarter Retrieval Pipeline: Multi-factor reranking: semantic similarity (60%), keyword overlap (20%), recency (10%), document authority (10%), plus department boost
- -Smarter Retrieval Pipeline: Source diversity: selects chunks from different documents to avoid redundancy
- -Smarter Retrieval Pipeline: Lowered match threshold to 0.30 for better recall
Bug Fixes
- -Fixed source format mismatch between API (document_title) and UI (title)
- -Edge case handling in system prompt (off-topic, ambiguous, multi-part questions)
Chat Bug Fix & Data Quality Improvements
Bug Fixes
- -Fixed "No response received" in chat UI: The SSE stream parser in TownChatPage.tsx expected 0: prefix but Vercel AI SDK v6 sends data: prefix — chat now correctly displays AI answers
- -Fixed confidence extraction: Confidence level now correctly extracted from nested object returned by API
Other Changes
- -Boilerplate stripping: Added stripBoilerplate() to scripts/chunk.ts — removes 15 patterns of CivicEngage CMS chrome (loading spinners, breadcrumbs, Google Translate picker, font probes, footer badges) before chunking
- -Oversized chunk safety: Added splitOversizedChunk() with 4-level recursive splitting (paragraphs → newlines → sentences → hard token split) to prevent embedding API errors
- -Lowered match threshold: Changed vector search threshold from 0.65 to 0.40 in chat API to return results for more queries
- -UI smoke test in checklist: Added browser-based UI verification as step 2 in the merge-to-main checklist in CLAUDE.md
Production-Grade Ingestion Pipeline
New Features
- -Comprehensive Crawl Coverage: Centralized URL registry with 40+ data sources from Research Report (config/crawl-sources.ts)
- -Comprehensive Crawl Coverage: Priority-based crawling (priority 1-5) with source categorization
- -Comprehensive Crawl Coverage: Incremental crawling with content-hash comparison (skips unchanged documents)
- -Comprehensive Crawl Coverage: Retry logic with exponential backoff (3 attempts, 1s/2s/4s delays)
- -Comprehensive Crawl Coverage: Enhanced exclusion patterns (search pages, login pages, binary formats)
- -Comprehensive Crawl Coverage: CLI options: --sources (crawl all sources), --high-priority (priority 4-5 only)
- -Accurate Token Counting: Replaced 4 char/token approximation with js-tiktoken for exact tokenization
- -Accurate Token Counting: Uses same tokenizer as OpenAI text-embedding-3-small model
- -Accurate Token Counting: Prevents chunks from exceeding embedding model limits (8191 tokens)
- -Accurate Token Counting: Precise overlap calculation with getLastNTokens() helper
- -Enhanced PDF Extraction: Heuristic-based complexity detection (replaces keyword-only approach)
- -Enhanced PDF Extraction: Analyzes: avg chars/page (<200 = scanned), table density (>10 = complex), extraction length (<100 = failed)
- -Enhanced PDF Extraction: Parallel processing with configurable concurrency limit (default: 3 PDFs in parallel)
- -Enhanced PDF Extraction: Extraction validation: flags multi-page PDFs with <100 chars extracted
- -Enhanced PDF Extraction: Warnings for fee schedules without tables, high non-printable character density
- -Content-Hash Change Detection: Replaced unreliable HTTP HEAD checks with full content-hash comparison
- -Content-Hash Change Detection: Separate tracking of last_crawled (every check) and last_changed (only when hash differs)
- -Content-Hash Change Detection: 0% false positives vs. ~50% with HTTP HEAD approach
- -Content-Hash Change Detection: Accurate staleness warnings based on actual content changes
- -Data Quality Validation (npm run validate)
- -Content-Hash Change Detection: Validates required metadata fields (document_title, document_type, document_url, etc.)
- -Content-Hash Change Detection: Detects duplicate chunks via content_hash
- -Content-Hash Change Detection: Verifies embedding dimensions (all 1536-dimensional)
- -Content-Hash Change Detection: Coverage report: identifies departments with <3 chunks (data gaps)
- -Content-Hash Change Detection: Checks for orphaned chunks (document_id references non-existent documents)
- -Content-Hash Change Detection: Documents without chunks report (pending processing)
- -Enhanced Metadata Tracking: Every chunk includes: chunk_index, total_chunks, content_hash
- -Enhanced Metadata Tracking: Department extraction from URL patterns
- -Enhanced Metadata Tracking: New database columns: last_crawled, last_changed
- -Enhanced Metadata Tracking: Performance indexes: content_hash, metadata GIN index, chunk_index
Other Changes
- -Database Migration: 003_enhance_metadata_tracking.sql adds last_crawled and last_changed columns, plus 5 performance indexes
- -New Scripts:
- -scripts/validate-ingestion.ts — post-ingestion quality validation
- -config/crawl-sources.ts — 40+ source URL registry
- -Package Updates:
- -js-tiktoken ^1.0.21 — exact token counting for OpenAI models
- -scripts/chunk.ts: Exact token counting, enhanced metadata (chunk_index, total_chunks, content_hash)
- -scripts/crawl.ts: Incremental crawling, retry logic, source registry integration
- -scripts/extract-pdf.ts: Heuristic complexity detection, parallel processing, validation
- -scripts/monitor.ts: Content-hash comparison instead of HTTP HEAD
- -package.json: Added validate script
- -200+ documents indexed (from 40+ sources)
- -100% token counting accuracy (vs. ±25% approximation)
- -0% change detection false positives (vs. ~50% with HTTP HEAD)
- -All chunks have complete metadata
- -Automated quality validation
All feature branches integrated into main
New Features
- -Permit Wizard (/[town]/permits)
- -Guided step-by-step workflow for 4 project types: deck, fence, renovation, addition
- -Decision tree asks 3 targeted questions per project type
- -Generates personalized permit summary: required permits, estimated fees, documents needed, timeline, department contacts, and tips
- -"Ask Navigator for More Details" links directly to chat with pre-filled question
- -Fully multi-tenant — adapts town name, department phone from town config
- -Full i18n support (English, Spanish, Chinese)
- -Chat Feedback Mechanism: Thumbs up/down on every AI response
- -Chat Feedback Mechanism: Optional comment field (up to 2000 characters)
- -Chat Feedback Mechanism: Posts to /api/feedback with session tracking
- -Chat Feedback Mechanism: Visual states: idle, submitting, submitted, error with auto-retry
- -Admin Dashboard (/admin)
- -Chat Feedback Mechanism: Real-time analytics: feedback trends, query volume, response quality
- -Chat Feedback Mechanism: System logs viewer with filtering
- -Chat Feedback Mechanism: Structured ingestion logging
- -Chat Feedback Mechanism: Cron sync endpoint (/api/cron/sync)
- -Multi-Tenant Architecture (/[town]/*)
- -Chat Feedback Mechanism: Dynamic [town] routing — all pages scoped per town
- -Chat Feedback Mechanism: Town configuration in config/towns.ts (brand colors, departments, feature flags)
- -Chat Feedback Mechanism: Per-town CSS theming via CSS custom properties
- -Chat Feedback Mechanism: TownProvider context for client components
- -Chat Feedback Mechanism: Legacy /chat redirects to /<default-town>/chat
- -Internationalization (i18n): 3 languages: English, Spanish, Chinese
- -Internationalization (i18n): Language toggle in header with localStorage persistence
- -Internationalization (i18n): Browser language auto-detection
- -Internationalization (i18n): All UI strings translated including permit wizard
- -Infrastructure: Town-scoped Row Level Security (Supabase migration 002_town_scoped_rls.sql)
- -Infrastructure: town_id parameter on feedback and department APIs
- -Infrastructure: Static page generation for all town routes
Other Changes
- -/ — root redirect
- -/[town] — town home page (Needham, Mock Town)
- -/[town]/chat — AI chat with feedback
- -/[town]/permits — permit wizard
- -/admin — admin dashboard
- -/chat — legacy redirect
- -11 API routes
Landing page with hero section, life situation tiles, popular questions, department directory
New Features
- -Landing page with hero section, life situation tiles, popular questions, department directory
- -Chat UI with typing indicators, source chips, confidence badges, follow-up suggestions
- -Markdown rendering in AI responses
RAG chat API with OpenAI embeddings
New Features
- -RAG chat API with OpenAI embeddings
- -Search endpoint
- -Admin document and ingestion management endpoints
- -Feedback collection API
Data ingestion pipeline: crawl, extract, chunk, embed
New Features
- -Data ingestion pipeline: crawl, extract, chunk, embed
- -Supabase schema and migrations
- -Next.js 14 project scaffold with Tailwind CSS
- -## v0.6.0 (In Progress)
- -Fun search loader with rotating status messages and town facts during search
- -Library events connector for Needham Public Library (64+ events)
- -Upstash vector embeddings enabled for library events
Bug Fixes
- -Fix header search bar visibility on homepage (prevent hydration flash)
- -Critical fix: connector embedding column bug that broke all ingestion (#152)
- -Transit: Eastern timezone fix + preferred stop selector (#148)
- -Content pipeline restored in GitHub Actions (connector ingest step restored) (#150, #149)