initial push

Dockerfile

@@ -0,0 +1,10 @@
+FROM python:3.13-slim
+ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
+WORKDIR /app
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+COPY app ./app
+COPY README.md .
+RUN mkdir -p /data
+EXPOSE 8000
+CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

README.md

@@ -0,0 +1,44 @@
+# What2Do
+
+A lightweight, local-first event aggregator. It crawls configured venue pages with CSS selectors or schema.org JSON-LD, normalizes events, and provides date/location/radius search.
+
+No LLM, ML, computer vision, browser automation, JavaScript build process, PostGIS, Redis, or external database is required.
+
+## Run
+
+```sh
+docker compose up --build
+```
+
+Open http://localhost:8000. The source admin is at http://localhost:8000/sources.
+
+Two example sources are installed automatically:
+
+- Dicey Riley's recurring weekly specials
+- La La La's dated event list
+
+Click **Crawl now** beside each source after startup. Crawls then run on each source's configured schedule while the app is running.
+
+## Extraction model
+
+Rules are relative to a repeating event container:
+
+- Event container, title, date, time, link, description, and location are CSS selectors.
+- Date parsing accepts a Python `strptime` format or flexible text parsing.
+- `json-ld` as the event container reads schema.org `Event` entries without CSS rules.
+- Weekly pages can create the next 12 occurrences from weekday labels.
+- The source's venue address is the fallback when event HTML omits a location.
+
+Use **Test extraction** to preview up to 20 records before saving. Static server-rendered HTML is intentionally used; sites that only render events with JavaScript are not supported in this low-overhead version.
+
+## Location search
+
+Addresses are geocoded and cached using OpenStreetMap's public Nominatim service. The search field also accepts `latitude,longitude` directly, and every source has optional coordinate fields for deterministic/offline venue positioning. Set a real contact in `NOMINATIM_USER_AGENT` before public/shared use and follow the [Nominatim usage policy](https://operations.osmfoundation.org/policies/nominatim/). For heavier use, configure or self-host a compatible geocoder.
+
+## Operational notes
+
+- SQLite data is persisted in the `what2do-data` Docker volume.
+- Fetches are limited to 5 MB and public HTTP(S) destinations to reduce SSRF risk.
+- The crawler does not bypass authentication or anti-bot systems.
+- Review each site's terms and `robots.txt`; only crawl sources you are permitted to index.
+- The MVP has no authentication. Bind it to localhost or place it behind an authenticated reverse proxy before exposing crawler administration to a network.

STRATEGIES.md

@@ -0,0 +1,95 @@
+# Generic extraction strategies
+
+Sources persist a JSON strategy. Runtime behavior is determined only by this configuration—there are no hostname-specific parsers.
+
+## HTML
+
+```json
+{
+  "mode": "html",
+  "request_url": "https://example.org/events",
+  "method": "GET",
+  "item_selector": ".event-card",
+  "title_selector": ".title",
+  "date_selector": ".date",
+  "time_selector": ".time",
+  "link_selector": "a.details",
+  "link_attribute": "href",
+  "description_selector": ".summary",
+  "location_selector": ".venue",
+  "date_format": ""
+}
+```
+
+Selectors may read attributes using `title_attribute`, `date_attribute`, `time_attribute`, `link_attribute`, `description_attribute`, and `location_attribute`. Use `:scope` to read an attribute from the event container itself.
+
+## JSON API
+
+```json
+{
+  "mode": "json",
+  "request_url": "https://example.org/api/sessions",
+  "method": "GET",
+  "items_path": "results.events",
+  "title_path": "name",
+  "date_path": "startDate",
+  "end_path": "endDate",
+  "description_path": "description",
+  "location_path": "venue.address",
+  "link_path": "url",
+  "disabled_path": "disabled"
+}
+```
+
+Paths use dot notation. Numeric path components index arrays.
+
+### Joining metadata
+
+```json
+{
+  "mode": "json",
+  "request_url": "https://example.org/api/sessions",
+  "items_path": "",
+  "date_path": "utcDate",
+  "link_path": "link",
+  "join_item_key": "movieId",
+  "join_title_path": "name",
+  "join_description_path": "summary",
+  "join_duration_path": "duration",
+  "join": {
+    "request_url": "https://example.org/api/movies",
+    "method": "GET",
+    "items_path": "",
+    "key_path": "id"
+  }
+}
+```
+
+## JSON containing HTML fragments
+
+```json
+{
+  "mode": "html_fragment",
+  "request_url": "https://example.org/ajax",
+  "method": "POST",
+  "body": {"action": "events", "venue": "123"},
+  "html_path": "html",
+  "item_selector": ".event-card",
+  "title_selector": ".title",
+  "date_selector": ".date",
+  "link_selector": "a",
+  "link_attribute": "href",
+  "page_parameter": "paged",
+  "page_start": 1,
+  "max_pages": 10,
+  "total_pages_path": "pages_total"
+}
+```
+
+## Safety limits
+
+- Request URLs still pass public-network SSRF validation.
+- Redirect destinations are validated for normal HTML fetches.
+- Responses default to a 5 MB limit.
+- Pagination is capped at 20 pages.
+- Browser-compatible headers are global; strategy headers may add endpoint-specific `Accept` or `Referer` values.

app/crawler.py

@@ -0,0 +1,287 @@
+import hashlib
+import ipaddress
+import json
+import re
+import socket
+from dataclasses import dataclass
+from datetime import datetime, timedelta
+from urllib.parse import urljoin, urlparse
+from zoneinfo import ZoneInfo
+
+import httpx
+from bs4 import BeautifulSoup
+from dateutil import parser as date_parser
+
+USER_AGENT = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36"
+BROWSER_HEADERS = {
+    "User-Agent": USER_AGENT,
+    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
+    "Accept-Language": "en-AU,en;q=0.9",
+    "Cache-Control": "no-cache",
+    "DNT": "1",
+    "From": "what2do local event index",
+}
+SOCIAL_WIDGET_MARKERS = ("instagram", "sbi_", "facebook", "social-feed", "tiktok")
+
+@dataclass
+class ExtractedEvent:
+    title: str
+    starts_at: datetime
+    ends_at: datetime | None = None
+    description: str = ""
+    url: str = ""
+    address: str = ""
+
+class CrawlError(Exception):
+    pass
+
+def validate_public_url(url: str) -> None:
+    parsed = urlparse(url)
+    if parsed.scheme not in {"http", "https"} or not parsed.hostname:
+        raise CrawlError("Only absolute http:// and https:// URLs are allowed.")
+    try:
+        infos = socket.getaddrinfo(parsed.hostname, parsed.port or (443 if parsed.scheme == "https" else 80))
+    except socket.gaierror as exc:
+        raise CrawlError(f"Host cannot be resolved: {exc}") from exc
+    for info in infos:
+        ip = ipaddress.ip_address(info[4][0])
+        if not ip.is_global:
+            raise CrawlError("URLs resolving to private, loopback, link-local, or reserved addresses are blocked.")
+
+def _http_client(timeout: int = 20) -> httpx.Client:
+    return httpx.Client(follow_redirects=True, timeout=timeout, headers=BROWSER_HEADERS)
+
+def fetch_html(url: str) -> str:
+    validate_public_url(url)
+    with _http_client() as client:
+        response = client.get(url)
+        response.raise_for_status()
+        validate_public_url(str(response.url))
+        content_type = response.headers.get("content-type", "")
+        if "html" not in content_type.lower():
+            raise CrawlError(f"Expected HTML but received {content_type or 'an unknown content type'}.")
+        if len(response.content) > 5_000_000:
+            raise CrawlError("Page exceeds the 5 MB crawl limit.")
+        return response.text
+
+def _text(node, selector: str) -> str:
+    if not selector:
+        return ""
+    found = node.select_one(selector)
+    return found.get_text(" ", strip=True) if found else ""
+
+def _link(node, selector: str, base_url: str) -> str:
+    found = node.select_one(selector) if selector else None
+    target = found or node
+    for attribute in ("href", "data-permalink", "data-url", "data-href"):
+        value = target.get(attribute)
+        if value:
+            return urljoin(base_url, str(value))
+    return base_url
+
+def _parse_date_range(value: str, source: dict, now: datetime) -> tuple[datetime, datetime | None]:
+    value = " ".join(value.replace("–", "-").replace("—", "-").split())
+    same_month = re.fullmatch(r"(\d{1,2})\s*-\s*(\d{1,2})\s+([A-Za-z]+)\s+(\d{4})(?:\s+(.+))?", value)
+    cross_month = re.fullmatch(r"(\d{1,2})\s+([A-Za-z]+)\s*-\s*(\d{1,2})\s+([A-Za-z]+)\s+(\d{4})(?:\s+(.+))?", value)
+    if same_month:
+        start_day, end_day, month, year, time_text = same_month.groups()
+        suffix = f" {time_text}" if time_text else ""
+        start = _parse_date(f"{start_day} {month} {year}{suffix}", source, now)
+        end = _parse_date(f"{end_day} {month} {year}{suffix}", source, now)
+        return start, end
+    if cross_month:
+        start_day, start_month, end_day, end_month, year, time_text = cross_month.groups()
+        suffix = f" {time_text}" if time_text else ""
+        start = _parse_date(f"{start_day} {start_month} {year}{suffix}", source, now)
+        end = _parse_date(f"{end_day} {end_month} {year}{suffix}", source, now)
+        return start, end
+    return _parse_date(value, source, now), None
+
+def _parse_date(value: str, source: dict, now: datetime) -> datetime:
+    value = " ".join(value.split())
+    tz = ZoneInfo(source.get("timezone") or "UTC")
+    fmt = source.get("date_format", "").strip()
+    if fmt:
+        candidates = [fmt]
+        # Event lists commonly omit a year. Resolve to the next occurrence.
+        for candidate in candidates:
+            parsed = datetime.strptime(value, candidate).replace(tzinfo=tz)
+            if "%Y" not in candidate and "%y" not in candidate:
+                parsed = parsed.replace(year=now.year)
+                if parsed < now - timedelta(days=2):
+                    parsed = parsed.replace(year=now.year + 1)
+            return parsed
+    if not re.search(r"\b\d{1,4}\b", value):
+        raise ValueError("Date text must contain a numeric day or year")
+    parsed = date_parser.parse(value, fuzzy=True, default=now.replace(tzinfo=None), tzinfos={"INFO": None, "IPAC": None, "IMAX": None})
+    if parsed.tzinfo is None:
+        parsed = parsed.replace(tzinfo=tz)
+    if not now.year - 1 <= parsed.year <= now.year + 10:
+        raise ValueError("Parsed year is outside the plausible event range")
+    return parsed.replace(microsecond=0)
+
+def _extract_json_ld(soup: BeautifulSoup, base_url: str, source: dict, now: datetime) -> list[ExtractedEvent]:
+    result = []
+    def walk(value):
+        if isinstance(value, list):
+            for item in value: yield from walk(item)
+        elif isinstance(value, dict):
+            if value.get("@type") == "Event" or "Event" in (value.get("@type") or []):
+                yield value
+            for key in ("@graph", "itemListElement"):
+                if key in value: yield from walk(value[key])
+    for script in soup.select('script[type="application/ld+json"]'):
+        try:
+            data = json.loads(script.string or script.get_text())
+        except (json.JSONDecodeError, TypeError):
+            continue
+        for event in walk(data):
+            location = event.get("location") or {}
+            address = location.get("address", "") if isinstance(location, dict) else ""
+            if isinstance(address, dict):
+                address = ", ".join(str(address.get(k, "")) for k in ("streetAddress", "addressLocality", "addressRegion", "postalCode") if address.get(k))
+            try:
+                start = _parse_date(str(event["startDate"]), source, now)
+                end = _parse_date(str(event["endDate"]), source, now) if event.get("endDate") else None
+            except (KeyError, ValueError, OverflowError):
+                continue
+            result.append(ExtractedEvent(title=str(event.get("name", "Untitled event")), starts_at=start, ends_at=end, description=str(event.get("description", "")), url=urljoin(base_url, str(event.get("url", base_url))), address=str(address)))
+    return result
+
+
+def _weekly_dates(day_text: str, now: datetime, weeks: int = 12) -> list[datetime]:
+    names = {name.lower(): i for i, name in enumerate(("Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"))}
+    match = next(((name, idx) for name, idx in names.items() if name in day_text.lower()), None)
+    if not match:
+        return []
+    delta = (match[1] - now.weekday()) % 7
+    first = (now + timedelta(days=delta)).replace(hour=0, minute=0, second=0, microsecond=0)
+    return [first + timedelta(weeks=i) for i in range(weeks)]
+
+def _selector_for(node) -> str:
+    if node.get("id"):
+        return f"#{node['id']}"
+    classes = [value for value in node.get("class", []) if not value.startswith(("elementor-element-", "post-"))]
+    return node.name + "".join(f".{value}" for value in classes[:2])
+
+def detect_rules(html: str, url: str, timezone: str = "Australia/Sydney") -> tuple[dict, list[ExtractedEvent], str]:
+    soup = BeautifulSoup(html, "html.parser")
+    now = datetime.now(ZoneInfo(timezone))
+    json_source = {"url": url, "timezone": timezone, "item_selector": "json-ld", "title_selector": "", "date_selector": "", "time_selector": "", "link_selector": "", "description_selector": "", "location_selector": "", "date_format": "", "recurring_weekly": 0}
+    json_events = extract_events(html, json_source)
+    if json_events:
+        return json_source, json_events, "Found schema.org JSON-LD Event data."
+
+    date_pattern = re.compile(r"(?:\b(?:mon|tue|wed|thu|fri|sat|sun)(?:day)?\b|\b\d{1,2}[\s/.-](?:\d{1,2}|jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)|\b(?:jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)[a-z]*\s+\d{1,2})", re.I)
+    candidates = []
+    container_selectors = ("article", "li", ".event", ".events", "[class*='event']", "[class*='Event']", ".event-list__item", ".item", "[class*='item']", ".wpupg-item", ".wpupg-item.wpupg-type-event", "[data-permalink]", ".wp-block-post", ".elementor-loop-item", ".tribe-events-calendar-list__event-row", ".em-event", ".mec-event-article")
+    for selector in container_selectors:
+        try: nodes = soup.select(selector)
+        except Exception: continue
+        if not 2 <= len(nodes) <= 300:
+            continue
+        nodes = [node for node in nodes if not any(marker in " ".join(str(value) for value in (node.get("class") or [])).casefold() for marker in SOCIAL_WIDGET_MARKERS)]
+        if len(nodes) < 2: continue
+        dated = [node for node in nodes if date_pattern.search(node.get_text(" ", strip=True))]
+        titled = [node for node in dated if node.find(["h1", "h2", "h3", "h4", "a"]) or node.select_one("[class*='title'], [itemprop='name']")]
+        score = len(titled) * 3 + len(dated) - max(0, len(nodes) - len(dated))
+        if len(titled) >= 2:
+            candidates.append((score, selector, titled))
+    if not candidates:
+        raise CrawlError("No repeated event-like HTML containers were detected. Try JSON-LD or configure CSS selectors manually.")
+    trials = []
+    for _, item_selector, nodes in sorted(candidates, reverse=True)[:8]:
+        title_selectors = []
+        date_selectors = []
+        link_selectors = []
+        for sample in nodes[:8]:
+            for node in sample.select("h1, h2, h3, h4, a, [class*='title'], [itemprop='name']"):
+                selector = _selector_for(node)
+                if selector not in title_selectors: title_selectors.append(selector)
+            for node in sample.find_all(["time", "span", "div", "li", "p", "ul"]):
+                if date_pattern.search(node.get_text(" ", strip=True)):
+                    selector = _selector_for(node)
+                    if selector not in date_selectors: date_selectors.append(selector)
+            for node in sample.find_all("a"):
+                selector = _selector_for(node)
+                if selector not in link_selectors: link_selectors.append(selector)
+            if any(sample.get(attribute) for attribute in ("data-permalink", "data-url", "data-href")) and ":scope" not in link_selectors:
+                link_selectors.insert(0, ":scope")
+        time_selectors = []
+        for sample in nodes[:8]:
+            for node in sample.find_all(["time", "span", "div", "p"]):
+                if re.search(r"\b\d{1,2}(?::\d{2})?\s*(?:am|pm)\b", node.get_text(" ", strip=True), re.IGNORECASE):
+                    selector = _selector_for(node)
+                    if selector not in time_selectors: time_selectors.append(selector)
+        for title_selector in title_selectors[:6]:
+            for date_selector in date_selectors[:10]:
+                for time_selector in [""] + time_selectors[:4]:
+                    for recurring in (0, 1):
+                        rules = {"url": url, "timezone": timezone, "item_selector": item_selector, "title_selector": title_selector, "date_selector": date_selector, "time_selector": time_selector, "link_selector": link_selectors[0] if link_selectors else "", "description_selector": "", "location_selector": "", "date_format": "", "recurring_weekly": recurring}
+                        events = extract_events(html, rules)
+                        unique_titles = len({event.title for event in events})
+                        explicit_dates = sum(1 for event in events if event.starts_at.second == 0 and event.starts_at.microsecond == 0 and now.year - 1 <= event.starts_at.year <= now.year + 10)
+                        if events and unique_titles >= 2 and explicit_dates == len(events):
+                            occurrence_penalty = 12 if recurring else 1
+                            trials.append((unique_titles * 10 + len(events) / occurrence_penalty, rules, events, len(nodes)))
+    if not trials:
+        raise CrawlError("Event-like containers were found, but no selector combination produced parseable events. Adjust selectors manually.")
+    _, rules, events, container_count = max(trials, key=lambda value: value[0])
+    winning_nodes = soup.select(rules["item_selector"])
+    sample = winning_nodes[0] if winning_nodes else None
+    if sample:
+        description_node = sample.find(["p", "summary"])
+        location_node = next((node for node in sample.find_all(["span", "div", "p"]) if re.search(r"\b(?:venue|location|address|theatre|hall|hotel|club)\b", node.get_text(" ", strip=True), re.I)), None)
+        rules["description_selector"] = _selector_for(description_node) if description_node else ""
+        rules["location_selector"] = _selector_for(location_node) if location_node else ""
+    return rules, events, f"Guessed CSS selectors from {container_count} repeated event-like containers."
+
+def extract_events(html: str, source: dict) -> list[ExtractedEvent]:
+    soup = BeautifulSoup(html, "html.parser")
+    tz = ZoneInfo(source.get("timezone") or "UTC")
+    now = datetime.now(tz)
+    item_selector = source.get("item_selector", "").strip()
+    if not item_selector or item_selector == "json-ld":
+        json_events = _extract_json_ld(soup, source["url"], source, now)
+        if json_events or item_selector == "json-ld":
+            return json_events
+    events = []
+    for item in soup.select(item_selector):
+        title = _text(item, source.get("title_selector", ""))
+        raw_date = _text(item, source.get("date_selector", ""))
+        raw_time = _text(item, source.get("time_selector", ""))
+        if not title or not raw_date:
+            continue
+        description = _text(item, source.get("description_selector", ""))
+        address = _text(item, source.get("location_selector", ""))
+        link = _link(item, source.get("link_selector", ""), source["url"])
+        if source.get("recurring_weekly"):
+            for start in _weekly_dates(raw_date, now):
+                events.append(ExtractedEvent(title=title, starts_at=start, description=description, url=link, address=address))
+            continue
+        try:
+            start, end = _parse_date_range(f"{raw_date} {raw_time}".strip(), source, now)
+        except (ValueError, OverflowError):
+            continue
+        events.append(ExtractedEvent(title=title, starts_at=start, ends_at=end, description=description, url=link, address=address))
+    return events
+
+def infer_page_fields(html: str, url: str) -> dict:
+    soup = BeautifulSoup(html, "html.parser")
+    title = ""
+    og_site = soup.select_one('meta[property="og:site_name"]')
+    if og_site:
+        content = og_site.get("content")
+        title = str(content).strip() if content else ""
+    if not title and soup.title:
+        title = re.split(r"[|–—-]", soup.title.get_text(" ", strip=True))[0].strip()
+    heading = soup.find("h1")
+    source_name = heading.get_text(" ", strip=True) if heading else title or urlparse(url).hostname or "Event source"
+    text = soup.get_text(" ", strip=True)
+    address_match = re.search(r"\b\d{1,5}[A-Za-z/-]*\s+[A-Za-z][A-Za-z .'-]+(?:Street|St|Road|Rd|Lane|Ln|Avenue|Ave|Drive|Dr|Way|Place|Pl),?\s+[A-Za-z ]+\s+(?:NSW|VIC|QLD|SA|WA|TAS|NT|ACT)\s+\d{4}\b", text, re.I)
+    address = address_match.group(0).strip() if address_match else ""
+    return {"name": source_name, "venue_name": title or source_name, "venue_address": address}
+
+def external_key(event: ExtractedEvent) -> str:
+    value = f"{event.title.casefold()}|{event.starts_at.isoformat()}|{event.address.casefold()}"
+    return hashlib.sha256(value.encode()).hexdigest()

app/db.py

@@ -0,0 +1,93 @@
+import os
+import sqlite3
+from contextlib import contextmanager
+from pathlib import Path
+
+DB_PATH = os.getenv("DATABASE_PATH", "data/what2do.db")
+
+SCHEMA = """
+PRAGMA journal_mode=WAL;
+PRAGMA foreign_keys=ON;
+CREATE TABLE IF NOT EXISTS sources (
+  id INTEGER PRIMARY KEY,
+  name TEXT NOT NULL,
+  url TEXT NOT NULL UNIQUE,
+  venue_name TEXT NOT NULL,
+  venue_address TEXT NOT NULL,
+  latitude REAL,
+  longitude REAL,
+  timezone TEXT NOT NULL DEFAULT 'Australia/Sydney',
+  strategy_json TEXT NOT NULL DEFAULT '{}',
+  item_selector TEXT NOT NULL,
+  title_selector TEXT NOT NULL,
+  date_selector TEXT NOT NULL,
+  time_selector TEXT NOT NULL DEFAULT '',
+  link_selector TEXT NOT NULL DEFAULT '',
+  description_selector TEXT NOT NULL DEFAULT '',
+  location_selector TEXT NOT NULL DEFAULT '',
+  date_format TEXT NOT NULL DEFAULT '',
+  recurring_weekly INTEGER NOT NULL DEFAULT 0,
+  enabled INTEGER NOT NULL DEFAULT 1,
+  crawl_interval_minutes INTEGER NOT NULL DEFAULT 360,
+  last_crawled_at TEXT,
+  last_status TEXT,
+  last_error TEXT,
+  created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
+);
+CREATE TABLE IF NOT EXISTS events (
+  id INTEGER PRIMARY KEY,
+  source_id INTEGER NOT NULL REFERENCES sources(id) ON DELETE CASCADE,
+  external_key TEXT NOT NULL,
+  title TEXT NOT NULL,
+  description TEXT NOT NULL DEFAULT '',
+  starts_at TEXT NOT NULL,
+  ends_at TEXT,
+  venue_name TEXT NOT NULL,
+  address TEXT NOT NULL,
+  latitude REAL,
+  longitude REAL,
+  event_url TEXT NOT NULL,
+  hidden INTEGER NOT NULL DEFAULT 0,
+  last_seen_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
+  created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
+  UNIQUE(source_id, external_key)
+);
+CREATE INDEX IF NOT EXISTS idx_events_starts_at ON events(starts_at);
+CREATE INDEX IF NOT EXISTS idx_events_source ON events(source_id);
+CREATE TABLE IF NOT EXISTS geocode_cache (
+  query TEXT PRIMARY KEY,
+  latitude REAL,
+  longitude REAL,
+  display_name TEXT,
+  updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
+);
+"""
+
+@contextmanager
+def connect():
+    path = Path(DB_PATH)
+    path.parent.mkdir(parents=True, exist_ok=True)
+    db = sqlite3.connect(path)
+    db.row_factory = sqlite3.Row
+    db.execute("PRAGMA foreign_keys=ON")
+    try:
+        yield db
+        db.commit()
+    finally:
+        db.close()
+
+def init_db():
+    with connect() as db:
+        db.executescript(SCHEMA)
+        event_columns = {row["name"] for row in db.execute("PRAGMA table_info(events)")}
+        if "hidden" not in event_columns:
+            db.execute("ALTER TABLE events ADD COLUMN hidden INTEGER NOT NULL DEFAULT 0")
+        source_columns = {row["name"] for row in db.execute("PRAGMA table_info(sources)")}
+        if "strategy_json" not in source_columns:
+            db.execute("ALTER TABLE sources ADD COLUMN strategy_json TEXT NOT NULL DEFAULT '{}'")
+        db.execute("""UPDATE sources SET strategy_json=json_object(
+          'mode','html','request_url',url,'method','GET','response_type','html',
+          'item_selector',item_selector,'title_selector',title_selector,'date_selector',date_selector,
+          'time_selector',time_selector,'link_selector',link_selector,'description_selector',description_selector,
+          'location_selector',location_selector,'date_format',date_format,'recurring_weekly',recurring_weekly
+        ) WHERE strategy_json='{}' OR strategy_json=''""")

app/main.py

@@ -0,0 +1,569 @@
+import asyncio
+import json
+import math
+import os
+import secrets
+import threading
+from calendar import Calendar
+from contextlib import asynccontextmanager
+from datetime import datetime, timedelta, timezone
+from urllib.parse import urljoin
+
+import httpx
+from fastapi import FastAPI, HTTPException, Query, Request
+from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
+from fastapi.staticfiles import StaticFiles
+from fastapi.templating import Jinja2Templates
+from bs4 import BeautifulSoup
+
+from .crawler import CrawlError, _http_client, detect_rules, external_key, fetch_html, infer_page_fields
+from .db import connect, init_db
+from .strategies import execute_strategy, parse_strategy
+
+TEMPLATES = Jinja2Templates(directory="app/templates")
+NOMINATIM_USER_AGENT = os.getenv("NOMINATIM_USER_AGENT", "what2do-local/0.1")
+SCHEDULER_INTERVAL = int(os.getenv("CRAWL_INTERVAL_MINUTES", "60"))
+PICKER_SNAPSHOTS = {}
+PICKER_LOCK = threading.Lock()
+PICKER_TTL = timedelta(minutes=15)
+EVENT_TAGS = {
+    "music": ("music", "band", "dj", "album", "tour", "concert", "punk", "emo", "rock", "singer"),
+    "art": ("art", "gallery", "exhibition", "paint", "creative", "workshop"),
+    "comedy": ("comedy", "comedian", "stand-up", "standup"),
+    "trivia": ("trivia", "quiz"),
+    "film": ("film", "movie", "cinema", "screening"),
+    "food": ("food", "dinner", "lunch", "brunch", "schnitty", "steak", "surf & turf"),
+    "sports": ("sport", "football", "rugby", "darts", "poker", "race"),
+    "community": ("community", "market", "fundraiser", "family", "festival"),
+    "nightlife": ("party", "club", "night", "bingo", "jackpot"),
+}
+
+SEEDS = [
+    {
+        "name": "Dicey Riley's weekly specials", "url": "https://www.diceyrileyshotel.com.au/pub-specials/",
+        "venue_name": "Dicey Riley's Hotel", "venue_address": "329-333 Crown St, Wollongong NSW 2500, Australia",
+        "latitude": -34.42538, "longitude": 150.89375,
+        "item_selector": "article.weekly_specials", "title_selector": "h2.elementor-heading-title",
+        "date_selector": ".elementor-post-info__item--type-custom", "time_selector": "", "link_selector": "a",
+        "date_format": "", "recurring_weekly": 1,
+    },
+    {
+        "name": "La La La's events", "url": "https://www.lalalas.com.au/all-events/",
+        "venue_name": "La La La's", "venue_address": "3 Globe Lane, Wollongong NSW 2500, Australia",
+        "latitude": -34.42569, "longitude": 150.89816,
+        "item_selector": ".slider_container > .items", "title_selector": ".feed_titles h2",
+        "date_selector": ".feed_dates ul", "time_selector": "", "link_selector": "a.slider_button",
+        "date_format": "%d %B %H:%M", "recurring_weekly": 0,
+    },
+]
+
+def source_values(form) -> dict:
+    raw_strategy = form.get("strategy_json", "{}").strip() or "{}"
+    try: strategy = json.loads(raw_strategy)
+    except json.JSONDecodeError as exc: raise ValueError(f"Strategy JSON is invalid: {exc.msg} at line {exc.lineno}") from exc
+    if strategy.get("mode", "html") == "html":
+        strategy.update({"mode": "html", "request_url": form.get("url", "").strip(), "method": strategy.get("method", "GET"), "response_type": "html", "item_selector": form.get("item_selector", "").strip(), "title_selector": form.get("title_selector", "").strip(), "date_selector": form.get("date_selector", "").strip(), "time_selector": form.get("time_selector", "").strip(), "link_selector": form.get("link_selector", "").strip(), "description_selector": form.get("description_selector", "").strip(), "location_selector": form.get("location_selector", "").strip(), "date_format": form.get("date_format", "").strip(), "recurring_weekly": 1 if form.get("recurring_weekly") else 0})
+    return {
+        "name": form.get("name", "").strip(), "url": form.get("url", "").strip(),
+        "venue_name": form.get("venue_name", "").strip(), "venue_address": form.get("venue_address", "").strip(),
+        "latitude": float(form["latitude"]) if form.get("latitude") else None,
+        "longitude": float(form["longitude"]) if form.get("longitude") else None,
+        "timezone": form.get("timezone", "Australia/Sydney").strip(), "item_selector": form.get("item_selector", "").strip(),
+        "title_selector": form.get("title_selector", "").strip(), "date_selector": form.get("date_selector", "").strip(),
+        "time_selector": form.get("time_selector", "").strip(), "link_selector": form.get("link_selector", "").strip(),
+        "description_selector": form.get("description_selector", "").strip(), "location_selector": form.get("location_selector", "").strip(),
+        "date_format": form.get("date_format", "").strip(), "recurring_weekly": 1 if form.get("recurring_weekly") else 0,
+        "strategy_json": json.dumps(strategy),
+        "enabled": 1 if form.get("enabled") else 0, "crawl_interval_minutes": int(form.get("crawl_interval_minutes") or 360),
+    }
+
+def seed_sources():
+    with connect() as db:
+        for source in SEEDS:
+            source_strategy = {"mode": "html", "request_url": source["url"], "method": "GET", "response_type": "html", "item_selector": source["item_selector"], "title_selector": source["title_selector"], "date_selector": source["date_selector"], "time_selector": source.get("time_selector", ""), "link_selector": source.get("link_selector", ""), "link_attribute": "href", "description_selector": "", "location_selector": "", "date_format": source.get("date_format", ""), "recurring_weekly": source.get("recurring_weekly", 0)}
+            fields = {**source, "strategy_json": json.dumps(source_strategy), "timezone": "Australia/Sydney", "time_selector": source.get("time_selector", ""), "link_selector": source.get("link_selector", ""), "description_selector": "", "location_selector": "", "enabled": 1, "crawl_interval_minutes": 360}
+            columns = ",".join(fields)
+            db.execute(f"INSERT OR IGNORE INTO sources ({columns}) VALUES ({','.join('?' for _ in fields)})", tuple(fields.values()))
+            db.execute("UPDATE sources SET latitude=COALESCE(latitude,?),longitude=COALESCE(longitude,?) WHERE url=?", (source.get("latitude"), source.get("longitude"), source["url"]))
+            db.execute("UPDATE events SET latitude=COALESCE(latitude,?),longitude=COALESCE(longitude,?) WHERE source_id=(SELECT id FROM sources WHERE url=?)", (source.get("latitude"), source.get("longitude"), source["url"]))
+
+def geocode(query: str):
+    if not query:
+        return None
+    try:
+        latitude, longitude = (float(value.strip()) for value in query.split(",", 1))
+        if -90 <= latitude <= 90 and -180 <= longitude <= 180:
+            return latitude, longitude, query
+    except (ValueError, TypeError):
+        pass
+    cache_key = query.casefold()
+    with connect() as db:
+        cached = db.execute("SELECT latitude, longitude, display_name FROM geocode_cache WHERE query=?", (cache_key,)).fetchone()
+        if cached:
+            return tuple(cached)
+    parts = [part.strip() for part in query.split(",") if part.strip()]
+    candidates = [query]
+    if len(parts) > 1:
+        candidates.append(", ".join(parts[1:]))
+    result = []
+    try:
+        for candidate in candidates:
+            response = httpx.get("https://nominatim.openstreetmap.org/search", params={"q": candidate, "format": "jsonv2", "limit": 1, "countrycodes": "au"}, headers={"User-Agent": NOMINATIM_USER_AGENT}, timeout=15)
+            response.raise_for_status()
+            result = response.json()
+            if result:
+                break
+    except (httpx.HTTPError, ValueError):
+        return None
+    if not result:
+        return None
+    value = (float(result[0]["lat"]), float(result[0]["lon"]), result[0].get("display_name", query))
+    with connect() as db:
+        db.execute("INSERT OR REPLACE INTO geocode_cache(query,latitude,longitude,display_name) VALUES(?,?,?,?)", (cache_key, *value))
+    return value
+
+def crawl_source(source_id: int):
+    with connect() as db:
+        row = db.execute("SELECT * FROM sources WHERE id=?", (source_id,)).fetchone()
+    if not row:
+        raise CrawlError("Source not found.")
+    source = dict(row)
+    try:
+        with _http_client(timeout=45) as client:
+            events = execute_strategy(source, client)
+        coords = (source["latitude"], source["longitude"])
+        if None in coords:
+            point = geocode(source["venue_address"])
+            coords = point[:2] if point else (None, None)
+        seen = []
+        with connect() as db:
+            db.execute("UPDATE sources SET latitude=?,longitude=? WHERE id=?", (*coords, source_id))
+            for event in events:
+                key = external_key(event)
+                seen.append(key)
+                address = event.address or source["venue_address"]
+                db.execute("""INSERT INTO events(source_id,external_key,title,description,starts_at,ends_at,venue_name,address,latitude,longitude,event_url)
+                  VALUES(?,?,?,?,?,?,?,?,?,?,?) ON CONFLICT(source_id,external_key) DO UPDATE SET title=excluded.title,description=excluded.description,
+                  starts_at=excluded.starts_at,ends_at=excluded.ends_at,venue_name=excluded.venue_name,address=excluded.address,latitude=excluded.latitude,longitude=excluded.longitude,event_url=excluded.event_url,last_seen_at=CURRENT_TIMESTAMP""",
+                  (source_id, key, event.title, event.description, event.starts_at.astimezone(timezone.utc).isoformat(), event.ends_at.astimezone(timezone.utc).isoformat() if event.ends_at else None, source["venue_name"], address, *coords, event.url))
+            if seen:
+                placeholders = ",".join("?" for _ in seen)
+                db.execute(f"DELETE FROM events WHERE source_id=? AND external_key NOT IN ({placeholders})", (source_id, *seen))
+            db.execute("UPDATE sources SET last_crawled_at=CURRENT_TIMESTAMP,last_status=?,last_error=NULL WHERE id=?", (f"Found {len(events)} events", source_id))
+        return len(events)
+    except Exception as exc:
+        with connect() as db:
+            db.execute("UPDATE sources SET last_crawled_at=CURRENT_TIMESTAMP,last_status='Failed',last_error=? WHERE id=?", (str(exc)[:1000], source_id))
+        raise
+
+async def scheduler():
+    while True:
+        await asyncio.sleep(max(1, SCHEDULER_INTERVAL) * 60)
+        with connect() as db:
+            rows = db.execute("SELECT id,last_crawled_at,crawl_interval_minutes FROM sources WHERE enabled=1").fetchall()
+        now = datetime.now(timezone.utc)
+        for row in rows:
+            last = datetime.fromisoformat(row["last_crawled_at"] + "+00:00") if row["last_crawled_at"] else None
+            if not last or now - last >= timedelta(minutes=row["crawl_interval_minutes"]):
+                try: await asyncio.to_thread(crawl_source, row["id"])
+                except Exception: pass
+
+@asynccontextmanager
+async def lifespan(app):
+    init_db(); seed_sources()
+    task = asyncio.create_task(scheduler())
+    yield
+    task.cancel()
+
+app = FastAPI(title="What2Do", lifespan=lifespan)
+app.mount("/static", StaticFiles(directory="app/static"), name="static")
+
+def classify_event(title: str, description: str) -> list[str]:
+    text = f"{title} {description}".casefold()
+    tags = [tag for tag, keywords in EVENT_TAGS.items() if any(keyword in text for keyword in keywords)]
+    return tags or ["other"]
+
+def distance_km(lat1, lon1, lat2, lon2):
+    p1, p2 = math.radians(lat1), math.radians(lat2)
+    dp, dl = math.radians(lat2-lat1), math.radians(lon2-lon1)
+    a = math.sin(dp/2)**2 + math.cos(p1)*math.cos(p2)*math.sin(dl/2)**2
+    return 6371 * 2 * math.atan2(math.sqrt(a), math.sqrt(1-a))
+
+@app.get("/", response_class=HTMLResponse)
+def index(request: Request, location: str = "", latitude: float | None = None, longitude: float | None = None, start: str = "", end: str = "", radius: float = 25, tag: list[str] = Query(default=[]), term: str = ""):
+    point = (latitude, longitude, location) if latitude is not None and longitude is not None else (geocode(location) if location else None)
+    local_now = datetime.now().astimezone()
+    default_start = local_now.replace(day=1, hour=0, minute=0, second=0, microsecond=0)
+    if default_start.month == 12:
+        default_end = default_start.replace(year=default_start.year + 1, month=1)
+    else:
+        default_end = default_start.replace(month=default_start.month + 1)
+    if not start:
+        start = default_start.strftime("%Y-%m-%dT%H:%M")
+    if not end:
+        end = default_end.strftime("%Y-%m-%dT%H:%M")
+    try: start_at = datetime.fromisoformat(start).astimezone(timezone.utc)
+    except ValueError:
+        start_at = default_start.astimezone(timezone.utc)
+        start = default_start.strftime("%Y-%m-%dT%H:%M")
+    try: end_at = datetime.fromisoformat(end).astimezone(timezone.utc)
+    except ValueError:
+        end_at = default_end.astimezone(timezone.utc)
+        end = default_end.strftime("%Y-%m-%dT%H:%M")
+    with connect() as db:
+        rows = [dict(r) for r in db.execute("SELECT events.*,sources.name source_name FROM events JOIN sources ON sources.id=events.source_id WHERE events.hidden=0 AND starts_at < ? AND COALESCE(ends_at,starts_at) >= ? ORDER BY starts_at", (end_at.isoformat(), start_at.isoformat())).fetchall()]
+    for event in rows:
+        event["distance"] = distance_km(point[0], point[1], event["latitude"], event["longitude"]) if point and event["latitude"] is not None else None
+        local_start = datetime.fromisoformat(event["starts_at"]).astimezone()
+        event["display_time"] = local_start.strftime("%a %d %b %Y, %H:%M")
+        if event["ends_at"]:
+            local_end = datetime.fromisoformat(event["ends_at"]).astimezone()
+            event["display_time"] += f" – {local_end.strftime('%a %d %b %Y, %H:%M')}"
+        event["day_key"] = local_start.strftime("%Y-%m-%d")
+        event["end_day_key"] = datetime.fromisoformat(event["ends_at"]).astimezone().strftime("%Y-%m-%d") if event["ends_at"] else event["day_key"]
+        event["tags"] = classify_event(event["title"], event["description"])
+    if point: rows = [event for event in rows if event["distance"] is not None and event["distance"] <= radius]
+    selected_tags = [value for value in tag if value in EVENT_TAGS or value == "other"]
+    if selected_tags: rows = [event for event in rows if set(selected_tags).intersection(event["tags"])]
+    if term:
+        needle = term.casefold()
+        rows = [event for event in rows if needle in f"{event['title']} {event['description']} {event['venue_name']} {event['address']}".casefold()]
+    calendar_start = start_at.astimezone()
+    month_days = list(Calendar(firstweekday=0).itermonthdates(calendar_start.year, calendar_start.month))
+    day_counts = {}
+    for event in rows:
+        covered_day = datetime.fromisoformat(event["day_key"]).date()
+        final_day = datetime.fromisoformat(event["end_day_key"]).date()
+        while covered_day <= final_day:
+            key = covered_day.strftime("%Y-%m-%d")
+            day_counts[key] = day_counts.get(key, 0) + 1
+            covered_day += timedelta(days=1)
+    calendar_days = [{"date": day, "key": day.strftime("%Y-%m-%d"), "count": day_counts.get(day.strftime("%Y-%m-%d"), 0), "in_month": day.month == calendar_start.month} for day in month_days]
+    map_events = [{"id": event["id"], "title": event["title"], "venue": event["venue_name"], "time": event["display_time"], "latitude": event["latitude"], "longitude": event["longitude"]} for event in rows if event["latitude"] is not None]
+    context = {"events": rows, "map_events": map_events, "calendar_days": calendar_days, "calendar_month": calendar_start.strftime("%B %Y"), "event_tags": EVENT_TAGS.keys(), "location": location, "latitude": latitude, "longitude": longitude, "start": start, "end": end, "radius": radius, "selected_tags": selected_tags, "term": term, "geocoded": point}
+    return TEMPLATES.TemplateResponse(request, "index.html", context)
+
+@app.get("/api/locations")
+def location_suggestions(q: str = ""):
+    query = q.strip()
+    if len(query) < 3:
+        return []
+    try:
+        response = httpx.get("https://photon.komoot.io/api/", params={"q": query, "limit": 10, "lang": "en"}, headers={"User-Agent": "what2do-local/0.1"}, timeout=10)
+        response.raise_for_status()
+        suggestions = []
+        for feature in response.json().get("features", []):
+            properties = feature.get("properties", {})
+            if properties.get("countrycode", "").upper() != "AU":
+                continue
+            coordinates = feature.get("geometry", {}).get("coordinates", [])
+            if len(coordinates) < 2:
+                continue
+            place = properties.get("name") or properties.get("city") or properties.get("district") or properties.get("county")
+            label = ", ".join(dict.fromkeys(value for value in (place, properties.get("city"), properties.get("state"), properties.get("postcode"), properties.get("country")) if value))
+            suggestion = {"label": label, "latitude": float(coordinates[1]), "longitude": float(coordinates[0])}
+            if label and suggestion not in suggestions:
+                suggestions.append(suggestion)
+        return suggestions[:5]
+    except (httpx.HTTPError, ValueError, KeyError, TypeError):
+        return []
+
+@app.get("/api/addresses")
+def address_suggestions(q: str = ""):
+    query = q.strip()
+    if len(query) < 3:
+        return []
+    try:
+        response = httpx.get("https://photon.komoot.io/api/", params={"q": query, "limit": 8, "lang": "en"}, headers={"User-Agent": "what2do-local/0.1"}, timeout=10)
+        response.raise_for_status()
+        results = []
+        for feature in response.json().get("features", []):
+            properties = feature.get("properties", {})
+            if properties.get("countrycode", "").upper() != "AU": continue
+            coordinates = feature.get("geometry", {}).get("coordinates", [])
+            if len(coordinates) < 2: continue
+            street = " ".join(value for value in (properties.get("housenumber"), properties.get("street")) if value)
+            label = ", ".join(dict.fromkeys(value for value in (properties.get("name"), street, properties.get("city"), properties.get("state"), properties.get("postcode"), properties.get("country")) if value))
+            item = {"label": label, "latitude": float(coordinates[1]), "longitude": float(coordinates[0])}
+            if label and item not in results: results.append(item)
+        return results[:5]
+    except (httpx.HTTPError, ValueError, KeyError, TypeError):
+        return []
+
+@app.get("/events/{event_id}", response_class=HTMLResponse)
+def event_detail(request: Request, event_id: int):
+    with connect() as db:
+        row = db.execute("SELECT events.*,sources.name source_name,sources.url source_url FROM events JOIN sources ON sources.id=events.source_id WHERE events.id=? AND events.hidden=0", (event_id,)).fetchone()
+    if not row:
+        raise HTTPException(404)
+    event = dict(row)
+    local_start = datetime.fromisoformat(event["starts_at"]).astimezone()
+    event["display_date"] = local_start.strftime("%A, %d %B %Y")
+    event["display_time"] = local_start.strftime("%H:%M")
+    event["display_end"] = None
+    if event["ends_at"]:
+        local_end = datetime.fromisoformat(event["ends_at"]).astimezone()
+        event["display_end"] = local_end.strftime("%A, %d %B %Y at %H:%M")
+    event["canonical_url"] = str(request.url)
+    return TEMPLATES.TemplateResponse(request, "event.html", {"event": event})
+
+@app.get("/sources", response_class=HTMLResponse)
+def sources(request: Request):
+    with connect() as db: rows = db.execute("SELECT *, (SELECT count(*) FROM events WHERE source_id=sources.id) event_count FROM sources ORDER BY name").fetchall()
+    return TEMPLATES.TemplateResponse(request, "sources.html", {"sources": rows})
+
+@app.get("/sources/new", response_class=HTMLResponse)
+def new_source(request: Request):
+    return TEMPLATES.TemplateResponse(request, "source_form.html", {"source": {"enabled": 1, "timezone": "Australia/Sydney", "crawl_interval_minutes": 360, "strategy_json": "{}"}, "action": "/sources", "preview": None})
+
+@app.get("/sources/{source_id}", response_class=HTMLResponse)
+def edit_source(request: Request, source_id: int):
+    with connect() as db: source = db.execute("SELECT * FROM sources WHERE id=?", (source_id,)).fetchone()
+    if not source: raise HTTPException(404)
+    return TEMPLATES.TemplateResponse(request, "source_form.html", {"source": {**dict(source), "strategy_json": json.dumps(parse_strategy(source["strategy_json"]), indent=2)}, "action": f"/sources/{source_id}", "preview": None})
+
+@app.post("/sources")
+async def create_source(request: Request):
+    form = await request.form(); values = source_values(form)
+    with connect() as db:
+        columns = ",".join(values); cursor = db.execute(f"INSERT INTO sources ({columns}) VALUES ({','.join('?' for _ in values)})", tuple(values.values()))
+    return RedirectResponse(f"/sources/{cursor.lastrowid}", 303)
+
+@app.post("/sources/{source_id}")
+async def update_source(request: Request, source_id: int):
+    form = await request.form(); values = source_values(form)
+    with connect() as db: db.execute(f"UPDATE sources SET {','.join(f'{key}=?' for key in values)} WHERE id=?", (*values.values(), source_id))
+    return RedirectResponse(f"/sources/{source_id}", 303)
+
+@app.get("/sources/{source_id}/events", response_class=HTMLResponse)
+def source_events(request: Request, source_id: int):
+    with connect() as db:
+        source = db.execute("SELECT * FROM sources WHERE id=?", (source_id,)).fetchone()
+        events = db.execute("SELECT * FROM events WHERE source_id=? ORDER BY starts_at", (source_id,)).fetchall()
+    if not source: raise HTTPException(404)
+    return TEMPLATES.TemplateResponse(request, "source_events.html", {"source": source, "events": events})
+
+@app.get("/sources/{source_id}/events/{event_id}", response_class=HTMLResponse)
+def edit_source_event(request: Request, source_id: int, event_id: int):
+    with connect() as db:
+        source = db.execute("SELECT * FROM sources WHERE id=?", (source_id,)).fetchone()
+        event = db.execute("SELECT * FROM events WHERE id=? AND source_id=?", (event_id, source_id)).fetchone()
+    if not source or not event: raise HTTPException(404)
+    return TEMPLATES.TemplateResponse(request, "event_form.html", {"source": source, "event": event})
+
+@app.post("/sources/{source_id}/events/{event_id}")
+async def update_source_event(request: Request, source_id: int, event_id: int):
+    form = await request.form()
+    values = (form.get("title", "").strip(), form.get("description", "").strip(), form.get("starts_at", "").strip(), form.get("ends_at", "").strip() or None, form.get("venue_name", "").strip(), form.get("address", "").strip(), float(form["latitude"]) if form.get("latitude") else None, float(form["longitude"]) if form.get("longitude") else None, form.get("event_url", "").strip(), event_id, source_id)
+    with connect() as db:
+        db.execute("UPDATE events SET title=?,description=?,starts_at=?,ends_at=?,venue_name=?,address=?,latitude=?,longitude=?,event_url=? WHERE id=? AND source_id=?", values)
+    return RedirectResponse(f"/sources/{source_id}/events", 303)
+
+@app.post("/sources/{source_id}/events/{event_id}/visibility")
+def toggle_event_visibility(source_id: int, event_id: int):
+    with connect() as db:
+        db.execute("UPDATE events SET hidden=CASE hidden WHEN 1 THEN 0 ELSE 1 END WHERE id=? AND source_id=?", (event_id, source_id))
+    return RedirectResponse(f"/sources/{source_id}/events", 303)
+
+@app.post("/sources/{source_id}/crawl")
+def run_crawl(source_id: int):
+    try: crawl_source(source_id)
+    except Exception: pass
+    return RedirectResponse("/sources", 303)
+
+@app.post("/sources/{source_id}/delete")
+def delete_source(source_id: int):
+    with connect() as db: db.execute("DELETE FROM sources WHERE id=?", (source_id,))
+    return RedirectResponse("/sources", 303)
+
+def sanitize_picker_html(html: str, source_url: str) -> str:
+    soup = BeautifulSoup(html, "html.parser")
+    for tag in soup.select("script, iframe, frame, frameset, object, embed, form, input, button, textarea, select, base, meta[http-equiv]"):
+        tag.decompose()
+    for node in soup.find_all(True):
+        for attribute in list(node.attrs):
+            lowered = attribute.lower()
+            if lowered.startswith("on") or lowered in {"srcdoc", "integrity", "nonce", "formaction"}:
+                del node.attrs[attribute]
+        for attribute in ("src", "href", "poster", "action"):
+            value = node.get(attribute)
+            if not value: continue
+            value = str(value).strip()
+            if value.lower().startswith(("javascript:", "data:text/html", "vbscript:")):
+                del node.attrs[attribute]
+            elif attribute in {"src", "poster"}:
+                node.attrs[attribute] = urljoin(source_url, value)
+            elif attribute == "href" and node.name == "link" and "stylesheet" in (node.get("rel") or []):
+                node.attrs[attribute] = urljoin(source_url, value)
+            elif attribute in {"href", "action"}:
+                node.attrs[attribute] = "#"
+        node.attrs.pop("target", None)
+    head = soup.head
+    if not head:
+        head = soup.new_tag("head")
+        if soup.html: soup.html.insert(0, head)
+        else: soup.insert(0, head)
+    style = soup.new_tag("style")
+    style.string = "html,body{min-height:100%;background:white;color:#17211b} body{margin:12px} *{cursor:crosshair!important} img{max-width:100%;height:auto} .w2d-hover{outline:3px solid #dd5b32!important;outline-offset:2px!important}.w2d-picked{outline:4px solid #234c3c!important;outline-offset:2px!important}"
+    head.append(style)
+    script = soup.new_tag("script")
+    script.string = PICKER_SCRIPT
+    soup.body.append(script) if soup.body else soup.append(script)
+    return str(soup)
+
+PICKER_SCRIPT = r'''(() => {
+  let hovered = null, picked = null;
+  const esc = value => CSS.escape(value);
+  function stableClasses(element) {
+    return [...element.classList].filter(name => !name.startsWith('w2d-') && !/^elementor-element-[a-f0-9]+$/.test(name) && !/^post-\d+$/.test(name)).slice(0, 3);
+  }
+  function idPattern(id) {
+    const match = id.match(/^(.*?[-_])\d{3,}$/);
+    return match && match[1].length >= 3 ? `[id^="${CSS.escape(match[1])}"]` : `#${esc(id)}`;
+  }
+  function simpleSelector(element) {
+    if (element.id) return idPattern(element.id);
+    const classes = stableClasses(element);
+    return element.tagName.toLowerCase() + classes.map(name => `.${esc(name)}`).join('');
+  }
+  function selectorFor(element) {
+    if (element.id) return idPattern(element.id);
+    const parts = [];
+    let node = element;
+    while (node && node.nodeType === 1 && node.tagName.toLowerCase() !== 'html') {
+      let part = node.tagName.toLowerCase();
+      const classes = stableClasses(node).slice(0, 2);
+      if (classes.length) part += classes.map(name => `.${esc(name)}`).join('');
+      const parent = node.parentElement;
+      if (parent && parent.querySelectorAll(`:scope > ${part}`).length > 1) {
+        const siblings = [...parent.children].filter(child => child.matches(part));
+        part += `:nth-of-type(${[...parent.children].indexOf(node) + 1})`;
+      }
+      parts.unshift(part);
+      const candidate = parts.join(' > ');
+      try { if (document.querySelectorAll(candidate).length === 1 || parts.length >= 4) return candidate; } catch (_) {}
+      node = parent;
+    }
+    return parts.join(' > ');
+  }
+  function relativeSelector(element, container) {
+    if (element === container) return ':scope';
+    const parts = [];
+    let node = element;
+    while (node && node !== container) {
+      let part = simpleSelector(node);
+      if (!node.id) {
+        const parent = node.parentElement;
+        if (parent) {
+          const matches = [...parent.children].filter(child => child.matches(part));
+          if (matches.length > 1) part += `:nth-of-type(${[...parent.children].indexOf(node) + 1})`;
+        }
+      }
+      parts.unshift(part); node = node.parentElement;
+    }
+    return parts.join(' > ');
+  }
+  function repeatedAncestor(element) {
+    let node = element;
+    let best = null;
+    let depth = 0;
+    while (node && node.tagName && node.tagName.toLowerCase() !== 'body') {
+      const selector = simpleSelector(node);
+      let count = 0;
+      try { count = document.querySelectorAll(selector).length; } catch (_) {}
+      if (count >= 2 && count <= 300) {
+        const text = node.textContent.trim();
+        const hasUsefulText = text.length >= 10;
+        const score = (hasUsefulText ? 30 : 0) + Math.min(count, 20) - depth * 4 - (node.querySelectorAll('*').length > 300 ? 20 : 0);
+        if (!best || score > best.score) best = { node, selector, count, score };
+      }
+      node = node.parentElement; depth += 1;
+    }
+    return best;
+  }
+  document.addEventListener('mouseover', event => {
+    event.preventDefault(); event.stopPropagation();
+    if (hovered) hovered.classList.remove('w2d-hover');
+    hovered = event.target; hovered.classList.add('w2d-hover');
+  }, true);
+  document.addEventListener('mouseout', event => { if (hovered) hovered.classList.remove('w2d-hover'); }, true);
+  document.addEventListener('click', event => {
+    event.preventDefault(); event.stopPropagation();
+    if (picked) picked.classList.remove('w2d-picked');
+    picked = event.target; picked.classList.add('w2d-picked');
+    const selector = selectorFor(picked);
+    const repeated = repeatedAncestor(picked);
+    parent.postMessage({ type: 'what2do-selector', selector, selectorCount: document.querySelectorAll(selector).length, generalizedSelector: simpleSelector(picked), generalizedCount: document.querySelectorAll(simpleSelector(picked)).length, containerSelector: repeated?.selector || '', containerCount: repeated?.count || 0, relativeSelector: repeated ? relativeSelector(picked, repeated.node) : selector, tag: picked.tagName.toLowerCase(), text: picked.textContent.trim().slice(0, 160), href: picked.getAttribute('href') }, location.origin);
+  }, true);
+})();'''
+
+@app.post("/picker/snapshots")
+async def create_picker_snapshot(request: Request):
+    form = await request.form()
+    url = form.get("url", "").strip()
+    try:
+        sanitized = sanitize_picker_html(fetch_html(url), url)
+    except Exception as exc:
+        return JSONResponse({"error": str(exc)}, status_code=400)
+    token = secrets.token_urlsafe(24)
+    now = datetime.now(timezone.utc)
+    with PICKER_LOCK:
+        expired = [key for key, value in PICKER_SNAPSHOTS.items() if now - value["created_at"] > PICKER_TTL]
+        for key in expired: PICKER_SNAPSHOTS.pop(key, None)
+        PICKER_SNAPSHOTS[token] = {"html": sanitized, "created_at": now}
+    return {"preview_url": f"/picker/snapshots/{token}"}
+
+@app.get("/picker/snapshots/{token}", response_class=HTMLResponse)
+def picker_snapshot(token: str):
+    with PICKER_LOCK:
+        snapshot = PICKER_SNAPSHOTS.get(token)
+    if not snapshot or datetime.now(timezone.utc) - snapshot["created_at"] > PICKER_TTL:
+        raise HTTPException(404, "Picker snapshot expired")
+    return HTMLResponse(snapshot["html"], headers={"Content-Security-Policy": "default-src 'none'; img-src https: http: data:; style-src 'unsafe-inline' https:; font-src https: data:; script-src 'unsafe-inline'; connect-src 'none'; frame-src 'none'; form-action 'none'; base-uri 'none'", "Cache-Control": "no-store", "X-Content-Type-Options": "nosniff"})
+
+@app.post("/auto-detect", response_class=HTMLResponse)
+async def auto_detect(request: Request):
+    form = await request.form()
+    url = form.get("url", "").strip()
+    source = {
+        "id": form.get("source_id"), "name": form.get("name", ""), "url": url,
+        "venue_name": form.get("venue_name", ""), "venue_address": form.get("venue_address", ""),
+        "latitude": form.get("latitude") or None, "longitude": form.get("longitude") or None,
+        "timezone": form.get("timezone", "Australia/Sydney"), "enabled": 1,
+        "crawl_interval_minutes": form.get("crawl_interval_minutes", 360),
+    }
+    try:
+        html = fetch_html(url)
+        rules, items, message = detect_rules(html, url, source["timezone"])
+        strategy = {"mode": "html", "request_url": url, "method": "GET", "response_type": "html", **rules, "link_attribute": "href"}
+        source["strategy_json"] = json.dumps(strategy, indent=2)
+        with _http_client(timeout=45) as client:
+            strategy_items = execute_strategy({**source, "strategy_json": source["strategy_json"]}, client)
+        if strategy_items:
+            items = strategy_items
+        inferred = infer_page_fields(html, url)
+        for key, value in inferred.items():
+            if value and not source.get(key): source[key] = value
+        source.update(rules)
+        if source.get("venue_address") and not source.get("latitude"):
+            point = geocode(source["venue_address"])
+            if point: source["latitude"], source["longitude"] = point[:2]
+        preview_data = [{"title": item.title, "starts_at": item.starts_at.isoformat(), "ends_at": item.ends_at.isoformat() if item.ends_at else None, "url": item.url} for item in items[:20]]
+        error = None
+    except Exception as exc:
+        source.update({"item_selector": "", "title_selector": "", "date_selector": "", "time_selector": "", "link_selector": "", "description_selector": "", "location_selector": "", "date_format": "", "recurring_weekly": 0, "strategy_json": "{}"})
+        preview_data, message, error = [], None, str(exc)
+    action = f"/sources/{source['id']}" if source.get("id") else "/sources"
+    return TEMPLATES.TemplateResponse(request, "source_form.html", {"source": source, "action": action, "preview": preview_data, "preview_error": error, "detect_message": message})
+
+@app.post("/preview", response_class=HTMLResponse)
+async def preview(request: Request):
+    form = await request.form(); values = source_values(form); values["url"] = form.get("url", "")
+    try:
+        with _http_client(timeout=45) as client:
+            items = execute_strategy(values, client)[:20]
+        preview_data = [{"title": item.title, "starts_at": item.starts_at.isoformat(), "ends_at": item.ends_at.isoformat() if item.ends_at else None, "url": item.url} for item in items]
+        error = None
+    except Exception as exc: preview_data, error = [], str(exc)
+    source_id = form.get("source_id")
+    return TEMPLATES.TemplateResponse(request, "source_form.html", {"source": values, "action": f"/sources/{source_id}" if source_id else "/sources", "preview": preview_data, "preview_error": error})

app/static/app.js

@@ -0,0 +1,132 @@
+const locationInput = document.querySelector('#location-input');
+const latitudeInput = document.querySelector('#latitude-input');
+const longitudeInput = document.querySelector('#longitude-input');
+const options = document.querySelector('#location-options');
+let locationTimer;
+
+if (locationInput) {
+  locationInput.addEventListener('input', () => {
+    latitudeInput.value = '';
+    longitudeInput.value = '';
+    clearTimeout(locationTimer);
+    const query = locationInput.value.trim();
+    if (query.length < 3) { options.hidden = true; return; }
+    locationTimer = setTimeout(async () => {
+      try {
+        const response = await fetch(`/api/locations?q=${encodeURIComponent(query)}`);
+        const places = await response.json();
+        options.replaceChildren();
+        places.forEach(place => {
+          const button = document.createElement('button');
+          button.type = 'button';
+          button.textContent = place.label;
+          button.addEventListener('click', () => {
+            locationInput.value = place.label;
+            latitudeInput.value = place.latitude;
+            longitudeInput.value = place.longitude;
+            options.hidden = true;
+          });
+          options.append(button);
+        });
+        if (places.length === 0) {
+          const message = document.createElement('div');
+          message.className = 'location-message';
+          message.textContent = 'No geographic locations found. Try a suburb, town, or postcode.';
+          options.append(message);
+        }
+        options.hidden = false;
+      } catch (_) { options.hidden = true; }
+    }, 300);
+  });
+  document.addEventListener('click', event => { if (!event.target.closest('.location-field')) options.hidden = true; });
+}
+
+const tagToggle = document.querySelector('#tag-toggle');
+const tagOptions = document.querySelector('#tag-options');
+if (tagToggle && tagOptions) {
+  tagToggle.addEventListener('click', () => { tagOptions.hidden = !tagOptions.hidden; });
+  tagOptions.addEventListener('change', () => {
+    const count = tagOptions.querySelectorAll('input:checked').length;
+    tagToggle.textContent = count ? `${count} selected` : 'All types';
+  });
+  document.addEventListener('click', event => {
+    if (!event.target.closest('.tag-field')) tagOptions.hidden = true;
+  });
+}
+
+const tabs = document.querySelectorAll('.tab');
+let map;
+tabs.forEach(tab => tab.addEventListener('click', () => {
+  tabs.forEach(item => item.classList.toggle('active', item === tab));
+  document.querySelectorAll('.view-panel').forEach(panel => panel.classList.toggle('active', panel.id === tab.dataset.view));
+  if (tab.dataset.view === 'map-view') initialiseMap();
+}));
+
+const eventRows = [...document.querySelectorAll('[data-event-start]')];
+const visibleCount = document.querySelector('#visible-event-count');
+const clearDay = document.querySelector('#clear-day');
+const dayEmpty = document.querySelector('#day-empty');
+const rangeStatus = document.querySelector('#range-status');
+let rangeStart = '';
+let rangeEnd = '';
+
+function formatDay(dayKey) {
+  return new Date(`${dayKey}T00:00:00`).toLocaleDateString(undefined, { day: 'numeric', month: 'short' });
+}
+
+function filterToRange(startDay, endDay = startDay) {
+  let count = 0;
+  eventRows.forEach(row => {
+    row.hidden = Boolean(startDay) && (row.dataset.eventStart > endDay || row.dataset.eventEnd < startDay);
+    if (!row.hidden) count += 1;
+  });
+  document.querySelectorAll('.calendar-day').forEach(day => {
+    const inRange = Boolean(startDay) && day.dataset.day >= startDay && day.dataset.day <= endDay;
+    day.classList.toggle('selected', inRange);
+    day.classList.toggle('range-start', day.dataset.day === startDay);
+    day.classList.toggle('range-end', day.dataset.day === endDay && endDay !== startDay);
+  });
+  if (visibleCount) visibleCount.textContent = count;
+  if (clearDay) clearDay.hidden = !startDay;
+  if (dayEmpty) dayEmpty.hidden = count !== 0;
+}
+
+document.querySelectorAll('.calendar-day').forEach(day => day.addEventListener('click', event => {
+  event.preventDefault();
+  const clicked = day.dataset.day;
+  if (!rangeStart || rangeEnd) {
+    rangeStart = clicked;
+    rangeEnd = '';
+    filterToRange(rangeStart);
+    if (rangeStatus) rangeStatus.textContent = `${formatDay(rangeStart)} selected · choose an end date`;
+  } else {
+    rangeEnd = clicked;
+    if (rangeEnd < rangeStart) [rangeStart, rangeEnd] = [rangeEnd, rangeStart];
+    filterToRange(rangeStart, rangeEnd);
+    if (rangeStatus) rangeStatus.textContent = `${formatDay(rangeStart)} – ${formatDay(rangeEnd)}`;
+    document.querySelector('#event-list')?.scrollIntoView({ behavior: 'smooth', block: 'start' });
+  }
+}));
+if (clearDay) clearDay.addEventListener('click', event => {
+  event.preventDefault(); rangeStart = ''; rangeEnd = ''; filterToRange('');
+  if (rangeStatus) rangeStatus.textContent = 'Select a start date';
+});
+
+function initialiseMap() {
+  if (map || !window.maplibregl) return;
+  const events = JSON.parse(document.querySelector('#map-events').textContent);
+  const center = events.length ? [events[0].longitude, events[0].latitude] : [150.8931, -34.4278];
+  map = new maplibregl.Map({ container: 'events-map', style: { version: 8, sources: { osm: { type: 'raster', tiles: ['https://tile.openstreetmap.org/{z}/{x}/{y}.png'], tileSize: 256, attribution: '© OpenStreetMap contributors' } }, layers: [{ id: 'osm', type: 'raster', source: 'osm' }] }, center, zoom: 12 });
+  map.addControl(new maplibregl.NavigationControl(), 'top-right');
+  const bounds = new maplibregl.LngLatBounds();
+  events.forEach(event => {
+    const popup = new maplibregl.Popup({ offset: 20 }).setHTML(`<strong>${escapeHtml(event.title)}</strong><br>${escapeHtml(event.venue)}<br><a href="/events/${event.id}">View event</a>`);
+    new maplibregl.Marker({ color: '#dd5b32' }).setLngLat([event.longitude, event.latitude]).setPopup(popup).addTo(map);
+    bounds.extend([event.longitude, event.latitude]);
+  });
+  if (events.length > 1) map.fitBounds(bounds, { padding: 60, maxZoom: 14 });
+}
+
+function escapeHtml(value) {
+  const node = document.createElement('div'); node.textContent = value; return node.innerHTML;
+}

app/static/event.js

@@ -0,0 +1,16 @@
+const mapNode = document.querySelector('#event-map');
+if (mapNode && window.maplibregl) {
+  const longitude = Number(mapNode.dataset.longitude);
+  const latitude = Number(mapNode.dataset.latitude);
+  const map = new maplibregl.Map({ container: mapNode, style: { version: 8, sources: { osm: { type: 'raster', tiles: ['https://tile.openstreetmap.org/{z}/{x}/{y}.png'], tileSize: 256, attribution: '© OpenStreetMap contributors' } }, layers: [{ id: 'osm', type: 'raster', source: 'osm' }] }, center: [longitude, latitude], zoom: 15 });
+  map.addControl(new maplibregl.NavigationControl(), 'top-right');
+  new maplibregl.Marker({ color: '#dd5b32' }).setLngLat([longitude, latitude]).setPopup(new maplibregl.Popup({ offset: 20 }).setText(mapNode.dataset.title)).addTo(map);
+}
+
+const shareButton = document.querySelector('#share-button');
+if (shareButton) shareButton.addEventListener('click', async () => {
+  try {
+    if (navigator.share) await navigator.share({ title: document.title, url: location.href });
+    else { await navigator.clipboard.writeText(location.href); shareButton.textContent = 'Link copied'; }
+  } catch (_) {}
+});

app/static/source.js

@@ -0,0 +1,118 @@
+const pickerFrame = document.querySelector('#selector-picker');
+const loadPicker = document.querySelector('#load-picker');
+const pickerStatus = document.querySelector('#picker-status');
+const pickerSelection = document.querySelector('#picker-selection');
+const pickedSelector = document.querySelector('#picked-selector');
+const pickedText = document.querySelector('#picked-text');
+const pickedCount = document.querySelector('#picked-count');
+const pickedContainer = document.querySelector('#picked-container');
+const pickedRelative = document.querySelector('#picked-relative');
+const pickerTarget = document.querySelector('#picker-target');
+const applySelector = document.querySelector('#apply-selector');
+const applyPattern = document.querySelector('#apply-pattern');
+let currentSelection = null;
+
+if (loadPicker && pickerFrame) loadPicker.addEventListener('click', async () => {
+  const url = document.querySelector('input[name="url"]')?.value.trim();
+  if (!url) { pickerStatus.textContent = 'Enter a page URL first.'; return; }
+  loadPicker.disabled = true; pickerStatus.textContent = 'Downloading and sanitizing page…';
+  try {
+    const body = new FormData(); body.append('url', url);
+    const response = await fetch('/picker/snapshots', { method: 'POST', body });
+    const result = await response.json();
+    if (!response.ok) throw new Error(result.error || 'Unable to create preview');
+    pickerFrame.src = result.preview_url; pickerFrame.hidden = false;
+    pickerStatus.textContent = 'Hover to highlight, then click an element.';
+  } catch (error) { pickerStatus.textContent = error.message; }
+  finally { loadPicker.disabled = false; }
+});
+
+window.addEventListener('message', event => {
+  if (!pickerFrame || event.source !== pickerFrame.contentWindow || event.data?.type !== 'what2do-selector') return;
+  currentSelection = event.data;
+  pickedSelector.textContent = event.data.selector;
+  pickedCount.textContent = `${event.data.selectorCount} exact match${event.data.selectorCount === 1 ? '' : 'es'}; ${event.data.generalizedCount} similar elements`;
+  pickedContainer.textContent = event.data.containerSelector || 'No repeated ancestor detected';
+  pickedRelative.textContent = event.data.containerSelector ? `${event.data.containerCount} containers · child: ${event.data.relativeSelector}` : '';
+  pickedText.textContent = event.data.text || event.data.tag;
+  applyPattern.disabled = !event.data.containerSelector;
+  pickerSelection.hidden = false;
+});
+
+function updateSelectorField(target, selector) {
+  const input = document.querySelector(`[name="${target}"]`);
+  if (input) input.value = selector;
+  try {
+    const strategy = JSON.parse(strategyJson.value || '{}');
+    if ((strategy.mode || 'html') === 'html') {
+      strategy[target] = selector;
+      strategyJson.value = JSON.stringify(strategy, null, 2);
+    }
+  } catch (_) {}
+}
+
+if (applySelector) applySelector.addEventListener('click', () => {
+  if (!currentSelection) return;
+  updateSelectorField(pickerTarget.value, currentSelection.selector);
+  pickerStatus.textContent = `Applied exact selector to ${pickerTarget.options[pickerTarget.selectedIndex].text}.`;
+});
+
+if (applyPattern) applyPattern.addEventListener('click', () => {
+  if (!currentSelection?.containerSelector) return;
+  const target = pickerTarget.value;
+  updateSelectorField('item_selector', currentSelection.containerSelector);
+  updateSelectorField(target, target === 'item_selector' ? currentSelection.containerSelector : currentSelection.relativeSelector);
+  pickerStatus.textContent = `Matched ${currentSelection.containerCount} event containers and applied a relative ${pickerTarget.options[pickerTarget.selectedIndex].text.toLowerCase()} selector.`;
+});
+
+const strategyTemplates = {
+  html: { mode: 'html', request_url: 'https://example.org/events', method: 'GET', item_selector: '.event-card', title_selector: '.title', date_selector: '.date', time_selector: '.time', link_selector: 'a', link_attribute: 'href', description_selector: '.description', location_selector: '.venue', date_format: '' },
+  json: { mode: 'json', request_url: 'https://example.org/api/events', method: 'GET', items_path: 'results', title_path: 'title', date_path: 'startDate', end_path: 'endDate', description_path: 'description', location_path: 'location.address', link_path: 'url', disabled_path: 'disabled' },
+  json_join: { mode: 'json', request_url: 'https://example.org/api/sessions', method: 'GET', items_path: '', date_path: 'startDate', link_path: 'url', join_item_key: 'itemId', join_title_path: 'name', join_description_path: 'description', join_duration_path: 'duration', join: { request_url: 'https://example.org/api/items', method: 'GET', items_path: '', key_path: 'id' } },
+  html_fragment: { mode: 'html_fragment', request_url: 'https://example.org/ajax', method: 'POST', body: { action: 'events' }, html_path: 'html', item_selector: '.event-card', title_selector: '.title', date_selector: '.date', link_selector: 'a', link_attribute: 'href', page_parameter: 'page', page_start: 1, max_pages: 10, total_pages_path: 'pages_total' }
+};
+const strategyTemplate = document.querySelector('#strategy-template');
+const strategyJson = document.querySelector('#strategy-json');
+if (strategyTemplate && strategyJson) strategyTemplate.addEventListener('change', () => {
+  const template = strategyTemplates[strategyTemplate.value];
+  if (template) strategyJson.value = JSON.stringify(template, null, 2);
+});
+
+const addressInput = document.querySelector('#venue-address');
+const addressOptions = document.querySelector('#address-options');
+const latitudeInput = document.querySelector('#venue-latitude');
+const longitudeInput = document.querySelector('#venue-longitude');
+let addressTimer;
+
+if (addressInput && addressOptions) {
+  addressInput.addEventListener('input', () => {
+    clearTimeout(addressTimer);
+    latitudeInput.value = '';
+    longitudeInput.value = '';
+    const query = addressInput.value.trim();
+    if (query.length < 3) { addressOptions.hidden = true; return; }
+    addressTimer = setTimeout(async () => {
+      try {
+        const response = await fetch(`/api/addresses?q=${encodeURIComponent(query)}`);
+        const addresses = await response.json();
+        addressOptions.replaceChildren();
+        addresses.forEach(address => {
+          const button = document.createElement('button');
+          button.type = 'button';
+          button.textContent = address.label;
+          button.addEventListener('click', () => {
+            addressInput.value = address.label;
+            latitudeInput.value = address.latitude;
+            longitudeInput.value = address.longitude;
+            addressOptions.hidden = true;
+          });
+          addressOptions.append(button);
+        });
+        addressOptions.hidden = addresses.length === 0;
+      } catch (_) { addressOptions.hidden = true; }
+    }, 300);
+  });
+  document.addEventListener('click', event => {
+    if (!event.target.closest('.address-field')) addressOptions.hidden = true;
+  });
+}

app/strategies.py

@@ -0,0 +1,164 @@
+import json
+from copy import deepcopy
+from datetime import datetime, timedelta
+from urllib.parse import urljoin
+from zoneinfo import ZoneInfo
+
+from bs4 import BeautifulSoup
+
+from .crawler import BROWSER_HEADERS, CrawlError, ExtractedEvent, _extract_json_ld, _parse_date_range, _weekly_dates, validate_public_url
+
+
+def parse_strategy(value) -> dict:
+    if isinstance(value, dict):
+        return deepcopy(value)
+    try:
+        return json.loads(value or "{}")
+    except (TypeError, json.JSONDecodeError):
+        return {}
+
+
+def json_path(value, path: str, default=None):
+    if not path:
+        return value
+    current = value
+    for component in path.split("."):
+        if current is None:
+            return default
+        if isinstance(current, list):
+            try: current = current[int(component)]
+            except (ValueError, IndexError): return default
+        elif isinstance(current, dict):
+            current = current.get(component, default)
+        else:
+            return default
+    return current
+
+
+def _value(node, selector: str, attribute: str = "") -> str:
+    if not selector:
+        return ""
+    target = node if selector == ":scope" else node.select_one(selector)
+    if not target:
+        return ""
+    if attribute:
+        return str(target.get(attribute, ""))
+    return target.get_text(" ", strip=True)
+
+
+def _request(client, strategy: dict, page: int | None = None):
+    url = str(strategy.get("request_url") or strategy.get("url") or "")
+    validate_public_url(url)
+    method = strategy.get("method", "GET").upper()
+    params = deepcopy(strategy.get("params") or {})
+    body = deepcopy(strategy.get("body") or {})
+    page_parameter = strategy.get("page_parameter", "")
+    if page is not None and page_parameter:
+        (params if method == "GET" else body)[page_parameter] = str(page)
+    headers = {**BROWSER_HEADERS, **(strategy.get("headers") or {})}
+    response = client.request(method, url, params=params, data=body if method != "GET" else None, headers=headers)
+    response.raise_for_status()
+    if len(response.content) > int(strategy.get("max_bytes", 5_000_000)):
+        raise CrawlError("Strategy response exceeds configured size limit.")
+    return response
+
+
+def _events_from_html(html: str, strategy: dict, source: dict) -> list[ExtractedEvent]:
+    soup = BeautifulSoup(html, "html.parser")
+    now = datetime.now(ZoneInfo(source.get("timezone") or "UTC"))
+    if strategy.get("item_selector") == "json-ld":
+        return _extract_json_ld(soup, source["url"], source, now)
+    events = []
+    for item in soup.select(strategy.get("item_selector", "")):
+        title = _value(item, strategy.get("title_selector", ""), strategy.get("title_attribute", ""))
+        raw_date = _value(item, strategy.get("date_selector", ""), strategy.get("date_attribute", ""))
+        for old, new in (strategy.get("date_replacements") or {}).items(): raw_date = raw_date.replace(old, new)
+        raw_time = _value(item, strategy.get("time_selector", ""), strategy.get("time_attribute", ""))
+        if not title or not raw_date: continue
+        config = {**source, "date_format": strategy.get("date_format", "")}
+        if strategy.get("recurring_weekly"):
+            for start in _weekly_dates(raw_date, now):
+                events.append(ExtractedEvent(title=title, starts_at=start, description=_value(item, strategy.get("description_selector", ""), strategy.get("description_attribute", "")), url=urljoin(source["url"], _value(item, strategy.get("link_selector", ""), strategy.get("link_attribute", "href"))), address=_value(item, strategy.get("location_selector", ""), strategy.get("location_attribute", ""))))
+            continue
+        try: start, end = _parse_date_range(f"{raw_date} {raw_time}".strip(), config, now)
+        except (ValueError, OverflowError): continue
+        duration = strategy.get("duration_minutes")
+        if duration and not end: end = start + timedelta(minutes=int(duration))
+        link = _value(item, strategy.get("link_selector", ""), strategy.get("link_attribute", "href"))
+        description = _value(item, strategy.get("description_selector", ""), strategy.get("description_attribute", ""))
+        location = _value(item, strategy.get("location_selector", ""), strategy.get("location_attribute", ""))
+        events.append(ExtractedEvent(title=title, starts_at=start, ends_at=end, description=description, url=urljoin(source["url"], link), address=location))
+    return events
+
+
+def _index_join(items: list, join_items: list, item_key: str, join_key: str) -> dict:
+    index = {}
+    for item in join_items:
+        keys = json_path(item, join_key, [])
+        if not isinstance(keys, list): keys = [keys]
+        for key in keys:
+            if key is not None: index[str(key)] = item
+    return index
+
+
+def _events_from_json(payload, strategy: dict, source: dict, join_index: dict | None = None) -> list[ExtractedEvent]:
+    items = json_path(payload, strategy.get("items_path", ""), [])
+    if not isinstance(items, list): return []
+    now = datetime.now(ZoneInfo(source.get("timezone") or "UTC"))
+    events = []
+    for item in items:
+        if strategy.get("disabled_path") and json_path(item, strategy["disabled_path"]): continue
+        joined = join_index.get(str(json_path(item, strategy.get("join_item_key", "")))) if join_index else None
+        def field(name, default=""):
+            item_path = strategy.get(f"{name}_path", "")
+            join_path = strategy.get(f"join_{name}_path", "")
+            value = json_path(item, item_path, None) if item_path else None
+            if (value is None or value == "") and joined is not None and join_path:
+                value = json_path(joined, join_path, default)
+            return value if value is not None else default
+        title = str(field("title") or strategy.get("title_fallback", "Untitled event"))
+        date_value = str(field("date"))
+        if not date_value: continue
+        try: start, end = _parse_date_range(date_value, source, now)
+        except (ValueError, OverflowError): continue
+        end_value = field("end")
+        if end_value:
+            try: end, _ = _parse_date_range(str(end_value), source, now)
+            except (ValueError, OverflowError): pass
+        duration = field("duration") or strategy.get("duration_minutes")
+        if duration and not end: end = start + timedelta(minutes=int(duration))
+        events.append(ExtractedEvent(title=title, starts_at=start, ends_at=end, description=str(field("description")), url=urljoin(source["url"], str(field("link"))), address=str(field("location"))))
+    return events
+
+
+def execute_strategy(source: dict, client) -> list[ExtractedEvent]:
+    strategy = parse_strategy(source.get("strategy_json"))
+    mode = strategy.get("mode", "html")
+    if mode == "html":
+        response = _request(client, strategy)
+        return _events_from_html(response.text, strategy, source)
+    if mode not in {"json", "html_fragment"}:
+        raise CrawlError(f"Unsupported extraction mode: {mode}")
+    join_index = None
+    join = strategy.get("join")
+    if join:
+        join_response = _request(client, join)
+        join_payload = join_response.json()
+        join_items = json_path(join_payload, join.get("items_path", ""), [])
+        join_index = _index_join([], join_items if isinstance(join_items, list) else [], strategy.get("join_item_key", ""), join.get("key_path", ""))
+    events = []
+    page = int(strategy.get("page_start", 1))
+    max_pages = min(int(strategy.get("max_pages", 1)), 20)
+    for _ in range(max_pages):
+        response = _request(client, strategy, page)
+        payload = response.json()
+        if mode == "html_fragment":
+            fragment = json_path(payload, strategy.get("html_path", "html"), "")
+            page_events = _events_from_html(str(fragment), strategy, source)
+        else:
+            page_events = _events_from_json(payload, strategy, source, join_index)
+        events.extend(page_events)
+        total_pages = json_path(payload, strategy.get("total_pages_path", ""), None)
+        if not strategy.get("page_parameter") or not page_events or (total_pages is not None and page >= int(total_pages)): break
+        page += 1
+    return events

app/templates/base.html

@@ -0,0 +1,5 @@
+<!doctype html>
+<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
+<title>{% block title %}What2Do{% endblock %}</title><link rel="stylesheet" href="/static/style.css">{% block head %}{% endblock %}</head>
+<body><header><a class="brand" href="/">What2Do</a><nav><a href="/">Find events</a><a href="/sources">Crawler sources</a></nav></header>
+<main>{% block content %}{% endblock %}</main><footer>Local-first event discovery · No AI, tracking, or proprietary map dependency</footer>{% block scripts %}{% endblock %}</body></html>

app/templates/event.html

@@ -0,0 +1,22 @@
+{% extends "base.html" %}
+{% block title %}{{ event.title }} · What2Do{% endblock %}
+{% block head %}<meta name="description" content="{{ event.title }} at {{ event.venue_name }}"><meta property="og:title" content="{{ event.title }}"><meta property="og:description" content="{{ event.display_date }} at {{ event.display_time }} · {{ event.venue_name }}"><link href="https://unpkg.com/maplibre-gl@5.7.1/dist/maplibre-gl.css" rel="stylesheet">{% endblock %}
+{% block content %}
+<a href="/" class="back-link">← All events</a>
+<article class="event-detail">
+  <div>
+    <div class="eyebrow">{{ event.display_date }}</div>
+    <div class="detail-title">{{ event.title }}</div>
+    <div class="detail-time">{{ event.display_time }}{% if event.display_end %} – {{ event.display_end }}{% endif %}</div>
+    {% if event.description %}<div class="description">{{ event.description }}</div>{% endif %}
+    <div class="detail-card card">
+      <div class="detail-field"><div class="detail-label">Venue</div><div class="detail-value">{{ event.venue_name }}</div></div>
+      <div class="detail-field"><div class="detail-label">Address</div><div class="detail-value">{{ event.address }}</div></div>
+      <div class="detail-field"><div class="detail-label">Source</div><div class="detail-value"><a href="{{ event.event_url }}" target="_blank" rel="noopener">Original event page ↗</a></div></div>
+    </div>
+    <div class="detail-actions"><button type="button" id="share-button">Share event</button><a class="button secondary" href="{{ event.event_url }}" target="_blank" rel="noopener">Visit source</a></div>
+  </div>
+  <div>{% if event.latitude is not none %}<div id="event-map" class="detail-map card" data-latitude="{{ event.latitude }}" data-longitude="{{ event.longitude }}" data-title="{{ event.venue_name }}"></div>{% else %}<div class="empty card">No map coordinates are available for this event.</div>{% endif %}</div>
+</article>
+{% endblock %}
+{% block scripts %}<script src="https://unpkg.com/maplibre-gl@5.7.1/dist/maplibre-gl.js"></script><script src="/static/event.js"></script>{% endblock %}

app/templates/event_form.html

@@ -0,0 +1,6 @@
+{% extends "base.html" %}
+{% block title %}Edit event · What2Do{% endblock %}
+{% block content %}
+<div class="section-head"><div><div class="eyebrow">EDIT NORMALIZED EVENT</div><div class="form-title">{{ event.title }}</div><div class="muted">Changes may be overwritten on the next crawl if the source still publishes different values.</div></div><a href="/sources/{{ source.id }}/events">← Source events</a></div>
+<form class="card event-admin-form" method="post" action="/sources/{{ source.id }}/events/{{ event.id }}"><div class="field">Title<input required name="title" value="{{ event.title }}"></div><div class="field">Description<textarea name="description" rows="6">{{ event.description }}</textarea></div><div class="two"><div class="field">Starts at<input required name="starts_at" value="{{ event.starts_at }}"></div><div class="field">Ends at<input name="ends_at" value="{{ event.ends_at or '' }}"></div></div><div class="two"><div class="field">Venue name<input required name="venue_name" value="{{ event.venue_name }}"></div><div class="field">Address<input required name="address" value="{{ event.address }}"></div></div><div class="two"><div class="field">Latitude<input type="number" step="any" name="latitude" value="{{ event.latitude if event.latitude is not none else '' }}"></div><div class="field">Longitude<input type="number" step="any" name="longitude" value="{{ event.longitude if event.longitude is not none else '' }}"></div></div><div class="field">Event URL<input required type="url" name="event_url" value="{{ event.event_url }}"></div><div class="sticky-actions"><input type="submit" class="primary" value="Save event"></div></form>
+{% endblock %}

app/templates/index.html

@@ -0,0 +1,31 @@
+{% extends "base.html" %}
+{% block title %}Find events · What2Do{% endblock %}
+{% block head %}<link href="https://unpkg.com/maplibre-gl@5.7.1/dist/maplibre-gl.css" rel="stylesheet">{% endblock %}
+{% block content %}
+<section class="hero"><div class="eyebrow">LOCAL EVENT INDEX</div><div class="hero-title">Find something worth going out for.</div><div class="hero-copy">Search events gathered from local venue websites.</div></section>
+<form class="search search-filters card" method="get" id="search-form">
+  <div class="field location-field">Near<input id="location-input" name="location" value="{{ location }}" placeholder="Start typing a suburb or address" autocomplete="off"><input type="hidden" id="latitude-input" name="latitude" value="{{ latitude if latitude is not none else '' }}"><input type="hidden" id="longitude-input" name="longitude" value="{{ longitude if longitude is not none else '' }}"><div id="location-options" class="location-options" hidden></div></div>
+  <div class="field">Search term<input name="term" value="{{ term }}" placeholder="Band, venue, activity..."></div>
+  <div class="field tag-field">Event types<button type="button" id="tag-toggle" class="tag-toggle">{{ selected_tags|length ~ ' selected' if selected_tags else 'All types' }}</button><div id="tag-options" class="tag-options" hidden>{% for option in event_tags %}<label class="tag-option"><input type="checkbox" name="tag" value="{{ option }}" {% if option in selected_tags %}checked{% endif %}><span>{{ option|title }}</span></label>{% endfor %}<label class="tag-option"><input type="checkbox" name="tag" value="other" {% if 'other' in selected_tags %}checked{% endif %}><span></span>Other</span></label></div></div>
+  <div class="field range-field">Within<input type="number" min="1" max="500" name="radius" value="{{ radius }}"><span class="suffix">km</span></div>
+  <button type="submit">Search</button>
+</form>
+{% if location and not geocoded %}<div class="notice error">That location could not be resolved. Select an autocomplete suggestion to apply distance filtering.</div>{% endif %}
+<div class="section-head"><div><div class="eyebrow">WHAT'S ON</div><div class="section-title"><span id="visible-event-count">{{ events|length }}</span> events</div></div><a class="button secondary" href="/sources/new">+ Add source</a></div>
+<div class="view-tabs"><button type="button" class="tab active" data-view="list-view">List</button><button type="button" class="tab" data-view="map-view">Map</button></div>
+<section id="list-view" class="view-panel active">
+  <div class="date-calendar card">
+    <div class="calendar-heading">Choose a range · {{ calendar_month }}</div>
+    <div class="calendar-tools"><span id="range-status">Select a start date</span><button type="button" id="clear-day" class="clear-day" hidden>Show all</button></div>
+    <div class="weekdays"><input type="button" value="Mon" disabled><input type="button" value="Tue" disabled><input type="button" value="Wed" disabled><input type="button" value="Thu" disabled><input type="button" value="Fri" disabled><input type="button" value="Sat" disabled><input type="button" value="Sun" disabled></div>
+    <div class="calendar-grid">{% for day in calendar_days %}<button type="button" class="calendar-day {{ '' if day.in_month else 'outside' }} {{ 'has-events' if day.count else '' }}" data-day="{{ day.key }}"><span>{{ day.date.day }}</span>{% if day.count %}<b>{{ day.count }}</b>{% endif %}</button>{% endfor %}</div>
+  </div>
+  <div id="event-list" class="day-events">
+    {% for event in events %}<a class="event-row card" href="/events/{{ event.id }}" data-event-start="{{ event.day_key }}" data-event-end="{{ event.end_day_key }}"><div><div class="date">{{ event.display_time }}</div><div class="event-title">{{ event.title }}</div><div class="muted">{{ event.venue_name }} · {{ event.address }}</div><div class="tag-list">{% for event_tag in event.tags %}<span class="pill">{{ event_tag }}</span>{% endfor %}</div></div><div class="event-link-label">View →</div></a>{% else %}<div class="empty card">No events match these filters.</div>{% endfor %}
+    <div id="day-empty" class="empty card" hidden>No events occur on this day.</div>
+  </div>
+</section>
+<section id="map-view" class="view-panel"><div id="events-map" class="map card"></div></section>
+<script id="map-events" type="application/json">{{ map_events|tojson }}</script>
+{% endblock %}
+{% block scripts %}<script src="https://unpkg.com/maplibre-gl@5.7.1/dist/maplibre-gl.js"></script><script src="/static/app.js"></script>{% endblock %}

app/templates/source_events.html

@@ -0,0 +1,6 @@
+{% extends "base.html" %}
+{% block title %}{{ source.name }} events · What2Do{% endblock %}
+{% block content %}
+<div class="section-head"><div><div class="eyebrow">SOURCE EVENTS</div><div class="form-title">{{ source.name }}</div><div class="muted">Edit normalized events or hide them from public search and maps.</div></div><a href="/sources">← Sources</a></div>
+<div class="admin-events">{% for event in events %}<article class="card admin-event {{ 'is-hidden' if event.hidden else '' }}"><div><div class="date">{{ event.starts_at }}{% if event.ends_at %} → {{ event.ends_at }}{% endif %}</div><h3>{{ event.title }}</h3><div class="muted">{{ event.venue_name }} · {{ event.address }}</div>{% if event.hidden %}<span class="pill">Hidden</span>{% endif %}</div><div class="actions"><a class="button secondary" href="/sources/{{ source.id }}/events/{{ event.id }}">Edit</a><form method="post" action="/sources/{{ source.id }}/events/{{ event.id }}/visibility"><button>{{ 'Show' if event.hidden else 'Hide' }}</button></form><a class="button secondary" href="/events/{{ event.id }}">Public page</a></div></article>{% else %}<div class="empty card">No events have been crawled for this source.</div>{% endfor %}</div>
+{% endblock %}

app/templates/source_form.html

@@ -0,0 +1,27 @@
+{% extends "base.html" %}
+{% block title %}Configure source · What2Do{% endblock %}
+{% block content %}
+<div class="section-head"><div><div class="eyebrow">SOURCE CONFIGURATION</div><div class="form-title">Configure website</div><div class="muted">Auto-detect standard HTML or configure a reusable extraction strategy.</div></div><a href="/sources">← Sources</a></div>
+{% if detect_message %}<div class="notice success">{{ detect_message }} Review the strategy below before saving.</div>{% endif %}
+{% if preview_error %}<div class="notice error">Detection or preview failed: {{ preview_error }}</div>{% endif %}
+<form class="config-grid" method="post" action="{{ action }}">
+<section class="card form-section"><div class="form-heading">Website and venue</div>
+<div class="field">Source name<input required name="name" value="{{ source['name'] or '' }}"></div>
+<div class="field">Page URL<div class="url-detect-row"><input required type="url" name="url" value="{{ source['url'] or '' }}" placeholder="https://venue.example/events"><input class="detect-submit" type="submit" value="Auto-detect HTML" formaction="/auto-detect" formnovalidate></div></div>
+<div class="field">Venue name<input required name="venue_name" value="{{ source['venue_name'] or '' }}"></div>
+<div class="field address-field">Venue address<input required id="venue-address" name="venue_address" value="{{ source['venue_address'] or '' }}" autocomplete="off"><div id="address-options" class="location-options" hidden></div></div>
+<div class="two"><div class="field">Latitude<input id="venue-latitude" type="number" step="any" name="latitude" value="{{ source['latitude'] if source['latitude'] is not none else '' }}"></div><div class="field">Longitude<input id="venue-longitude" type="number" step="any" name="longitude" value="{{ source['longitude'] if source['longitude'] is not none else '' }}"></div></div>
+<div class="field">Timezone<input required name="timezone" value="{{ source['timezone'] or 'Australia/Sydney' }}"></div></section>
+<section class="card picker-panel"><div class="form-heading">Visual selector picker</div><div class="hint">Loads a sanitized, non-interactive copy of the downloaded HTML. Dynamic JavaScript content may not appear.</div><div class="picker-controls"><label class="field">Assign clicks to<select id="picker-target"><option value="item_selector">Event container</option><option value="title_selector">Title</option><option value="date_selector">Date or range</option><option value="time_selector">Time</option><option value="link_selector">Link</option><option value="description_selector">Description</option><option value="location_selector">Location</option></select></label><button type="button" id="load-picker">Load visual picker</button></div><div id="picker-status" class="hint"></div><iframe id="selector-picker" class="selector-picker" sandbox="allow-scripts" title="Sanitized website selector picker" hidden></iframe><div id="picker-selection" class="picker-selection" hidden><div><strong>Sele</strong>cted</strong><code id="picked-selector"></code><span id="picked-count"></span></div><div><strong>R</strong>epeated event pattern</strong><code id="picked-container"></code><span id="picked-relative"></span></div><span id="picked-text"></span><div class="picker-actions"><button type="button" id="apply-selector">Use exact selector</button><button type="button" id="apply-pattern">Use repeated pattern</button></div></div></section>
+<details class="card rules-details" {% if detect_message or preview_error %}open{% endif %}><summary>Extraction strategy — advanced</summary><section class="form-section">
+<div class="hint">HTML mode uses the convenient selector fields below. JSON and HTML-fragment modes use the generic JSON strategy directly.</div>
+<div class="two"><div class="field">Event container selector<input name="item_selector" value="{{ source['item_selector'] or '' }}"></div><div class="field">Title selector<input name="title_selector" value="{{ source['title_selector'] or '' }}"></div><div class="field">Date or range selector<input name="date_selector" value="{{ source['date_selector'] or '' }}"></div><div class="field">Time selector<input name="time_selector" value="{{ source['time_selector'] or '' }}"></div><div class="field">Link selector<input name="link_selector" value="{{ source['link_selector'] or '' }}"></div><div class="field">Description selector<input name="description_selector" value="{{ source['description_selector'] or '' }}"></div><div class="field">Location selector<input name="location_selector" value="{{ source['location_selector'] or '' }}"></div><div class="field">Date format<input name="date_format" value="{{ source['date_format'] or '' }}"></div></div>
+<label class="check"><input type="checkbox" name="recurring_weekly" {% if source['recurring_weekly'] %}checked{% endif %}> Generate weekly occurrences from weekday labels</label>
+<div class="field">Strategy template<select id="strategy-template"><option value="">Keep current strategy</option><option value="html">HTML page</option><option value="json">JSON API</option><option value="json_join">JSON API with metadata join</option><option value="html_fragment">API returning HTML fragments</option></select></div><div class="field">Persisted strategy JSON<textarea id="strategy-json" name="strategy_json" rows="18">{{ source['strategy_json'] or '{}' }}</textarea><div class="hint">Supports request configuration, dotted JSON paths, CSS selectors and attributes, pagination, joins, and response limits. See STRATEGIES.md.</div></div>
+</section></details>
+<section class="card form-section"><div class="form-heading">Scheduling</div><div class="field">Crawl every (minutes)<input type="number" min="15" name="crawl_interval_minutes" value="{{ source['crawl_interval_minutes'] or 360 }}"></div><label class="check"><input type="checkbox" name="enabled" {% if source['enabled'] %}checked{% endif %}> Enable scheduled crawling</label></section>
+<div class="sticky-actions">{% if source.get('id') %}<input type="hidden" name="source_id" value="{{ source['id'] }}">{% endif %}<input type="submit" value="Test strategy" formaction="/preview"><input type="submit" class="primary" value="Save source"></div></form>
+
+{% if preview is not none %}<section class="card preview"><div class="form-heading">Preview ({{ preview|length }})</div>{% for event in preview %}<div><strong>{{ event.title }}</strong><span>{{ event.starts_at }}{% if event.ends_at %} → {{ event.ends_at }}{% endif %}</span></div>{% else %}<p>No events matched this strategy.</p>{% endfor %}</section>{% endif %}
+{% endblock %}
+{% block scripts %}<script src="/static/source.js"></script>{% endblock %}

app/templates/sources.html

@@ -0,0 +1,6 @@
+{% extends "base.html" %}
+{% block title %}Sources · What2Do{% endblock %}
+{% block content %}
+<div class="section-head"><div><div class="eyebrow">CRAWLER ADMIN</div><div class="form-title">Sources</div><div class="muted">Configure crawlers and manage their normalized events.</div></div><a class="button" href="/sources/new">+ Add website</a></div>
+<div class="source-grid">{% for source in sources %}<article class="card source"><div class="source-title"><div><h3>{{ source.name }}</h3><a href="{{ source.url }}" target="_blank" rel="noopener">{{ source.url }}</a></div><span class="pill {{ 'good' if source.enabled else '' }}">{{ 'Enabled' if source.enabled else 'Paused' }}</span></div><div class="source-stats"><div><small>Venue</</small>small><strong>{{ source.venue_name }}</strong></div><div><small>Events</small></small><strong>{{ source.event_count }}</strong></div><div><small></small>Last result</small><strong>{{ source.last_status or 'Not crawled' }}</strong></div></div>{% if source.last_error %}<div class="notice error">{{ source.last_error }}</div>{% endif %}<div class="actions"><form method="post" action="/sources/{{ source.id }}/crawl"><button>Crawl now</button></form><a class="button secondary" href="/sources/{{ source.id }}">Configure</a><a class="button secondary" href="/sources/{{ source.id }}/events">Manage events</a><form method="post" action="/sources/{{ source.id }}/delete" onsubmit="return confirm('Delete this source and its events?')"><button class="danger">Delete</button></form></div></article>{% endfor %}</div>
+{% endblock %}

compose.yaml

@@ -0,0 +1,16 @@
+services:
+  web:
+    build: .
+    ports:
+      - "8000:8000"
+    environment:
+      DATABASE_PATH: /data/what2do.db
+      PUBLIC_BASE_URL: http://localhost:8000
+      CRAWL_INTERVAL_MINUTES: "60"
+      NOMINATIM_USER_AGENT: what2do-local/0.1 (set-a-contact@example.com)
+    volumes:
+      - what2do-data:/data
+    restart: unless-stopped
+
+volumes:
+  what2do-data:

requirements.txt

@@ -0,0 +1,7 @@
+fastapi==0.116.1
+uvicorn==0.35.0
+httpx==0.28.1
+beautifulsoup4==4.13.4
+python-multipart==0.0.20
+jinja2==3.1.6
+python-dateutil==2.9.0.post0