what2do/README.md

What2Do

A lightweight, local-first event aggregator. It crawls configured venue pages with CSS selectors or schema.org JSON-LD, normalizes events, and provides date/location/radius search.

No LLM, ML, computer vision, browser automation, JavaScript build process, PostGIS, Redis, or external database is required.

Run

docker compose up --build

Open http://localhost:8000. The source admin is at http://localhost:8000/sources.

Two example sources are installed automatically:

• Dicey Riley's recurring weekly specials
• La La La's dated event list

Click Crawl now beside each source after startup. Crawls then run on each source's configured schedule while the app is running.

Extraction model

Rules are relative to a repeating event container:

• Event container, title, date, time, link, description, and location are CSS selectors.
• Date parsing accepts a Python strptime format or flexible text parsing.
• json-ld as the event container reads schema.org Event entries without CSS rules.
• Weekly pages can create the next 12 occurrences from weekday labels.
• The source's venue address is the fallback when event HTML omits a location.

Use Test extraction to preview up to 20 records before saving. Static server-rendered HTML is intentionally used; sites that only render events with JavaScript are not supported in this low-overhead version.

Location search

Addresses are geocoded and cached using OpenStreetMap's public Nominatim service. The search field also accepts latitude,longitude directly, and every source has optional coordinate fields for deterministic/offline venue positioning. Set a real contact in NOMINATIM_USER_AGENT before public/shared use and follow the Nominatim usage policy. For heavier use, configure or self-host a compatible geocoder.

Operational notes

• SQLite data is persisted in the what2do-data Docker volume.
• Fetches are limited to 5 MB and public HTTP(S) destinations to reduce SSRF risk.
• The crawler does not bypass authentication or anti-bot systems.
• Review each site's terms and robots.txt; only crawl sources you are permitted to index.
• The MVP has no authentication. Bind it to localhost or place it behind an authenticated reverse proxy before exposing crawler administration to a network.