192 lines
13 KiB
Markdown
192 lines
13 KiB
Markdown
# Chatbot MCP server
|
||
|
||
The website chatbot runs in n8n. Its AI Agent answers from a read-only MCP server in this app instead
|
||
of scraping the websites. The server combines two sources:
|
||
|
||
- **Editorial content from TYPO3**: the content feed of all public sites (ep-reisen, ser,
|
||
ski-boarderweek, ski-wochenenden, snowzone), mirrored into `content_document` by `app:content:sync`.
|
||
- **Trip data of our own**: dates, rooms, prices, availability and pickups from the product export
|
||
(`ProductExportBuilder`, the data behind `GET /api/exports/products`).
|
||
|
||
TYPO3 sends no dates or prices, and nothing flows back to TYPO3.
|
||
|
||
## Endpoint
|
||
|
||
`POST /mcp` (Streamable HTTP; `GET`/`DELETE` are part of the transport), served by `symfony/mcp-bundle`
|
||
from `config/packages/mcp.yaml`. Tools only, no resources or prompts. `POST /mcp/trips` is the same
|
||
server without the editorial content, see [Trips only](#trips-only-mcptrips).
|
||
|
||
Authentication is a static bearer token per client, `Authorization: Bearer <token>`, checked by
|
||
`App\Security\McpAccessTokenHandler` on the `mcp` firewall. `APP_MCP_TOKENS` lists the clients as
|
||
`name:token` pairs separated by commas, e.g. `APP_MCP_TOKENS="n8n:3f9c…,claude-desktop:a71e…"`. Give every
|
||
system its own token, so one can be revoked by removing its entry without touching the others. Use tokens
|
||
without commas, e.g. `openssl rand -hex 32`. A malformed list (no name, a duplicate name, a token shared by
|
||
two clients) fails loudly with a 500 on `/mcp` rather than locking a client out silently. If the list is
|
||
empty, every request is refused. It is not the OAuth2 client-credentials flow of `/api`: n8n's MCP Client Tool node sends a fixed
|
||
header and cannot renew a token every 10 minutes.
|
||
|
||
The client name becomes the authenticated user. Tool calls are rate-limited per client (`chatbot_mcp`
|
||
limiter, 60 per minute), so testing from Claude Desktop does not eat into n8n's budget. The limit is there to stop
|
||
runaway agent loops. Each call is logged on the `chatbot` channel (`var/log/<env>.chatbot.log`, file
|
||
only) with the client, tool, arguments, result count and duration. Nothing identifies the person chatting.
|
||
|
||
### n8n
|
||
|
||
MCP Client Tool node: endpoint `https://<host>/mcp`, transport *HTTP Streamable*, authentication *Bearer*
|
||
with n8n's own token from the stage's `APP_MCP_TOKENS`. The agent prompt should tell the bot to cite each answer's `url`, and
|
||
that prices and availability are a snapshot (`dataAsOf`) and only the booking page is binding.
|
||
|
||
## Tools
|
||
|
||
| Tool | Use | Returns |
|
||
|---|---|---|
|
||
| `search_trips` | The website's travel search: which trips exist, when, where, how much | One result per trip × hotel × travel date (default 10, max 25), with trip `code` and `hotelCode`, "ab" price, nights, inclusions, room types, availability, link; `total` and `dataAsOf` |
|
||
| `get_trip` | One trip by product `code`, or one date of it by date code such as `SSLMP061226` (optionally `hotelCode`) | Product text with FAQs, hotels with facts, all upcoming dates with bookable rooms, pickups for the first 3 dates |
|
||
| `search_content` | Questions not about dates or prices | Up to 8 hits with a ~600-character excerpt |
|
||
| `get_document` | The full text of one `search_content` hit, by `url` | Markdown body, truncated at 20 000 characters |
|
||
|
||
### Trips only: `/mcp/trips`
|
||
|
||
A second server (`chatbot_trips` in `config/packages/mcp.yaml`) for clients that must not answer from
|
||
editorial website text, which can contradict the booking system, most visibly on prices. A client picks
|
||
the variant by the path it is pointed at; tokens, rate limit and log are those of `/mcp`.
|
||
|
||
- Only `search_trips` and `get_trip`. `search_content` and `get_document` do not exist here.
|
||
- `get_trip` (`App\Chatbot\Mcp\TripsOnly\GetTripTool`) leaves out the product `description`, which
|
||
includes the FAQs, and each hotel's `description`.
|
||
- What remains from TYPO3 is structured: trip and hotel names, `url`, `site`, destination, concept and
|
||
the hotel `facts` (beds, altitude, distances, sauna, wifi, …; no prices). Which trips exist is still
|
||
decided by TYPO3's products and their `searchable` flag, and the free-text `query` of `search_trips`
|
||
still falls back to the page texts to find a trip, without returning any of it.
|
||
- The server instructions tell the model to refer to the trip's `url` for anything the tools don't answer.
|
||
|
||
To give a client the editorial content back, point it at `/mcp` again.
|
||
|
||
Editorial text (product descriptions, FAQs, pages) is maintained by hand in TYPO3 and can lag behind BusPro, e.g. an
|
||
old price or a changed inclusion. The server instructions and the tool descriptions therefore tell the model that the
|
||
structured trip fields of `search_trips` and `get_trip` override whatever the text says about dates, prices, pickups
|
||
and inclusions.
|
||
|
||
Every result carries the public `url`. Output is compact JSON with Markdown in the text fields.
|
||
An empty result is `{"results": []}`. Invalid input produces a tool error the model can read.
|
||
|
||
Trip rules (`App\Chatbot\TripCatalog`):
|
||
|
||
- TYPO3 products are joined to the export on `product.code`. Several products can share a code, and
|
||
each is listed as its own trip. Hotels are joined on any entry of the TYPO3 hotel's `codes` (BusPro
|
||
uses the matchcode aliases). A code that is one hotel's own and another's alias resolves to its owner.
|
||
- BusPro has hotel names of its own (`hotel.xml`, joined on the hotel id). They are internal wording
|
||
("NEU Sportclub Lederer Wochenenden und Kurztrips", "Vacanceole Résidence L'Edelweiss"), so a hotel is
|
||
shown under its TYPO3 title, and under BusPro's name only when TYPO3 has no hotel for its code.
|
||
`query` searches both. Trips have no name in BusPro's export, only a code; their names are TYPO3's.
|
||
- A room is bookable unless its status is `Buchungsstop`. The baby row (room code `Baby`) is not a room.
|
||
`available` counts rooms, so the places of a date at a hotel are `Σ available × maxPax` over its
|
||
bookable rooms. The tools never output room counts, only `frei` or `auf Anfrage`.
|
||
- `search_trips` only offers products that TYPO3 lists in its own website search (`searchable`) and does
|
||
not mark as unbookable. Closed group trips and staff trainings are `searchable: false`. `get_trip` still
|
||
answers when someone names their code.
|
||
- Pickup surcharges are per person and per leg, so each pickup carries `surchargeOutbound`, `surchargeReturn` and
|
||
their sum `surchargeRoundTrip`. A single figure was read by the model as the whole price. A leg listed without a
|
||
price is `0`, because BusPro bills only priced legs; `surchargeReturn` is absent only where the point is not
|
||
served on the way back.
|
||
- The export's `guide` (name, phone) and all BusPro ids are never output.
|
||
|
||
`search_trips` mirrors the search page (`/skireisen/suche/`), whose data is TYPO3's import of our own
|
||
product export. Its rules follow TYPO3's `DateImportService` and search repository, derived per date and
|
||
hotel in `App\Model\Chatbot\TravelDateFacts`:
|
||
|
||
- **"ab" price:** the cheapest bookable room with places left, otherwise the cheapest bookable room.
|
||
`priceMin`/`priceMax` apply to it. The site's price bands are presets of this.
|
||
- **Travel period:** `dateFrom`/`dateTo` match by containment (start on or after, end on or before), as on
|
||
the site, not by overlap. `period` names one of the site's presets, from the `chatbot_travel_periods`
|
||
parameter in `config/services.yaml`. It's duplicated from the TYPO3 search configuration on purpose, so
|
||
update both each season. Expired periods are no longer offered.
|
||
- **`persons`:** the site's condition (a room takes groups this small: smallest `minPax` ≤ persons) **and**
|
||
enough places left. The site checks only the first.
|
||
- **Room type:** capacity, as TYPO3's `RoomTypeResolver` (`App\Enum\RoomType`): `maxPax` 1–8, 9 or more
|
||
is `9er+`. A label containing "bett im" or "mit ander" is `mit anderen` (a shared bed, sold per bed).
|
||
- **Accommodation:** the TYPO3 hotel `type` (1 Ferienwohnung, 2 Sportclub, 3 Apartment, 4 Gästehaus,
|
||
5 Pension), `App\Enum\HotelType`.
|
||
- **Board:** the items of selection group 71 ("4x Abendessen", "Frühstück inkl. … Lunchpaket", "Vollpension",
|
||
…), matched by word against any item. The site keys on the first item only. "Halbpension" occurs in no
|
||
label, so dinner is asked as "Abendessen".
|
||
- **Inclusions:** selection groups 7 (Skipass), 71 (board), 86 (Programm), 87 (Kurse), 89 (Events), and the
|
||
outbound bus when its price is 0. `bus: true` asks for any bookable outbound bus.
|
||
- **Nights:** days between start and end. The site's "short" is ≤ 4 and its "long" is ≥ 5.
|
||
- **`query`:** the site has no free text. A code or a name is taken at its word, so that naming a trip or
|
||
hotel returns that one and not every page mentioning it. The first of these that finds anything decides
|
||
(`TripCatalog::resolveQuery()`, `App\Chatbot\Search\NameQuery`):
|
||
1. a product code, a hotel code or alias, or a date code;
|
||
2. names holding every word of the query, compared without accents, case and German stopwords, each
|
||
word as a prefix: "Lederer" is "Sportclub Lederer", "Chaviere" is "Val Chavière". Names are those of
|
||
products, hotels (TYPO3's and BusPro's), cities, regions, countries and concepts;
|
||
3. names the query spells out among other words ("Hotel Chamois d'Or in Val Thorens"), where the most
|
||
specific kind wins: product or hotel, then city, region, country or concept;
|
||
4. the page texts of those records, every term required, then any term.
|
||
|
||
A matching hotel selects its own travel dates, not all hotels of its trips. A destination or concept
|
||
selects its trips. Matches of one step are combined, so "Österreich Sportclub" is everything in Austria
|
||
plus every Sportclub; `country` and `concept` narrow instead.
|
||
- **Order:** by start date, then "ab" price. The site sorts by concept first, which the feed doesn't carry.
|
||
- Results link to the product page. Links with the hotel preselected and the site's "Restplätze" badge need
|
||
two fields the TYPO3 content feed doesn't send yet (`hotelUrls`, `lowContingent`).
|
||
|
||
Search (`App\Chatbot\Search\ContentSearch`) is MariaDB full text in boolean mode over
|
||
`title, breadcrumb, body_md`. Each term becomes a prefix match, because InnoDB has no German stemming,
|
||
and German stopwords are dropped in PHP. Relevance is weighted by type: products and hotels, then
|
||
destinations and concepts, then pages, then blog posts.
|
||
|
||
## Content sync
|
||
|
||
```bash
|
||
php bin/console app:content:sync # delta: what the manifest says changed
|
||
php bin/console app:content:sync --refresh # also refetch items older than 7 days, oldest first
|
||
php bin/console app:content:sync --ignore-window # deliberate bulk run outside 01:00–05:00
|
||
php bin/console app:content:check-freshness # fails if the last good manifest is > 60 min old
|
||
```
|
||
|
||
Scheduled on prod (`config/packages/zenstruck_schedule.yaml`): an hourly delta sync,
|
||
`--refresh` on Sunday at 01:05, and the freshness check hourly at :10 with `--max-age 150`, so that
|
||
`email_on_failure` reports a stalled sync but not a single missed run. Website content changes
|
||
rarely, and an hour's delay is acceptable for the chatbot; trip data and availability do not come
|
||
from this mirror. The refresh is off the quarter hour because a run that
|
||
finds another one holding the lock exits successfully without doing anything.
|
||
|
||
The feed is read with `X-Api-Key: <APP_CMS_API_KEY>` from `APP_CMS_API_BASE_URL`. The manifest lists
|
||
every site whichever host serves it. The key is only sent to hosts of the sites the manifest names.
|
||
|
||
**Load rules.** These are agreed with the TYPO3 side, so do not tune them:
|
||
|
||
- One request at a time, and at most one page fetch per second.
|
||
- A response slower than 2 s makes the next request wait 5 s. Three slow pages in a row end the page
|
||
phase for the run.
|
||
- A `503` is TYPO3's kill switch. The run stops, and `Retry-After` pauses later runs (`content_sync_state.paused_until`).
|
||
Three consecutive 5xx responses or timeouts (10 s) also stop the run. Nothing is retried in a loop.
|
||
- Outside 01:00–05:00 Europe/Berlin a run fetches at most 50 pages and leaves the rest for later.
|
||
|
||
**Robustness.**
|
||
|
||
- A failed, invalid or empty manifest changes nothing.
|
||
- A record that is requested but absent from the records response is deleted, and so is an item that
|
||
is missing from the manifest.
|
||
- A page answering 404 or 5xx is remembered as `failed`, and one that converts to no text as `empty`.
|
||
Each is stored with its `modified` and not refetched until that changes or a `--refresh` reaches it.
|
||
- A timeout is retried on the next run.
|
||
|
||
**Document model.** One `content_document` row per `(type, uid, language)`. A record's structured
|
||
payload is kept in `data`, and HTML is converted to Markdown (`HtmlToMarkdownConverter`). A page whose
|
||
`detailOf` names a record is appended to that record's `body_md` and kept out of the index
|
||
(`merged_into`), so a product is never found twice. Merges are recomputed on every run. A record without
|
||
a reachable page (`site`/`url` null) is not shown in search results, unless a detail page lends it its URL.
|
||
|
||
After a complete run that changed something, the codes the trip join cannot place are logged on `core`
|
||
(`Content feed: unmatched codes`).
|
||
|
||
## Configuration
|
||
|
||
| Variable | Meaning |
|
||
|---|---|
|
||
| `APP_CMS_API_BASE_URL` | TYPO3 base URL per stage (also used by `CmsDataProvider`) |
|
||
| `APP_CMS_API_KEY` | TYPO3 `X-Api-Key`: the content feed and its `contentUrl` pages, and `typo3.client` (`CmsDataProvider`) |
|
||
| `APP_MCP_TOKENS` | Bearer tokens for `/mcp` and `/mcp/trips`, one per client: `name:token,name:token` |
|