118 lines
7.2 KiB
Markdown
118 lines
7.2 KiB
Markdown
# Chatbot MCP server
|
||
|
||
The website chatbot runs in n8n. Its AI Agent answers from a read-only MCP server in this app instead
|
||
of scraping the websites. The server combines two sources:
|
||
|
||
- **Editorial content from TYPO3**: the content feed of all public sites (ep-reisen, ser,
|
||
ski-boarderweek, ski-wochenenden, snowzone), mirrored into `content_document` by `app:content:sync`.
|
||
- **Trip data of our own**: dates, rooms, prices, availability and pickups from the product export
|
||
(`ProductExportBuilder`, the data behind `GET /api/exports/products`).
|
||
|
||
TYPO3 sends no dates or prices, and nothing flows back to TYPO3.
|
||
|
||
## Endpoint
|
||
|
||
`POST /mcp` (Streamable HTTP; `GET`/`DELETE` are part of the transport), served by `symfony/mcp-bundle`
|
||
from `config/packages/mcp.yaml`. Tools only, no resources or prompts.
|
||
|
||
Authentication is a static bearer token per client, `Authorization: Bearer <token>`, checked by
|
||
`App\Security\McpAccessTokenHandler` on the `mcp` firewall. `APP_MCP_TOKENS` lists the clients as
|
||
`name:token` pairs separated by commas, e.g. `APP_MCP_TOKENS="n8n:3f9c…,claude-desktop:a71e…"`. Give every
|
||
system its own token, so one can be revoked by removing its entry without touching the others. Use tokens
|
||
without commas, e.g. `openssl rand -hex 32`. A malformed list (no name, a duplicate name, a token shared by
|
||
two clients) fails loudly with a 500 on `/mcp` rather than locking a client out silently. If the list is
|
||
empty, every request is refused. It is not the OAuth2 client-credentials flow of `/api`: n8n's MCP Client Tool node sends a fixed
|
||
header and cannot renew a token every 10 minutes.
|
||
|
||
The client name becomes the authenticated user. Tool calls are rate-limited per client (`chatbot_mcp`
|
||
limiter, 60 per minute), so testing from Claude Desktop does not eat into n8n's budget. The limit is there to stop
|
||
runaway agent loops. Each call is logged on the `chatbot` channel (`var/log/<env>.chatbot.log`, file
|
||
only) with the client, tool, arguments, result count and duration. Nothing identifies the person chatting.
|
||
|
||
### n8n
|
||
|
||
MCP Client Tool node: endpoint `https://<host>/mcp`, transport *HTTP Streamable*, authentication *Bearer*
|
||
with n8n's own token from the stage's `APP_MCP_TOKENS`. The agent prompt should tell the bot to cite each answer's `url`, and
|
||
that prices and availability are a snapshot (`dataAsOf`) and only the booking page is binding.
|
||
|
||
## Tools
|
||
|
||
| Tool | Use | Returns |
|
||
|---|---|---|
|
||
| `search_trips` | Which trips exist, when, where, how much | Up to 10 trips with their next 3 matching dates, `priceFrom`, availability, `dataAsOf` |
|
||
| `get_trip` | One trip by product `code`, or one date of it by date code such as `SSLMP061226` (optionally `hotelCode`) | Product text with FAQs, hotels with facts, all upcoming dates with bookable rooms, pickups for the first 3 dates |
|
||
| `search_content` | Questions not about dates or prices | Up to 8 hits with a ~600-character excerpt |
|
||
| `get_document` | The full text of one `search_content` hit, by `url` | Markdown body, truncated at 20 000 characters |
|
||
|
||
Every result carries the public `url`. Output is compact JSON with Markdown in the text fields.
|
||
An empty result is `{"results": []}`. Invalid input produces a tool error the model can read.
|
||
|
||
Trip rules (`App\Chatbot\TripCatalog`):
|
||
|
||
- TYPO3 products are joined to the export on `product.code`. Several products can share a code, and
|
||
each is listed as its own trip. Hotels are joined on any entry of the TYPO3 hotel's `codes` (BusPro
|
||
uses the matchcode aliases). A code that is one hotel's own and another's alias resolves to its owner.
|
||
- A room is bookable unless its status is `Buchungsstop`. `available` counts rooms, so the places of a
|
||
date at a hotel are `Σ available × maxPax` over its bookable rooms. `persons` filters on that sum, and
|
||
`maxPrice` on the lowest bookable room price. The tools never output room counts, only `frei` or
|
||
`auf Anfrage`.
|
||
- `search_trips` only offers products that TYPO3 lists in its own website search (`searchable`) and does
|
||
not mark as unbookable. Closed group trips and staff trainings are `searchable: false`. `get_trip` still
|
||
answers when someone names their code.
|
||
- The export's `guide` (name, phone) and all BusPro ids are never output.
|
||
|
||
Search (`App\Chatbot\Search\ContentSearch`) is MariaDB full text in boolean mode over
|
||
`title, breadcrumb, body_md`. Each term becomes a prefix match, because InnoDB has no German stemming,
|
||
and German stopwords are dropped in PHP. Relevance is weighted by type: products and hotels, then
|
||
destinations and concepts, then pages, then blog posts.
|
||
|
||
## Content sync
|
||
|
||
```bash
|
||
php bin/console app:content:sync # delta: what the manifest says changed
|
||
php bin/console app:content:sync --refresh # also refetch items older than 7 days, oldest first
|
||
php bin/console app:content:sync --ignore-window # deliberate bulk run outside 01:00–05:00
|
||
php bin/console app:content:check-freshness # fails if the last good manifest is > 60 min old
|
||
```
|
||
|
||
Not scheduled yet. The intended schedule is a delta sync every 15 minutes, `--refresh` on Sunday at
|
||
01:00, and the freshness check hourly, so that `email_on_failure` reports a stalled sync.
|
||
|
||
The feed is read with `X-Api-Key: <APP_CMS_API_KEY>` from `APP_CMS_API_BASE_URL`. The manifest lists
|
||
every site whichever host serves it. The key is only sent to hosts of the sites the manifest names.
|
||
|
||
**Load rules.** These are agreed with the TYPO3 side, so do not tune them:
|
||
|
||
- One request at a time, and at most one page fetch per second.
|
||
- A response slower than 2 s makes the next request wait 5 s. Three slow pages in a row end the page
|
||
phase for the run.
|
||
- A `503` is TYPO3's kill switch. The run stops, and `Retry-After` pauses later runs (`content_sync_state.paused_until`).
|
||
Three consecutive 5xx responses or timeouts (10 s) also stop the run. Nothing is retried in a loop.
|
||
- Outside 01:00–05:00 Europe/Berlin a run fetches at most 50 pages and leaves the rest for later.
|
||
|
||
**Robustness.**
|
||
|
||
- A failed, invalid or empty manifest changes nothing.
|
||
- A record that is requested but absent from the records response is deleted, and so is an item that
|
||
is missing from the manifest.
|
||
- A page answering 404 or 5xx is remembered as `failed`, and one that converts to no text as `empty`.
|
||
Each is stored with its `modified` and not refetched until that changes or a `--refresh` reaches it.
|
||
- A timeout is retried on the next run.
|
||
|
||
**Document model.** One `content_document` row per `(type, uid, language)`. A record's structured
|
||
payload is kept in `data`, and HTML is converted to Markdown (`HtmlToMarkdownConverter`). A page whose
|
||
`detailOf` names a record is appended to that record's `body_md` and kept out of the index
|
||
(`merged_into`), so a product is never found twice. Merges are recomputed on every run. A record without
|
||
a reachable page (`site`/`url` null) is not shown in search results, unless a detail page lends it its URL.
|
||
|
||
After a complete run that changed something, the codes the trip join cannot place are logged on `core`
|
||
(`Content feed: unmatched codes`).
|
||
|
||
## Configuration
|
||
|
||
| Variable | Meaning |
|
||
|---|---|
|
||
| `APP_CMS_API_BASE_URL` | TYPO3 base URL per stage (also used by `CmsDataProvider`) |
|
||
| `APP_CMS_API_KEY` | TYPO3 `X-Api-Key`: the content feed and its `contentUrl` pages, and `typo3.client` (`CmsDataProvider`) |
|
||
| `APP_MCP_TOKENS` | Bearer tokens for `/mcp`, one per client: `name:token,name:token` |
|