feat: chatbot trip search finds trips by trip name, hotel name and code
This commit is contained in:
fromme committed
1 parent
26d4140c18
commit
fe40e1807e
8 files changed
+448
-52
No files matched your search
+20
-2
@@ -40,7 +40,7 @@ that prices and availability are a snapshot (`dataAsOf`) and only the booking pa
|
||||
|
||||
| Tool | Use | Returns |
|
||||
|---|---|---|
|
||||
| `search_trips` | The website's travel search: which trips exist, when, where, how much | One result per trip × hotel × travel date (default 10, max 25), with "ab" price, nights, inclusions, room types, availability, link; `total` and `dataAsOf` |
|
||||
| `search_trips` | The website's travel search: which trips exist, when, where, how much | One result per trip × hotel × travel date (default 10, max 25), with trip `code` and `hotelCode`, "ab" price, nights, inclusions, room types, availability, link; `total` and `dataAsOf` |
|
||||
| `get_trip` | One trip by product `code`, or one date of it by date code such as `SSLMP061226` (optionally `hotelCode`) | Product text with FAQs, hotels with facts, all upcoming dates with bookable rooms, pickups for the first 3 dates |
|
||||
| `search_content` | Questions not about dates or prices | Up to 8 hits with a ~600-character excerpt |
|
||||
| `get_document` | The full text of one `search_content` hit, by `url` | Markdown body, truncated at 20 000 characters |
|
||||
@@ -57,7 +57,7 @@ the variant by the path it is pointed at; tokens, rate limit and log are those o
|
||||
- What remains from TYPO3 is structured: trip and hotel names, `url`, `site`, destination, concept and
|
||||
the hotel `facts` (beds, altitude, distances, sauna, wifi, …; no prices). Which trips exist is still
|
||||
decided by TYPO3's products and their `searchable` flag, and the free-text `query` of `search_trips`
|
||||
still matches page text to find a trip, without returning any of it.
|
||||
still falls back to the page texts to find a trip, without returning any of it.
|
||||
- The server instructions tell the model to refer to the trip's `url` for anything the tools don't answer.
|
||||
|
||||
To give a client the editorial content back, point it at `/mcp` again.
|
||||
@@ -75,6 +75,10 @@ Trip rules (`App\Chatbot\TripCatalog`):
|
||||
- TYPO3 products are joined to the export on `product.code`. Several products can share a code, and
|
||||
each is listed as its own trip. Hotels are joined on any entry of the TYPO3 hotel's `codes` (BusPro
|
||||
uses the matchcode aliases). A code that is one hotel's own and another's alias resolves to its owner.
|
||||
- BusPro has hotel names of its own (`hotel.xml`, joined on the hotel id). They are internal wording
|
||||
("NEU Sportclub Lederer Wochenenden und Kurztrips", "Vacanceole Résidence L'Edelweiss"), so a hotel is
|
||||
shown under its TYPO3 title, and under BusPro's name only when TYPO3 has no hotel for its code.
|
||||
`query` searches both. Trips have no name in BusPro's export, only a code; their names are TYPO3's.
|
||||
- A room is bookable unless its status is `Buchungsstop`. The baby row (room code `Baby`) is not a room.
|
||||
`available` counts rooms, so the places of a date at a hotel are `Σ available × maxPax` over its
|
||||
bookable rooms. The tools never output room counts, only `frei` or `auf Anfrage`.
|
||||
@@ -109,6 +113,20 @@ hotel in `App\Model\Chatbot\TravelDateFacts`:
|
||||
- **Inclusions:** selection groups 7 (Skipass), 71 (board), 86 (Programm), 87 (Kurse), 89 (Events), and the
|
||||
outbound bus when its price is 0. `bus: true` asks for any bookable outbound bus.
|
||||
- **Nights:** days between start and end. The site's "short" is ≤ 4 and its "long" is ≥ 5.
|
||||
- **`query`:** the site has no free text. A code or a name is taken at its word, so that naming a trip or
|
||||
hotel returns that one and not every page mentioning it. The first of these that finds anything decides
|
||||
(`TripCatalog::resolveQuery()`, `App\Chatbot\Search\NameQuery`):
|
||||
1. a product code, a hotel code or alias, or a date code;
|
||||
2. names holding every word of the query, compared without accents, case and German stopwords, each
|
||||
word as a prefix: "Lederer" is "Sportclub Lederer", "Chaviere" is "Val Chavière". Names are those of
|
||||
products, hotels (TYPO3's and BusPro's), cities, regions, countries and concepts;
|
||||
3. names the query spells out among other words ("Hotel Chamois d'Or in Val Thorens"), where the most
|
||||
specific kind wins: product or hotel, then city, region, country or concept;
|
||||
4. the page texts of those records, every term required, then any term.
|
||||
|
||||
A matching hotel selects its own travel dates, not all hotels of its trips. A destination or concept
|
||||
selects its trips. Matches of one step are combined, so "Österreich Sportclub" is everything in Austria
|
||||
plus every Sportclub; `country` and `concept` narrow instead.
|
||||
- **Order:** by start date, then "ab" price. The site sorts by concept first, which the feed doesn't carry.
|
||||
- Results link to the product page. Links with the hotel preselected and the site's "Restplätze" badge need
|
||||
two fields the TYPO3 content feed doesn't send yet (`hotelUrls`, `lowContingent`).
|
||||
|
||||
@@ -19,7 +19,7 @@ use Mcp\Schema\ToolAnnotations;
|
||||
|
||||
#[McpTool(
|
||||
name: 'search_trips',
|
||||
description: 'The website\'s travel search: find bookable trips by travel period, destination, concept, price, group size, accommodation, board, room type, duration and bus. Use for any question about which trips exist, when, where or how much. Returns one result per trip, hotel and travel date with the "ab" price per person in EUR, nights, what is included, room types and a link; get_trip gives the details of one. Prices and availability are a snapshot as of dataAsOf; the booking page is binding.',
|
||||
description: 'The website\'s travel search: find bookable trips by travel period, destination, concept, price, group size, accommodation, board, room type, duration and bus. Use for any question about which trips exist, when, where or how much. Returns one result per trip, hotel and travel date with the "ab" price per person in EUR, nights, what is included, room types and a link; get_trip gives the details of one, by its code and, for one hotel, its hotelCode. Prices and availability are a snapshot as of dataAsOf; the booking page is binding.',
|
||||
annotations: new ToolAnnotations(readOnlyHint: true, openWorldHint: false),
|
||||
)]
|
||||
final class SearchTripsTool
|
||||
@@ -32,7 +32,7 @@ final class SearchTripsTool
|
||||
}
|
||||
|
||||
public function __invoke(
|
||||
#[Schema(description: 'Free text matched against trip, hotel, destination and concept names and texts, e.g. "Val Thorens" or "Snowboardkurs".')]
|
||||
#[Schema(description: 'A trip, hotel, destination or concept name, or a trip, hotel or date code, e.g. "Sportclub Lederer", "Chamois d\'Or", "Val Thorens", "SSLMP". A hotel name returns only that hotel. Use the name alone, without a question around it. Words that name nothing are looked up in the page texts, e.g. "Snowboardkurs".')]
|
||||
?string $query = null,
|
||||
#[Schema(description: 'A named travel period of the website search: Vorsaison, um Weihnachten, Silvester, 1 Januarwoche, Ostern, Nachsaison. Use instead of dateFrom/dateTo.')]
|
||||
?string $period = null,
|
||||
|
||||
@@ -19,7 +19,7 @@ final class ContentSearchQuery
|
||||
private const MIN_TOKEN_LENGTH = 3;
|
||||
private const MAX_TERMS = 12;
|
||||
|
||||
private const STOPWORDS = [
|
||||
public const STOPWORDS = [
|
||||
'aber', 'alle', 'allem', 'allen', 'aller', 'alles', 'als', 'also', 'am', 'an', 'ander', 'andere', 'anderen',
|
||||
'auch', 'auf', 'aus', 'bei', 'beim', 'bin', 'bis', 'bist', 'da', 'damit', 'dann', 'das', 'dass', 'dein',
|
||||
'deine', 'dem', 'den', 'denn', 'der', 'des', 'dessen', 'die', 'dies', 'diese', 'diesem', 'diesen', 'dieser',
|
||||
@@ -55,12 +55,14 @@ final class ContentSearchQuery
|
||||
}
|
||||
|
||||
/**
|
||||
* The boolean-mode query, or null when nothing searchable is left.
|
||||
* The boolean-mode query, or null when nothing searchable is left. With $allTerms a
|
||||
* document has to match every term, which a name or a phrase wants and a question
|
||||
* does not.
|
||||
*/
|
||||
public static function booleanQuery(string $text): ?string
|
||||
public static function booleanQuery(string $text, bool $allTerms = false): ?string
|
||||
{
|
||||
$terms = self::terms($text);
|
||||
|
||||
return [] === $terms ? null : implode(' ', array_map(static fn (string $term): string => $term.'*', $terms));
|
||||
return [] === $terms ? null : implode(' ', array_map(static fn (string $term): string => ($allTerms ? '+' : '').$term.'*', $terms));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace App\Chatbot\Search;
|
||||
|
||||
use Symfony\Component\String\UnicodeString;
|
||||
|
||||
/**
|
||||
* Compares free text with the names of trips, hotels, destinations and concepts.
|
||||
*
|
||||
* Names are compared word by word, without accents and case ("Chaviere" is "Chavière"),
|
||||
* and without German stopwords. Unlike the full-text index, two-letter words count: they
|
||||
* tell "Chamois d'Or" from "Chalets Chamois".
|
||||
*/
|
||||
final class NameQuery
|
||||
{
|
||||
private const MIN_TOKEN_LENGTH = 2;
|
||||
|
||||
/**
|
||||
* @return list<string>
|
||||
*/
|
||||
public static function tokens(string $text): array
|
||||
{
|
||||
$words = preg_split('/[^\p{L}\p{N}]+/u', mb_strtolower($text), -1, \PREG_SPLIT_NO_EMPTY) ?: [];
|
||||
$tokens = [];
|
||||
|
||||
foreach ($words as $word) {
|
||||
if (in_array($word, ContentSearchQuery::STOPWORDS, true)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$token = (new UnicodeString($word))->ascii()->lower()->toString();
|
||||
|
||||
if (strlen($token) >= self::MIN_TOKEN_LENGTH || ctype_digit($token)) {
|
||||
$tokens[$token] = true;
|
||||
}
|
||||
}
|
||||
|
||||
return array_map(strval(...), array_keys($tokens));
|
||||
}
|
||||
|
||||
/**
|
||||
* Every word of the query begins a word of the name: "Arolles" is within
|
||||
* "3* Hotel Les Arolles (inkl. VP)".
|
||||
*
|
||||
* @param list<string> $query
|
||||
* @param list<string> $name
|
||||
*/
|
||||
public static function within(array $query, array $name): bool
|
||||
{
|
||||
if ([] === $query || [] === $name) {
|
||||
return false;
|
||||
}
|
||||
|
||||
foreach ($query as $word) {
|
||||
if ([] === array_filter($name, static fn (string $nameWord): bool => str_starts_with($nameWord, $word))) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* The query spells the name out among other words: "Hotel Chamois d'Or in Val Thorens"
|
||||
* contains "Chamois d'Or".
|
||||
*
|
||||
* @param list<string> $query
|
||||
* @param list<string> $name
|
||||
*/
|
||||
public static function contains(array $query, array $name): bool
|
||||
{
|
||||
return [] !== $name && [] === array_diff($name, $query);
|
||||
}
|
||||
}
|
||||
+194
-40
@@ -5,7 +5,9 @@ declare(strict_types=1);
|
||||
namespace App\Chatbot;
|
||||
|
||||
use App\BusProNet\Utility\DateCodeUtility;
|
||||
use App\BusProNet\XmlLoader\HotelLoader;
|
||||
use App\Chatbot\Search\ContentSearchQuery;
|
||||
use App\Chatbot\Search\NameQuery;
|
||||
use App\Entity\ContentDocument;
|
||||
use App\Enum\HotelType;
|
||||
use App\Enum\RoomType;
|
||||
@@ -24,9 +26,11 @@ use Symfony\Component\Clock\ClockInterface;
|
||||
* The join is on the product code; several TYPO3 products may share one, and each is its
|
||||
* own trip. Hotels are joined on any of the TYPO3 hotel's `codes`, because BusPro uses the
|
||||
* matchcode aliases; a code carried by more than one hotel resolves to the one whose own
|
||||
* `code` it is.
|
||||
* `code` it is. BusPro's own hotel names (hotel.xml) are internal wording, so they are
|
||||
* only searched, and shown just where TYPO3 has no hotel for a code.
|
||||
*
|
||||
* search() mirrors the website's travel search, get() details one trip. A room is bookable
|
||||
* search() mirrors the website's travel search, get() details one trip. Free text is read
|
||||
* as a code or a name before it is looked up in the page texts, see resolveQuery(). A room is bookable
|
||||
* unless its status is `Buchungsstop`; places and prices are derived per date x hotel by
|
||||
* TravelDateFacts. Room counts never leave this class: the website does not show them, so
|
||||
* the tools say only whether a date is freely bookable or on request.
|
||||
@@ -47,6 +51,7 @@ class TripCatalog
|
||||
public function __construct(
|
||||
private readonly ProductExportBuilder $exportBuilder,
|
||||
private readonly ContentDocumentRepository $repository,
|
||||
private readonly HotelLoader $hotelLoader,
|
||||
private readonly ClockInterface $clock,
|
||||
private readonly LoggerInterface $logger,
|
||||
) {
|
||||
@@ -63,6 +68,7 @@ class TripCatalog
|
||||
{
|
||||
$catalog = $this->load();
|
||||
$today = $this->today();
|
||||
$named = null === $criteria->query ? null : $this->resolveQuery($catalog, $criteria->query);
|
||||
$results = [];
|
||||
|
||||
foreach ($this->candidateProducts($catalog, $criteria) as $product) {
|
||||
@@ -75,6 +81,15 @@ class TripCatalog
|
||||
|
||||
$hotel = $catalog['hotelsByCode'][$this->normalize($row['hotel']['code'] ?? null)][0] ?? null;
|
||||
$hotelData = $hotel?->getData() ?? [];
|
||||
|
||||
if (null !== $named
|
||||
&& !isset($named['products'][$product->getUid()])
|
||||
&& !isset($named['hotels'][$hotel?->getUid() ?? 0])
|
||||
&& !isset($named['busProHotels'][$row['hotel']['busProId'] ?? 0])
|
||||
&& (null === $named['dateCode'] || $named['dateCode'] !== $this->normalize($row['date']['code'] ?? null))) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$facts = TravelDateFacts::fromExportRow($row);
|
||||
|
||||
if (!$this->matches($row, $facts, HotelType::tryFrom((int) ($hotelData['type'] ?? 0)), $criteria)) {
|
||||
@@ -87,7 +102,8 @@ class TripCatalog
|
||||
'product' => $header['name'],
|
||||
'code' => $header['code'],
|
||||
'dateCode' => $row['date']['code'],
|
||||
'hotel' => $hotel?->getTitle() ?? $row['hotel']['code'],
|
||||
'hotel' => $this->hotelName($catalog, $hotel, $row),
|
||||
'hotelCode' => $row['hotel']['code'],
|
||||
'accommodation' => HotelType::tryFrom((int) ($hotelData['type'] ?? 0))?->label(),
|
||||
'destination' => $header['destination'],
|
||||
'concept' => $header['concept'],
|
||||
@@ -228,6 +244,7 @@ class TripCatalog
|
||||
* productsByCode: array<string, list<ContentDocument>>,
|
||||
* hotelsByCode: array<string, list<ContentDocument>>,
|
||||
* rowsByCode: array<string, list<array<string, mixed>>>,
|
||||
* busProHotels: array<int, array{code: string, name: ?string}>,
|
||||
* }
|
||||
*/
|
||||
private function load(): array
|
||||
@@ -264,11 +281,21 @@ class TripCatalog
|
||||
}
|
||||
|
||||
$rowsByCode = [];
|
||||
$busProHotels = [];
|
||||
$masterData = $this->hotelLoader->loadAll();
|
||||
|
||||
foreach ($export->rows as $row) {
|
||||
$code = $this->normalize($row['product']['code'] ?? null);
|
||||
if ('' !== $code) {
|
||||
$rowsByCode[$code][] = $row;
|
||||
}
|
||||
|
||||
$hotelId = (int) ($row['hotel']['busProId'] ?? 0);
|
||||
if (0 !== $hotelId && !isset($busProHotels[$hotelId])) {
|
||||
// BusPro breaks names over lines ("Vacanceole\nRésidence L'Edelweiss").
|
||||
$name = trim((string) preg_replace('/\s+/u', ' ', isset($masterData[$hotelId]) ? (string) $masterData[$hotelId]->name : ''));
|
||||
$busProHotels[$hotelId] = ['code' => $this->normalize($row['hotel']['code'] ?? null), 'name' => '' === $name ? null : $name];
|
||||
}
|
||||
}
|
||||
|
||||
return [
|
||||
@@ -277,12 +304,14 @@ class TripCatalog
|
||||
'productsByCode' => $productsByCode,
|
||||
'hotelsByCode' => $hotelsByCode,
|
||||
'rowsByCode' => $rowsByCode,
|
||||
'busProHotels' => $busProHotels,
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* Published products narrowed by everything the content index can resolve: site, day
|
||||
* trip, concept, destination and free text.
|
||||
* trip, concept and destination. Free text is applied per travel date, since it may
|
||||
* name one hotel of a trip.
|
||||
*
|
||||
* @param array<string, mixed> $catalog
|
||||
*
|
||||
@@ -297,7 +326,6 @@ class TripCatalog
|
||||
$countryUids = null === $criteria->country ? null : $this->matchingUids($records, ['country'], $criteria->country, matchCode: true);
|
||||
$regionUids = null === $criteria->region ? null : $this->matchingUids($records, ['region'], $criteria->region);
|
||||
$cityUids = null === $criteria->region ? null : $this->matchingUids($records, ['city'], $criteria->region);
|
||||
$queryUids = null === $criteria->query ? null : $this->productUidsMatchingText($records, $criteria->query);
|
||||
|
||||
$products = [];
|
||||
|
||||
@@ -315,8 +343,7 @@ class TripCatalog
|
||||
|| (null !== $criteria->site && $record->getSite() !== $criteria->site)
|
||||
|| (null !== $criteria->daytrip && (bool) ($data['daytrip'] ?? false) !== $criteria->daytrip)
|
||||
|| (null !== $conceptUids && !in_array($data['conceptUid'] ?? null, $conceptUids, true))
|
||||
|| (null !== $countryUids && !in_array($data['countryUid'] ?? null, $countryUids, true))
|
||||
|| (null !== $queryUids && !in_array($record->getUid(), $queryUids, true))) {
|
||||
|| (null !== $countryUids && !in_array($data['countryUid'] ?? null, $countryUids, true))) {
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -366,27 +393,155 @@ class TripCatalog
|
||||
}
|
||||
|
||||
/**
|
||||
* Products reached by a full-text match on themselves or on a hotel, destination or
|
||||
* concept they belong to.
|
||||
* What a free-text query asks for: whole trips, single hotels of trips, or one travel
|
||||
* date. Null when it restricts nothing.
|
||||
*
|
||||
* @param array<string, ContentDocument> $records
|
||||
* A code or a name is taken at its word, so naming a trip or hotel returns that one and
|
||||
* not every page that mentions it. In order, stopping at the first that finds anything:
|
||||
* a product, hotel or date code; names holding every word of the query; names the query
|
||||
* spells out among other words; and only then the page texts.
|
||||
*
|
||||
* @return list<int>
|
||||
* @param array<string, mixed> $catalog
|
||||
*
|
||||
* Hotels are known twice: by their TYPO3 record and by BusPro's own name and code, which
|
||||
* also covers a hotel TYPO3 has no record for
|
||||
*
|
||||
* @return array{products: array<int, true>, hotels: array<int, true>, busProHotels: array<int, true>, dateCode: ?string}|null
|
||||
*/
|
||||
private function productUidsMatchingText(array $records, string $text): array
|
||||
private function resolveQuery(array $catalog, string $text): ?array
|
||||
{
|
||||
$booleanQuery = ContentSearchQuery::booleanQuery($text);
|
||||
/** @var array<string, ContentDocument> $records */
|
||||
$records = $catalog['records'];
|
||||
|
||||
if (null === $booleanQuery) {
|
||||
return array_values(array_map(static fn (ContentDocument $record): int => $record->getUid(), array_filter($records, static fn (ContentDocument $record): bool => 'product' === $record->getType())));
|
||||
$matched = $this->recordsWithCode($catalog, $text) ?? $this->recordsNamed($catalog, $text);
|
||||
|
||||
if (null !== $matched) {
|
||||
return $this->tripsOf($records, $matched);
|
||||
}
|
||||
|
||||
$dateCode = (new DateCodeUtility())->sanitize(trim($text));
|
||||
|
||||
foreach ($catalog['export']->rows as $row) {
|
||||
if ($this->normalize($row['date']['code'] ?? null) === $dateCode) {
|
||||
return ['products' => [], 'hotels' => [], 'busProHotels' => [], 'dateCode' => $dateCode];
|
||||
}
|
||||
}
|
||||
|
||||
$terms = ContentSearchQuery::terms($text);
|
||||
|
||||
if ([] === $terms) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Every term first: "Sauna Whirlpool" means both. A question rarely uses the page's
|
||||
// own words for everything, so any term is the fallback.
|
||||
foreach (count($terms) > 1 ? [true, false] : [false] as $allTerms) {
|
||||
$matched = [];
|
||||
foreach ($this->repository->matchRecords((string) ContentSearchQuery::booleanQuery($text, $allTerms), self::RECORD_TYPES, self::LANGUAGE) as $hit) {
|
||||
$matched[$hit['type']][$hit['uid']] = true;
|
||||
}
|
||||
|
||||
if ([] !== $matched) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return $this->tripsOf($records, $matched);
|
||||
}
|
||||
|
||||
/**
|
||||
* The products and the hotel carrying the text as their code, hotel aliases included.
|
||||
* Matches are keyed by record type, BusPro's hotels as `busProHotel` by their id.
|
||||
*
|
||||
* @param array<string, mixed> $catalog
|
||||
*
|
||||
* @return array<string, array<int, true>>|null
|
||||
*/
|
||||
private function recordsWithCode(array $catalog, string $text): ?array
|
||||
{
|
||||
$code = (new DateCodeUtility())->sanitize(trim($text));
|
||||
$matched = [];
|
||||
foreach ($this->repository->matchRecords($booleanQuery, self::RECORD_TYPES, self::LANGUAGE) as $hit) {
|
||||
$matched[$hit['type']][$hit['uid']] = true;
|
||||
|
||||
foreach ($catalog['productsByCode'][$code] ?? [] as $product) {
|
||||
$matched['product'][$product->getUid()] = true;
|
||||
}
|
||||
|
||||
$uids = [];
|
||||
if (isset($catalog['hotelsByCode'][$code][0])) {
|
||||
$matched['hotel'][$catalog['hotelsByCode'][$code][0]->getUid()] = true;
|
||||
}
|
||||
|
||||
foreach ($catalog['busProHotels'] as $hotelId => $busProHotel) {
|
||||
if ($busProHotel['code'] === $code) {
|
||||
$matched['busProHotel'][$hotelId] = true;
|
||||
}
|
||||
}
|
||||
|
||||
return [] === $matched ? null : $matched;
|
||||
}
|
||||
|
||||
/**
|
||||
* The records the text names. A name holding every word of the query wins: "Lederer"
|
||||
* is "Sportclub Lederer". Otherwise the query may spell a name out among other words,
|
||||
* and there the most specific kind wins, so that "Sportclub Lederer Saalbach" is the
|
||||
* hotel and not everything in Saalbach.
|
||||
*
|
||||
* @param array<string, mixed> $catalog
|
||||
*
|
||||
* @return array<string, array<int, true>>|null
|
||||
*/
|
||||
private function recordsNamed(array $catalog, string $text): ?array
|
||||
{
|
||||
$query = NameQuery::tokens($text);
|
||||
$names = [];
|
||||
|
||||
foreach ($catalog['records'] as $record) {
|
||||
$names[] = [$record->getType(), $record->getUid(), $record->getTitle()];
|
||||
}
|
||||
|
||||
foreach ($catalog['busProHotels'] as $hotelId => $busProHotel) {
|
||||
$names[] = ['busProHotel', $hotelId, $busProHotel['name'] ?? ''];
|
||||
}
|
||||
|
||||
$within = [];
|
||||
$contained = [];
|
||||
|
||||
foreach ($names as [$type, $id, $title]) {
|
||||
$name = NameQuery::tokens($title);
|
||||
|
||||
if (NameQuery::within($query, $name)) {
|
||||
$within[$type][$id] = true;
|
||||
} elseif (NameQuery::contains($query, $name)) {
|
||||
$contained[$type][$id] = true;
|
||||
}
|
||||
}
|
||||
|
||||
if ([] !== $within) {
|
||||
return $within;
|
||||
}
|
||||
|
||||
foreach ([['product', 'hotel', 'busProHotel'], ['city'], ['region'], ['country', 'concept']] as $types) {
|
||||
$matched = array_intersect_key($contained, array_flip($types));
|
||||
|
||||
if ([] !== $matched) {
|
||||
return $matched;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Matched records as trips: a product, destination or concept stands for the whole
|
||||
* trips, a hotel only for itself, since a trip may have many.
|
||||
*
|
||||
* @param array<string, ContentDocument> $records
|
||||
* @param array<string, array<int, true>> $matched by type and uid
|
||||
*
|
||||
* @return array{products: array<int, true>, hotels: array<int, true>, busProHotels: array<int, true>, dateCode: null}
|
||||
*/
|
||||
private function tripsOf(array $records, array $matched): array
|
||||
{
|
||||
$products = [];
|
||||
|
||||
foreach ($records as $record) {
|
||||
if ('product' !== $record->getType()) {
|
||||
@@ -394,33 +549,18 @@ class TripCatalog
|
||||
}
|
||||
|
||||
$data = $record->getData() ?? [];
|
||||
$hotelUids = is_array($data['hotelUids'] ?? null) ? $data['hotelUids'] : [];
|
||||
$regionUids = array_merge([$data['regionUid'] ?? null], is_array($data['additionalRegionUids'] ?? null) ? $data['additionalRegionUids'] : []);
|
||||
|
||||
$hit = isset($matched['product'][$record->getUid()])
|
||||
|| [] !== array_intersect($hotelUids, array_keys($matched['hotel'] ?? []))
|
||||
if (isset($matched['product'][$record->getUid()])
|
||||
|| [] !== array_intersect($regionUids, array_keys($matched['region'] ?? []))
|
||||
|| isset($matched['country'][$data['countryUid'] ?? -1])
|
||||
|| isset($matched['city'][$data['cityUid'] ?? -1])
|
||||
|| isset($matched['concept'][$data['conceptUid'] ?? -1]);
|
||||
|
||||
// Hotels also name their products, for products whose hotelUids miss them.
|
||||
if (!$hit) {
|
||||
foreach (array_keys($matched['hotel'] ?? []) as $hotelUid) {
|
||||
$hotelProducts = ($records['hotel:'.$hotelUid] ?? null)?->getData()['productUids'] ?? [];
|
||||
if (is_array($hotelProducts) && in_array($record->getUid(), $hotelProducts, true)) {
|
||||
$hit = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if ($hit) {
|
||||
$uids[] = $record->getUid();
|
||||
|| isset($matched['concept'][$data['conceptUid'] ?? -1])) {
|
||||
$products[$record->getUid()] = true;
|
||||
}
|
||||
}
|
||||
|
||||
return $uids;
|
||||
return ['products' => $products, 'hotels' => $matched['hotel'] ?? [], 'busProHotels' => $matched['busProHotel'] ?? [], 'dateCode' => null];
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -509,8 +649,9 @@ class TripCatalog
|
||||
}
|
||||
}
|
||||
|
||||
// A trip without upcoming dates still describes its hotels.
|
||||
if ([] === $hotels) {
|
||||
// A trip without upcoming dates still describes its hotels. One whose dates are at
|
||||
// a hotel the website does not know has none to describe.
|
||||
if ([] === $hotelCodes) {
|
||||
foreach ((array) ($product->getData()['hotelUids'] ?? []) as $uid) {
|
||||
$hotel = is_int($uid) ? ($catalog['records']['hotel:'.$uid] ?? null) : null;
|
||||
if (null !== $hotel) {
|
||||
@@ -568,7 +709,7 @@ class TripCatalog
|
||||
];
|
||||
|
||||
$dates[$dateId]['hotels'][] = [
|
||||
'hotel' => $hotel?->getTitle() ?? $row['hotel']['code'],
|
||||
'hotel' => $this->hotelName($catalog, $hotel, $row),
|
||||
'hotelCode' => $row['hotel']['code'],
|
||||
'rooms' => $rooms,
|
||||
'bookable' => [] !== $rooms,
|
||||
@@ -644,6 +785,19 @@ class TripCatalog
|
||||
return array_values(array_filter(array_unique(array_map($this->normalize(...), $codes))));
|
||||
}
|
||||
|
||||
/**
|
||||
* The website's name for the hotel of a travel date, else BusPro's, else its code.
|
||||
*
|
||||
* @param array<string, mixed> $catalog
|
||||
* @param array<string, mixed> $row
|
||||
*/
|
||||
private function hotelName(array $catalog, ?ContentDocument $hotel, array $row): string
|
||||
{
|
||||
return $hotel?->getTitle()
|
||||
?? $catalog['busProHotels'][$row['hotel']['busProId'] ?? 0]['name']
|
||||
?? (string) $row['hotel']['code'];
|
||||
}
|
||||
|
||||
private function dataAsOf(Export $export): string
|
||||
{
|
||||
return ($export->sourceGeneratedAt ?? $export->generatedAt)->format('c');
|
||||
|
||||
@@ -18,6 +18,11 @@ class ContentSearchQueryTest extends TestCase
|
||||
);
|
||||
}
|
||||
|
||||
public function testAllTermsCanBeRequired(): void
|
||||
{
|
||||
self::assertSame('+sauna* +whirlpool*', ContentSearchQuery::booleanQuery('Sauna mit Whirlpool', allTerms: true));
|
||||
}
|
||||
|
||||
public function testTermsAreUniqueAndLowerCased(): void
|
||||
{
|
||||
self::assertSame(['gepäck', 'snowboard'], ContentSearchQuery::terms('Gepäck GEPÄCK Snowboard'));
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace App\Tests\Chatbot\Search;
|
||||
|
||||
use App\Chatbot\Search\NameQuery;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
|
||||
class NameQueryTest extends TestCase
|
||||
{
|
||||
public function testTokensAreFoldedUniqueAndWithoutStopwords(): void
|
||||
{
|
||||
self::assertSame(['hotel', 'chamois', 'or', 'val', 'chaviere'], NameQuery::tokens('Das Hotel "Chamois d\'Or" in Val Chavière, Hotel!'));
|
||||
self::assertSame(['chalet', '2100', '3'], NameQuery::tokens('Chalet 2100 (3*)'));
|
||||
self::assertSame(['waldschlossli', 'grosse'], NameQuery::tokens('WALDSCHLÖSSLI Größe'));
|
||||
self::assertSame([], NameQuery::tokens('Was ist das?'));
|
||||
}
|
||||
|
||||
public function testWithinNeedsEveryQueryWordToBeginAWordOfTheName(): void
|
||||
{
|
||||
$name = NameQuery::tokens('3* Hotel Les Arolles (inkl. VP)');
|
||||
|
||||
self::assertTrue(NameQuery::within(NameQuery::tokens('arolles'), $name));
|
||||
self::assertTrue(NameQuery::within(NameQuery::tokens('Hotel Aroll'), $name));
|
||||
self::assertFalse(NameQuery::within(NameQuery::tokens('Sportclub Arolles'), $name));
|
||||
self::assertFalse(NameQuery::within(NameQuery::tokens('rolles'), $name));
|
||||
self::assertFalse(NameQuery::within([], $name));
|
||||
}
|
||||
|
||||
public function testContainsNeedsTheWholeNameInTheQuery(): void
|
||||
{
|
||||
$query = NameQuery::tokens('Hotel Chamois d\'Or in Val Thorens');
|
||||
|
||||
self::assertTrue(NameQuery::contains($query, NameQuery::tokens('Chamois d\'Or')));
|
||||
self::assertTrue(NameQuery::contains($query, NameQuery::tokens('Val Thorens')));
|
||||
self::assertFalse(NameQuery::contains($query, NameQuery::tokens('Chalets Chamois')));
|
||||
self::assertFalse(NameQuery::contains($query, []));
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,8 @@ declare(strict_types=1);
|
||||
|
||||
namespace App\Tests\Chatbot;
|
||||
|
||||
use App\BusProNet\Model\Hotel;
|
||||
use App\BusProNet\XmlLoader\HotelLoader;
|
||||
use App\Chatbot\TripCatalog;
|
||||
use App\Entity\ContentDocument;
|
||||
use App\Enum\ContentFetchState;
|
||||
@@ -35,6 +37,7 @@ class TripCatalogTest extends TestCase
|
||||
'code' => 'VTH',
|
||||
'dateCode' => 'VTH2',
|
||||
'hotel' => 'Hotel Zwei',
|
||||
'hotelCode' => 'HOT2',
|
||||
'accommodation' => null,
|
||||
'destination' => ['country' => 'Frankreich'],
|
||||
'concept' => 'Skiurlaub ohne Programm',
|
||||
@@ -115,6 +118,91 @@ class TripCatalogTest extends TestCase
|
||||
self::assertCount(0, $this->catalog($rows)->search(new TripCriteria(concept: 'party'))['results']);
|
||||
}
|
||||
|
||||
public function testANameInTheQueryReturnsWhatItNamesAndNotWhatMentionsIt(): void
|
||||
{
|
||||
$rows = [
|
||||
$this->row('VTH', 2, '2026-12-05', 'HOT1', [$this->room(500)]),
|
||||
$this->row('VTH', 2, '2026-12-05', 'HOT2', [$this->room(480)]),
|
||||
$this->row('VTH', 3, '2027-01-09', 'HOT1', [$this->room(520)]),
|
||||
];
|
||||
// Every page mentions every word: the texts must not be asked when a name answers.
|
||||
$catalog = $this->catalog($rows, static fn (): array => [['type' => 'product', 'uid' => 1]]);
|
||||
$search = static fn (string $query): array => array_map(
|
||||
static fn (array $result): string => $result['dateCode'].'/'.$result['hotelCode'],
|
||||
$catalog->search(new TripCriteria(query: $query))['results'],
|
||||
);
|
||||
|
||||
self::assertSame(['VTH2/HOT2'], $search('Hotel Zwei'), 'a hotel stands for itself, not for its trip');
|
||||
self::assertSame(['VTH2/HOT2'], $search('zwei'));
|
||||
self::assertSame(['VTH2/HOT2'], $search('Gibt es das Hotel Zwei in Frankreich?'), 'spelled out among other words, the hotel beats the country');
|
||||
self::assertSame(['VTH2/HOT2', 'VTH2/HOT1', 'VTH3/HOT1'], $search('Hotel'));
|
||||
self::assertSame(['VTH2/HOT2', 'VTH2/HOT1', 'VTH3/HOT1'], $search('val thorens'));
|
||||
self::assertSame(['VTH2/HOT2', 'VTH2/HOT1', 'VTH3/HOT1'], $search('Urlaub in Frankreich'));
|
||||
self::assertSame([], $search('Gruppenreise Uni X'), 'a hidden trip is not offered, and nothing else in its place');
|
||||
}
|
||||
|
||||
public function testACodeInTheQueryFindsItsTripHotelOrDate(): void
|
||||
{
|
||||
$catalog = $this->catalog([
|
||||
$this->row('VTH', 2, '2026-12-05', 'HOT1', [$this->room(500)]),
|
||||
$this->row('VTH', 2, '2026-12-05', 'HOT2', [$this->room(480)]),
|
||||
$this->row('VTH', 3, '2027-01-09', 'HOT1', [$this->room(520)]),
|
||||
]);
|
||||
$search = static fn (string $query): array => array_map(
|
||||
static fn (array $result): string => $result['dateCode'].'/'.$result['hotelCode'],
|
||||
$catalog->search(new TripCriteria(query: $query))['results'],
|
||||
);
|
||||
|
||||
self::assertSame(['VTH2/HOT2', 'VTH2/HOT1', 'VTH3/HOT1'], $search('vth'));
|
||||
self::assertSame(['VTH2/HOT1', 'VTH3/HOT1'], $search('HOT1ALIAS'));
|
||||
self::assertSame(['VTH3/HOT1'], $search('vth-3'));
|
||||
}
|
||||
|
||||
public function testHotelsAreAlsoFoundByTheirBusProNameAndShownByItWithoutAWebsiteRecord(): void
|
||||
{
|
||||
$catalog = $this->catalog([
|
||||
$this->row('VTH', 2, '2026-12-05', 'HOT1', [$this->room(500)]),
|
||||
$this->row('VTH', 2, '2026-12-05', 'HOT2', [$this->room(480)]),
|
||||
$this->row('VTH', 2, '2026-12-05', 'HOT3', [$this->room(520)]),
|
||||
]);
|
||||
$search = static fn (string $query): array => array_map(
|
||||
static fn (array $result): string => $result['hotelCode'].'/'.$result['hotel'],
|
||||
$catalog->search(new TripCriteria(query: $query))['results'],
|
||||
);
|
||||
|
||||
self::assertSame(['HOT3/Oceanpeak'], $search('oceanpeak'), 'no website record: found and shown by BusPro\'s name');
|
||||
self::assertSame(['HOT3/Oceanpeak'], $search('hot3'));
|
||||
self::assertSame(['HOT3/Oceanpeak'], $search('Hotel Oceanpeak in Frankreich'));
|
||||
self::assertSame(['HOT1/Hotel Eins'], $search('Residence Eins'), 'BusPro\'s wording finds the hotel, the website\'s names it');
|
||||
self::assertSame('Oceanpeak', $catalog->get('VTH', 'HOT3')['dates'][0]['hotels'][0]['hotel'] ?? null);
|
||||
self::assertSame([], $catalog->get('VTH', 'HOT3')['hotels'] ?? null, 'not the trip\'s other hotels');
|
||||
}
|
||||
|
||||
public function testOtherWordsAreLookedUpInThePageTextsEveryTermBeforeAnyTerm(): void
|
||||
{
|
||||
$rows = [
|
||||
$this->row('VTH', 2, '2026-12-05', 'HOT1', [$this->room(500)]),
|
||||
$this->row('VTH', 2, '2026-12-05', 'HOT2', [$this->room(480)]),
|
||||
];
|
||||
$asked = [];
|
||||
$hits = static function (string $booleanQuery) use (&$asked): array {
|
||||
$asked[] = $booleanQuery;
|
||||
|
||||
return match ($booleanQuery) {
|
||||
'+sauna* +whirlpool*' => [['type' => 'hotel', 'uid' => 11]],
|
||||
'kamin* whirlpool*' => [['type' => 'country', 'uid' => 2]],
|
||||
default => [],
|
||||
};
|
||||
};
|
||||
$search = fn (string $query): array => array_column($this->catalog($rows, $hits)->search(new TripCriteria(query: $query))['results'], 'hotelCode');
|
||||
|
||||
self::assertSame(['HOT2'], $search('Sauna Whirlpool'));
|
||||
self::assertSame(['+sauna* +whirlpool*'], $asked);
|
||||
self::assertSame(['HOT2', 'HOT1'], $search('Kamin Whirlpool'), 'no page has both, so any of them');
|
||||
self::assertSame([], $search('Kamin'));
|
||||
self::assertSame(['HOT2', 'HOT1'], $search('was ist das'), 'nothing searchable restricts nothing');
|
||||
}
|
||||
|
||||
public function testGetJoinsHotelsOnTheirAliasesAndPreferTheOwner(): void
|
||||
{
|
||||
$trip = $this->catalog([
|
||||
@@ -192,9 +280,10 @@ class TripCatalogTest extends TestCase
|
||||
}
|
||||
|
||||
/**
|
||||
* @param list<array<string, mixed>> $rows
|
||||
* @param list<array<string, mixed>> $rows
|
||||
* @param (\Closure(string): list<array{type: string, uid: int}>)|null $textHits the full-text index, by boolean query
|
||||
*/
|
||||
private function catalog(array $rows): TripCatalog
|
||||
private function catalog(array $rows, ?\Closure $textHits = null): TripCatalog
|
||||
{
|
||||
$export = $this->createStub(ProductExportBuilder::class);
|
||||
$export->method('build')->willReturn(new Export($rows, 0, 0, new \DateTimeImmutable('2026-10-08 09:00:00+00:00'), new \DateTimeImmutable('2026-10-08 06:00:00+00:00')));
|
||||
@@ -213,9 +302,21 @@ class TripCatalogTest extends TestCase
|
||||
|
||||
$repository = $this->createStub(ContentDocumentRepository::class);
|
||||
$repository->method('findRecords')->willReturn($records);
|
||||
$repository->method('matchRecords')->willReturn([]);
|
||||
$repository->method('matchRecords')->willReturnCallback($textHits ?? static fn (): array => []);
|
||||
|
||||
return new TripCatalog($export, $repository, new MockClock('2026-10-08 10:00:00'), new NullLogger());
|
||||
$hotels = [];
|
||||
foreach (['HOT1' => "Agence X\nRésidence Eins Wochenreisen", 'HOT3' => 'Oceanpeak'] as $code => $name) {
|
||||
$hotel = new Hotel();
|
||||
$hotel->id = crc32($code);
|
||||
$hotel->code = $code;
|
||||
$hotel->name = $name;
|
||||
$hotels[$hotel->id] = $hotel;
|
||||
}
|
||||
|
||||
$hotelLoader = $this->createStub(HotelLoader::class);
|
||||
$hotelLoader->method('loadAll')->willReturn($hotels);
|
||||
|
||||
return new TripCatalog($export, $repository, $hotelLoader, new MockClock('2026-10-08 10:00:00'), new NullLogger());
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
Reference in new issue
Block a user