Help

Overview Admin Chat UI Design Curated Answers Search Settings Conversational Intelligence Data Sync Upload Documents Human Handoff Admin Console Authorisation Contact Support

Site Search Index Fields

Airgentic Help

Indexed fields are the values stored for each page or document in the search index.

Fields are used for result display, filtering, sorting, AI answers, and optional plain-text query matching.


Standard Fields

Most services use these standard fields:

Field Purpose
title Result title and search ranking.
description Result summary.
image Result thumbnail.
date Publication or update date.
URL Result link.
body Main searchable page text.
page_type Search result category.

For existing web crawl services, title, description, image, and date are usually configured through the crawler's Standard Result Fields tab. Additional indexed fields are managed through Search Configuration > Fields.


Standard Result Field Mappings

Open the web crawl configuration screen and go to Standard Result Fields.

Standard result field mappings define which metadata fields should be checked for each standard result field, such as title, description, image, and date.

For example, a title field might check:

title_element
og:title
title
dc.title

The first non-empty value is used.

See Crawler Standard Result Fields.

In newer services, standard fields can also be managed through Search Fields. Existing services do not switch automatically. If Use Search Fields for standard result fields is not enabled, the standard title, description, image, and date rows shown in Fields are informational/read-only and the web crawl Standard Result Fields settings still take precedence.

If Use Search Fields for standard result fields is enabled, the Search Fields definitions become the source for those standard fields and the web crawl mappings no longer control them.


Additional Search Fields

Advanced Site Search can use additional Search Fields, such as:

  • host
  • section
  • format
  • delivery
  • subject_area

These fields can be used for filters, result-card badges, search scopes, sorting, plain-text query matching, or search context controls.

Open Search Configuration > Fields to review or configure Search Fields. This screen writes to the service search descriptor and is the long-term place for fields used by filters and result cards.

Each Search Field can define:

  • a field name, such as delivery
  • a display label, such as Delivery
  • a field type, usually Keyword / filter value for simple filters
  • where the value comes from
  • whether the field should be shown as a search filter
  • for text fields, whether typed search queries should match against that field

The field editor includes guidance beside the Field type selector. Use Keyword / filter value for most exact filters, such as delivery, credential, format, or subject area. Use Hierarchy / drill-down path when the value represents levels, such as Programs > Engineering > Civil.


Field Sources

Choose the Source that matches where the value should come from.

Source Use when Example
HTML metadata The website already exposes the value in metadata tags. Check bcit.delivery, programDelivery, then et:delivery in order.
XPath The value is in page HTML but not in reliable metadata tags. //span[@class='faculty']/text()
JSON-LD The value is in structured data on the page. BlogPostingdatePublished, or a JSONPath for advanced cases.
Computed from URL The value should be derived from the page URL or URL patterns. Host, URL section, URL depth, or Mapping rules.

The first non-empty value wins for HTML metadata, XPath, and JSON-LD priority lists.

HTML Metadata

This is the default source for most fields. List the metadata field names to check in priority order. Airgentic uses the first non-empty value it finds.

Example for a Delivery field:

bcit.delivery
programDelivery
et:delivery

XPath

Use XPath when the website stores the value in page HTML rather than metadata or JSON-LD.

Add one or more XPath expressions in priority order. Airgentic evaluates them during HTML Processing and stores the first matching value.

Example:

//span[@class='training-level']/text()

XPath fields require HTML Processing before the value appears in search. When you save XPath changes, the admin console explains whether HTML Processing is needed and can start it for you.

JSON-LD

Use JSON-LD when the page includes structured data that defines the field value.

For each rule you can use:

  • Simple mode: optional @type plus property name, such as BlogPosting and datePublished
  • Advanced mode: a JSONPath expression for more complex structured data

Example simple rule:

Type Property
BlogPosting datePublished

JSON-LD fields also require HTML Processing before values appear in search.

URL Mapping Rules

Use Mapping rules when a field value should be assigned from URL patterns rather than page content.

Choose Computed from URL, then set Computed value to Mapping rules. Add rules in order; the first match wins. Pages with no matching rule are omitted from that field.

Example:

Field value if URL contains
Certificate IV /qualifications/certificate-iv
Diploma /qualifications/diploma
Mini MBA /courses/mini-mba

Mapping rules usually require an index update so the assigned values are stored in the search index. They do not require HTML Processing unless the field also uses XPath or JSON-LD.

Other Computed URL Values

Other Computed from URL options generate values from the page URL itself, such as:

Computed value Example
Host www.example.edu
URL section Programs > Engineering
URL depth 3
URL extension pdf
URL path /programs/engineering

Computed URL values are useful for current-site search and hierarchical filters. They usually require an index update rather than HTML Processing.


Advanced Field Settings

The Advanced settings section is grouped by purpose:

  • Ranking controls whether a text field is included in normal typed search queries. Leave Plain-text search boost at 0 for fields that should only be used as filters, badges, sorting fields, or layout data.
  • Multi-Value Handling is for metadata values that contain more than one value. For example, if a page has Online, Burnaby, enable Multi-valued and split on ,.
  • Hierarchy Settings appears when Field type is Hierarchy / drill-down path. Use the delimiter to split levels, such as > in Programs > Engineering > Civil. Max hierarchy depth controls how many levels are indexed.
  • Field Value Ordering controls how values are sorted in the filter UI. The Sort by dropdown uses friendly options such as Most common first, then A-Z, Alphabetical A-Z, Alphabetical Z-A, and Least common first, then A-Z. The common default is Most common first, then A-Z.
  • Search Result Sorting uses eligible indexed fields from Search Configuration > Scopes. Date, number, keyword/filter, and URL/path fields can usually be offered as sort options. Text and hierarchy fields are not usually sortable.
  • Regex Cleanup can remove or rewrite patterns after a metadata value is extracted. For example, a regex can keep only the text before a pipe suffix.

The internal Role field is only shown for standard result fields such as title, description, image, and date. The stored Elasticsearch field preview appears after a field has been saved.

Some advanced settings may initially be visible to superusers or managed by Airgentic support.


Plain-Text Search Boosts

The Plain-text search boost setting applies to descriptor-backed Site Search pages and only to fields with type Text / searchable content.

Use it when a custom text field should be searched as part of a visitor's normal typed query. For example, a program_name or course_summary field might have a positive boost so searches match against that text as well as the standard page title and body.

The value range is 0 to 5:

  • 0 means the field is not included in normal typed query matching.
  • A positive value includes the field and boosts matches in that field.
  • Higher values make matches in that field count more strongly.

Filters and result-card badges do not automatically make a field searchable. A keyword field such as delivery, credential, or campus can still be used for filters and badges, but it is not searched by normal typed queries unless it is modelled as a text field with a positive plain-text search boost.

The standard title and body fields are always searched as the baseline.


Multi-Value and Hierarchy Fields

Multi-value and hierarchy settings solve different problems:

Setting Use when Example
Multi-valued One page can have more than one value for the same field. Online, Burnaby becomes Online and Burnaby.
Hierarchy / drill-down path One value represents levels in a tree. Programs > Engineering > Civil.

A hierarchy field can also be multi-valued if a page can belong to multiple hierarchy paths, but the hierarchy delimiter and maximum depth are controlled by the Field type, not by the Multi-valued checkbox.


Filter Display

When Show this field as a search filter is enabled, the editor shows filter controls and a small preview of the selected filter style.

Common choices:

  • List of checkboxes/buttons for most filters.
  • Single-select menu when only one value should be selected at a time.
  • Hierarchy / drill-down for hierarchy fields.
  • Range for numeric or date-like values.
  • On/off toggle for yes/no filters.

Use Match any selected value for most filters. Use Must match all selected values only when documents can contain multiple values and visitors need results that contain every selected value.

To decide which filters appear in each search tab, configure scopes on Search Configuration > Scopes. See Configuring Search Scopes.

To show a field directly on search results, add it as a metadata badge or custom layout block on Search Configuration > Result Cards. See Configuring Result Cards and Layouts.


Reprocessing

Adding or changing indexed fields may require background processing before values appear in search.

When you save changes on Search Configuration > Fields, the admin console previews the impact and may show a confirmation dialog explaining what is needed.

Change Typical requirement
Change result display only No reprocess.
Add or change HTML metadata field Re-extract already crawled pages, or run an index update.
Add or change XPath field HTML Processing (filter_html), which also runs indexing when it completes.
Add or change JSON-LD field HTML Processing (filter_html), which also runs indexing when it completes.
Add or change URL mapping rules Index update.
Add or change other computed URL field Index update.
Change a field type Full reindex.

The save dialog explains the planned job in plain language, such as Run HTML Processing and indexing now or Initiate index update now. Leave the checkbox enabled when you want changes to appear in search as soon as possible. Uncheck it if you want to save now and run processing manually later.

Display-only changes, such as filter labels, result-card badges, or scope visibility, take effect after save without reprocessing.

You have unsaved changes