Analyzers, tokenizers and token filters

Key points

• An analyzer is three stages: zero or more character filters, exactly one tokenizer, then token filters.
• The default standard analyzer splits on Unicode word boundaries and lowercases, with no stopwords.
• Run POST _analyze to see the exact tokens a chain produces before you index anything.
• The same analyzer runs at index time and query time, or the two sets of terms never line up.
• search_analyzer overrides the query side deliberately - index with edge ngrams, query plain.
• A keyword field skips analysis completely, so the whole value becomes one single term.

Example

POST /_analyze
{
  "tokenizer": "standard",
  "filter": ["lowercase", "porter_stem"],
  "text": "The Cafes were RUNNING"
}

# tokens: [the, cafe, were, run]
#   standard tokenizer -> The, Cafes, were, RUNNING
#   lowercase          -> the, cafes, were, running
#   porter_stem        -> cafes becomes cafe, running becomes run

If a search returns nothing, run _analyze on both the field value and the query text first.