Filter context, caching and the slow query paths

Key points

• Query context scores every match; filter context only answers yes or no. Filters skip scoring entirely and are cached, so anything that is not a relevance signal belongs in filter.
• Inside bool, must and should score, while filter and must_not do not. Moving a date range from must to filter is often the single biggest win available.
• The node query cache stores filter results as bitsets, but only for filters reused across queries. A filter containing now changes on every request and can never be cached - round it to now/h to make it cacheable.
• Deep pagination is the classic cluster killer: from: 10000 makes every shard collect 10,000 hits and the coordinator sort them all. Use search_after, or a point-in-time for stable paging.
• Wildcards with a leading *, regex queries and script queries cannot use the inverted index and degrade toward a scan. An ngram or prefix field at index time is far cheaper than a leading wildcard at query time.
• Use the profile API rather than guessing. It breaks a query into per-shard phases and shows which clause actually spent the time.

Example

// SLOW: everything scored, nothing cacheable.
{ "query": { "bool": { "must": [
    { "match":  { "title": "laptop" } },
    { "term":   { "status": "ACTIVE" } },
    { "range":  { "created_at": { "gte": "now-7d" } } }
] } } }

// FAST: only the relevance clause scores; the rest are cached filters.
{ "query": { "bool": {
    "must":   [ { "match": { "title": "laptop" } } ],
    "filter": [
      { "term":  { "status": "ACTIVE" } },
      { "range": { "created_at": { "gte": "now-7d/h" } } }
    ]
} } }
//                                              ^^^^^^^^
//   "now-7d"    changes every millisecond -> NEVER cached
//   "now-7d/h"  rounds to the hour        -> cached for an hour

---

// DEEP PAGINATION - this is how clusters fall over.
{ "from": 10000, "size": 10 }
//   every shard collects 10,010 hits, the coordinator sorts them all,
//   and discards 10,000. Cost grows linearly with the offset.
//   index.max_result_window caps it at 10,000 for a reason.

// CORRECT - search_after, using the last sort values.
{ "size": 10,
  "sort": [ { "created_at": "desc" }, { "_id": "asc" } ],
  "search_after": [ "2026-08-03T10:00:00Z", "doc-12345" ] }
//   constant cost per page, whatever the offset

---

// Which clause is slow? Do not guess.
{ "profile": true, "query": { ... } }
//   -> per-shard breakdown, per query clause, in nanoseconds

/*
 * QUERY COST, roughly ascending:
 *
 *   term / terms filter (cached)   cheapest
 *   match on an analysed field     cheap
 *   range on a date or number      cheap, and cacheable if rounded
 *   prefix "lap*"                  moderate - uses the index
 *   wildcard "*top"                EXPENSIVE - leading wildcard, no index
 *   regexp                         expensive
 *   script query                   per document, no index at all
 *
 * Anything in the bottom three should usually be solved at INDEX time
 * with an ngram, prefix or dedicated field.
 */

Put non-scoring clauses in filter and round date maths to make them cacheable - and never paginate with a large from.