Skip to main content
By default, a search runs against every available signal at once and merges the results — you don’t have to decide up front whether the moment you want is visual or spoken.

​
The modes parameter

This is the default. Pass a subset to restrict which signals are searched:
Text visible in frame — signage, slide text, captions — is indexed separately as ocr segments and can also surface in results regardless of modes.

​
How results merge

Visual and transcript search run concurrently, not sequentially, so combining modes doesn’t cost you latency — total time is bounded by the slower of the two, not their sum (see Performance). Their ranked lists are merged with Reciprocal Rank Fusion (k=60): a segment can rank highly because it matched visually, because of what was said, or both.

​
Filtering by scene label

Every video is automatically segmented at shot boundaries, and each shot gets zero or more auto-detected labels (e.g. outdoor, whiteboard, person). Pass labels to restrict results to segments carrying specific labels:
The response’s meta.facets.scene_labels field returns the count of every label present across the matched videos, so you can build filter UI without a separate request.

​
Precedence when combining image and text

If you send both an image and a text query together, the request is no longer a pure “combined query” across modes — it’s image-first with text re-scoring. See Image Search for that blend.