Vision

Video Search

The Vision API provides powerful video search capabilities through the /v1/search endpoint, which allows you to search through your uploaded videos using semantic queries.

Search Videos

Search through your videos using natural language queries:

Bash

$curl -X POST https://vision-agent.api.reka.ai/v1/search \
>  -H "X-Api-Key: YOUR_API_KEY" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "query": "person walking on beach",
>    "max_results": 10,
>    "threshold": 0.2
>  }'

Python

1import requests
2import json
3
4url = f"{BASE_URL}/v1/search"
5
6
7# Search request body
8payload = {
9    "query": "person walking on beach",
10    "max_results": 10,
11    "threshold": 0.2
12}
13
14headers = {
15    "X-Api-Key": REKA_API_KEY,
16    "Content-Type": "application/json",
17}
18
19response = requests.post(url, json=payload, headers=headers)
20print(response.status_code, response.json())

Search Parameters

General parameters:

  • query (required): Natural language search query
  • max_results (optional): Maximum number of results to return (default: 10, max: 200)
  • threshold (optional): Confidence threshold level from 0 (lowest) to 1 (highest). Different queries may need different thresholds based on their specificity and the nature of your search.
  • video_ids (optional): List of specific video IDs to search within. If not provided, the search runs across all of your videos (or the groups you include).
  • group_ids (optional): List of group IDs to search within. Omit the field—or pass ["default"]—to search the default group only. If you specify video_ids, that filter takes precedence and group_ids is ignored.
  • generate_report (optional): Whether to generate a report and explanations for the search results (default: false)

Time filtering parameters:

  • datetime_from (optional): Filter results from datetime (ISO 8601 format, e.g., 2024-01-15T10:00:00Z)
  • datetime_to (optional): Filter results to datetime (ISO 8601 format, e.g., 2024-01-15T18:00:00Z)
  • timestamp_from (optional): Filter results from video timestamp in seconds (relative to video start)
  • timestamp_to (optional): Filter results to video timestamp in seconds (relative to video start)

Search Response

The search endpoint returns relevant video chunks with confidence scores relative to your search query:

1{
2  "results": [
3    {
4      "video_chunk_id": "123e4567-e89b-12d3-a456-426614174000",
5      "video_id": "550e8400-e29b-41d4-a716-446655440000",
6      "user_id": "user-123",
7      "score": 0.95,
8      "start_timestamp": 10.5,
9      "end_timestamp": 25.3,
10      "plain_text_caption": "Person walking on the beach",
11      "plain_text_transcript": "The waves are crashing on the shore",
12      "explanation": "This video chunk shows a person walking along the shoreline with waves in the background, matching the search query for 'person walking on beach'",
13      "s3_presigned_url": "https://s3.amazonaws.com/bucket/video.mp4?presigned=..."
14    }
15  ],
16  "report": "Found 1 result matching 'person walking on beach'. The result shows a clear match with high relevance (0.95 score)."
17}

Note: Confidence scores (0.0 to 1.0) are relative to your specific search query. A score of 0.95 for “person walking on beach” indicates high relevance to that particular query, not an absolute measure of query matching. Thresholds may vary significantly for each query and will require experimentation - one query might have a highest score of 0.8 while another query has 0.2 as the highest score, depending on the specificity and content of your videos.

Response Fields

Result Item Fields

  • video_chunk_id: Unique identifier for the specific video chunk
  • video_id: ID of the parent video
  • user_id: User ID who owns the video
  • score: Confidence score (0.0 to 1.0) relative to your search query
  • start_timestamp: Start time of the chunk in seconds
  • end_timestamp: End time of the chunk in seconds
  • plain_text_caption (optional): Text caption for the video chunk
  • plain_text_transcript (optional): Transcription of speech in the video chunk
  • explanation (optional): Explanation of the video chunk content and its relevance to the search query
  • s3_presigned_url (optional): Direct access URL for the video chunk
  • debug_info (optional): Debug information for internal use

Top-Level Response Fields

  • results: Array of video chunk results matching your search query
  • report (optional): Summary report of the search results (only included when generate_report is set to true)

Search Examples

Basic Content Search

$# Search for videos containing people
>curl -X POST https://vision-agent.api.reka.ai/v1/search \
>  -H "X-Api-Key: YOUR_API_KEY" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "query": "people walking",
>  }'

Search with threshold and max number of results

$# Find only highly relevant videos
>curl -X POST https://vision-agent.api.reka.ai/v1/search \
>  -H "X-Api-Key: YOUR_API_KEY" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "query": "outdoor activities",
>    "max_results": 10,
>    "threshold": 0.3
>  }'

Search with no threshold

$# Get more results with lower relevance requirements
>curl -X POST https://vision-agent.api.reka.ai/v1/search \
>  -H "X-Api-Key: YOUR_API_KEY" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "query": "children playing",
>    "max_results": 10
>  }'

Python Search Example

1import requests
2
3url = f"{BASE_URL}/v1/search"
4
5# Search request body
6payload = {
7    "query": "a man", # natural language search query
8    "max_results": 10, # maximum number of results to display
9
10    # Optional list of video ids to search within, if None, will search all videos
11    # uploaded by user
12    "video_ids": ["3f8e2bde-9a90-4ac9-904a-5cd44f3c55e3", "12ad4a7b-3f97-4dd4-bd6b-e00d9d2a3d79", "9d06b838-2b9d-4fd7-896f-1ef1f34f8a92"]
13    # Optional threshold for the search query
14    "threshold": 0.3
15}
16
17headers = {
18    "X-Api-Key": REKA_API_KEY,
19    "Content-Type": "application/json",
20}
21
22response = requests.post(url, json=payload, headers=headers)
23print(response.status_code, response.text)

Best Practices

  1. Use descriptive queries for better search results
  2. Set appropriate thresholds based on your query type and needs:
    • Specific queries (e.g., “red car driving on highway”): Use higher thresholds (0.7-0.9) for precise matches
    • General queries (e.g., “people”): Use lower thresholds (0.2-0.5) to capture more results
    • Exploratory searches: Use very low thresholds (0.1-0.3) to discover content
    • Production applications: Start with moderate thresholds (0.4-0.6) and adjust based on results
  3. Understand confidence scores are relative to your query, not absolute video quality
  4. Set reasonable limits to manage response size
  5. Experiment with different thresholds for the same query to find optimal results
  6. Consider query complexity: Simple queries may need lower thresholds, complex queries may need higher thresholds
  7. Remember threshold variability: Each query may have different score ranges - one query’s highest score might be 0.8 while another’s is 0.2, requiring different threshold strategies

Search Tips

  • Be specific: “person walking on beach” is better than “walking”
  • Use natural language: The search understands conversational queries
  • Adjust threshold based on query type:
    • Specific objects/actions: Higher thresholds (0.7+)
    • General concepts: Lower thresholds (0.3-0.6)
    • Broad categories: Very low thresholds (0.1-0.3)
  • Consider your use case: Use lower thresholds for exploratory searches, higher for precise filtering
  • Monitor confidence scores: Higher scores indicate better matches for your specific query
  • Remember relativity: A 0.9 score for one query might be different than 0.9 for another query
  • Test different thresholds: The same query may work better with different thresholds depending on your video content