Vision

Video Q&A

The Vision API provides powerful question-answering capabilities for your videos. Once a video has been indexed, you can ask natural language questions about its content and receive AI-powered answers.

Prerequisites

Before using Video Q&A, ensure your video has been successfully indexed:

  1. Upload a video with index=true
  2. Check indexing status - it should be "indexed"
  3. Wait for processing if status is "indexing"

Ask Questions

Use the /qa/chat endpoint to ask questions about your videos:

$curl -X POST https://vision-agent.api.reka.ai/qa/chat \
>  -H "X-Api-Key: YOUR_API_KEY" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "video_id": "550e8400-e29b-41d4-a716-446655440000",
>    "messages": [
>      {
>        "role": "user",
>        "content": "What is happening in this video?"
>      }
>    ]
>  }'

Request Parameters

  • video_id (optional): ID of the video to analyze
  • video_url (optional): URL of the video to analyze (either video_id or video_url is required)
  • messages (optional): List of chat messages

Using Chat Messages

For multi-turn conversations, use the messages parameter:

$curl -X POST https://vision-agent.api.reka.ai/qa/chat \
>  -H "X-Api-Key: YOUR_API_KEY" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "video_id": "550e8400-e29b-41d4-a716-446655440000",
>    "messages": [
>      {
>        "role": "user",
>        "content": "What is happening in this video?"
>      }
>    ]
>  }'

Using Video URL

You can also provide a video URL instead of a video ID:

$curl -X POST https://vision-agent.api.reka.ai/qa/chat \
>  -H "X-Api-Key: YOUR_API_KEY" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "video_url": "https://example.com/video.mp4",
>    "messages": [
>      {
>        "role": "user",
>        "content": "Describe the main events in this video"
>      }
>    ]
>  }'

Streaming Responses

For real-time responses, use the /qa/stream endpoint:

$curl -X POST https://vision-agent.api.reka.ai/qa/stream \
>  -H "X-Api-Key: YOUR_API_KEY" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "video_id": "550e8400-e29b-41d4-a716-446655440000",
>    "messages": [
>      {
>        "role": "user",
>        "content": "What is happening in this video?"
>      }
>    ]
>  }'

This returns a Server-Sent Events (SSE) stream with real-time updates.

Response Format

Chat Response

1{
2  "answer": "The video shows a person walking on the beach during sunset.",
3  "confidence": 0.85,
4  "video_id": "550e8400-e29b-41d4-a716-446655440000",
5  "question": "What is happening in this video?",
6  "timestamp": 1640995200
7}

Stream Response

1{
2  "event": "qa_stream",
3  "data": {
4    "status": "processing",
5    "answer": "The video shows a person walking...",
6    "confidence": 0.85,
7    "video_id": "550e8400-e29b-41d4-a716-446655440000",
8    "question": "What is happening in this video?"
9  }
10}

Question Examples

Here are some example questions you can ask:

  • General: “What is happening in this video?”
  • Specific: “What color is the car in the video?”
  • Temporal: “What happens at the beginning of the video?”
  • Analytical: “How many people are in the scene?”
  • Descriptive: “Describe the setting and atmosphere”

Best Practices

  1. Be specific in your questions for better answers
  2. Check indexing status before asking questions
  3. Use streaming for long videos or complex questions
  4. Provide either video_id or video_url, not both

Error Handling

  • Video not found: Ensure the video_id is correct
  • Video not indexed: Wait for indexing to complete
  • Indexing failed: Re-upload the video with index=true
  • Invalid request: Ensure only one of video_id or video_url is provided