VQA Prediction & Evaluation

Setting Up a VQA Project

On the Project Detail page, you can set up VQA (Visual Question Answering) questions for a project, defining the questions and answer formats the system will apply to each image.

1. Go to Home > Project and select your project.

2. On the Project Detail page, find the VQA setup section.

3. Click Edit to add or modify questions, including the question text and answer type.

4. Use Edit Ontology Alias to adjust question name mappings if needed.

Running VQA Predictions

Free Text Prompts and Raw Output (Single-Image Testing)

VQA Prediction now supports Free Text Prompts, for both single-image testing and batch runs across a Data Slice. You're no longer limited to the predefined question structure — this makes it easy to test different prompt formats, evaluate LLM compatibility, and compare prompt accuracy using quantitative metrics.

Single-Image Prediction

1. Go to Data Visualization → Review Image → Single Image Prediction.

2. Switch Prompt Type to Free Text and enter your prompt in the text box.

3. Adjust Advanced Settings (Temperature, Max Tokens, Top P) as needed.

4. Click Run Prediction to send the prompt to the VLM.

5. View the results under Structured Result (mapped to your ontology) or Raw Data (the VLM's raw response).

Batch Prediction (Data Slice)

1. Go to the Data Slice you want to test and select Prediction.

2. Switch Prompt Type to Free Text and enter your prompt.

3. Adjust Advanced Settings as needed.

4. Click Start Prediction and track progress on the Prediction Status page.

5. Once complete, view results in Data Visualization. The VLM's answers are matched to the ontology questions in order; any unmatched or missing answers are marked NA.

Using Prediction Results for Annotation

You can quickly create an annotation task by naming it, selecting a data slice, and applying pre-annotation results.

What You Can Do

• Name the task for easy identification and management

• Select the Data Slices to annotate from the associated project

• Apply Pre-Annotation results to speed up the labeling process

• Data without pre-annotation is still added to the task, marked as having no pre-annotation

• Expand Advanced for additional customization options

Metrics Report: New Visualization Tools and Detail Report

The VQA Metrics Report has been redesigned with new visualization tools, letting you filter prediction results by question and answer status to quickly find prompts that need adjustment.

Creating a Metrics Report

1. Select a Data Slice and choose Run Metrics.

2. Select the prediction you want to report on, then run it to generate the report.

Once the report finishes running on the Data Slice page, click Go to Report to open it.

Reading the Metrics Report

1. Click the Data tab to see every image included in the report.

2. Use the Selector panel to filter images by question and answer status (Correct / Incorrect / NA).

3. Click any image to see Ground Truth and Prediction side by side under the Result tab, or switch to Raw Output to see the VLM's full response.

4. Use Previous / Next to browse the filtered images, or click View in Data Visualization to see the image in its original context.