Exporting Data to DataVerse

The VisionAI Observ can export data to DataVerse - Data Center, allowing you to filter and sample images there for AI model training, enabling the continuous evolution of your AI models.

Quick Start

Export data to DataVerse, a powerful tool for data filtering and sampling, enabling AI models to learn continuously. Click on the video below for a quick overview:

View Your Exported Data

Enter "Detection," switch to "Dataset," and view the status of the data exported to DataVerse. Clicking "View" allows you to review the data in Dataverse.

• Dataset / Description: Name and description of the dataset.

• Model: The AI model related to the exported data.

• Status: The current status.

• View: When the data export is successful, you can click View in Dataverse to check the data condition, deciding how to proceed with labeling and AI model training.

When exporting large batches of data, processing and data transfer time is required.

Filtering and Data Export to DataVerse

Click "Export to DataVerse" in the top right corner to import event images and pre-collected images into Dataverse. You will first need to log into your DataVerse account.

How to visit DataVerse? Start using it here 👉 https://linkervision.gitbook.io/dataverse/

You can choose to import data into an existing DataVerse project or create a new one.

• Dataset Name: The name of the dataset.

• Description: A description of the dataset.

• Model Repository: Select the AI model repository related to the exported data. The system will specifically produce data for tasks detected by this AI model as an export base.

• Model Version: Select the AI model version related to the exported data. The system will specifically produce data for tasks detected by this AI model as an export base.

• Task: Filter the tasks to be exported. Select from several cameras that require training.

• Event & Status: Filter the events and their status for the data to be exported. You can select information such as false alerts.

• Time Range: Filter the time for the data to be exported. Select specific times for particular situations.

• Generate IQA: Whether to generate Image Quality information in DataVerse. Refer to details + DataVerse Generate IQA

Click "Create" to establish the dataset and export data to DataVerse.

Tips 💡

We strongly recommend formulating a data filtering strategy before exporting to Dataverse.

Excessive similar data consume labeling resources and offer limited aid in AI model training.

When needing to retrain AI models, we can first understand the current detection status of the AI model, label events (correct or false alerts), and filter data for particularly inaccurate scenarios or cameras before importing into DataVerse.

This prevents massive exports and facing an overwhelming amount of data in DataVerse, making it difficult to find suitable training material.

DataVerse Visualization: Viewing the Exported Dataset

DataVerse is designed with a data-centric concept, making it easy to find and filter samples, select appropriate data for labeling and AI training, and then return the well-trained AI model to Observ for continuous evolution.

In DataVerse's Data Visualization, you can see the exported data and also:

• Class: Filter related categories through Class. You can specifically select objects that are harder to detect.

• IQA: Through image analysis, filter out images that are too dark, overexposed, or blurry, or use this method to find day or night images. Refer to details + DataVerse Generate IQA
• Annotation: Filter by Annotation to compare the results produced by different AI Models.

• Tag: Narrow down the scope with information carried by Tags. After exporting data from Observ, the attributes of the original tasks are still included in Tags for easy querying. This includes the scene, task, camera, event, event status, image type (event or collected images), and more.

• Finally, you can use DataVerse's Data Slice sampling mechanism to select appropriate data for labeling and training. Refer to details + DataVerse Data Sampling

Practical Consideration 🧐

In practical use of Observ, you'll find that the images from fixed cameras are very similar.

Through the Image Collection settings, you can set an AI Confidence Score range or extend the interval for capturing images to select more unique photos + Collecting AI Data.

Or, make a good data filtering decision before exporting to DataVerse, pre-filtering once. Additionally, you can collect special photos through some custom events.

Once you have imported data into DataVerse, you can also take IQA filtering, select data by Class, or pick data by Tag, and use the sampling mechanism (e.g., average sampling by camera) to create your own dataset for model retraining.