top of page
Voltaire Staff

Meta's latest AI video tool can track objects across frames



Meta has launched an upgraded video editing-friendly AI model, Segment Anything Model (SAM 2), built to handle intricate computer vision tasks and advanced video editing.

 

"SAM 2 can segment any object and consistently follow it across all frames of a video in real-time, unlocking new possibilities for video editing and new experiences in mixed reality," Meta said in a blog post.

 

The new model builds on a transformer architecture and includes a streaming memory component, letting it process video segments efficiently.

 

SAM 2 is trained on its extensive video segmentation dataset, known as the SA-V dataset, ensuring strong performance in various applications.

 

The company said that the initial version of SAM had diverse applications, such as helping marine scientists in segmenting sonar images to study coral reefs, supporting disaster relief through satellite imagery analysis, and assisting in medical fields by segmenting cellular images for skin cancer detection.