Researchers release tool to label animal interactions in video — first software that tracks both individuals and their behavior together

A new open-source software called SiLVi lets researchers annotate videos of animals by marking both who is in the frame and what they're doing together — a capability that didn't exist before because existing tools did one or the other, but not both. This matters because animal behavior research relies on understanding not just individual actions but interactions between animals, and manual labeling of interactions is the bottleneck preventing automated analysis at scale.

Wildlife researchers have spent years collecting massive video libraries from camera traps and drones, but most of that footage goes unanalyzed because labeling animal interactions by hand is expensive and slow. SiLVi removes that bottleneck by letting researchers mark interactions directly in video and generate the structured data that computer vision models need to learn from. The software bridges two previously separate tasks — individual tracking and behavioral annotation — which means researchers can now train models on the kinds of behaviors that actually matter: mating, aggression, cooperation, predation. This unlocks a category of behavioral analysis that has been stuck in the field notes.