Abstract
Today, people can easily record memorable moments, ranging from concerts, sports events, lectures, family gatherings, and birthday parties with multiple consumer cameras. However, synchronizing these cross-camera streams remains challenging. Existing methods assume controlled settings, specific targets, manual correction, or costly hardware. We present VisualSync, an optimization framework based on multi-view dynamics that aligns unposed, unsynchronized videos at millisecond accuracy. Our key insight is that any moving 3D point, when co-visible in two cameras, obeys epipolar constraints once properly synchronized. To exploit this, VisualSync leverages off-the-shelf 3D reconstruction, feature matching, and dense tracking to extract tracklets, relative poses, and cross-view correspondences. It then jointly minimizes the epipolar error to estimate each camera's time offset. Experiments on four diverse, challenging datasets show that VisualSync outperforms baseline methods, achieving an median synchronization error below 50 ms.
Insight
When cameras are time-aligned, keypoint tracks align with epipolar lines; misalignment causes deviations. Minimizing these deviations across tracklets recovers the correct time offset.
Green dots show a pair of correspondences in two videos. The red line shows the epipolar line of two cameras. The deviation of the green dots from the red line tells the synchronization error.
Unsynchronized video pair (green dot not on the red line)
Synchronized video pair (green dot on the red line)
Method
We extract cross-video temporal correspondences and estimate camera poses to compute epipolar lines. We estimate pairwise video offsets by minimizing epipolar violations over matched correspondences. Later we perform global optimization to align all videos.
