The Magic of Distance Matrices

Описание: In this video, I discuss distance matrices, which summarize all of the geometry needed to describe a point cloud up to a rigid motion. I then talk about how we can use them to find repetitions and sections in videos and music, among other applications.

Notes here:
https://ctralie.substack.com/p/the-ma...

Table of Contents:
00:00 Intro
02:45 Definition of point clouds
07:43 Definition of distance matrices
16:20 Diagonals, blocks, and warping
21:06 Efficient computation
29:06 Video applications
37:57 Theo says hi
43:46 Digital music applications
50:19 Other applications

NOTE: (at 33:00) The technique I came up with for quickly computing the distance matrices actually works fine on video at the original resolution. What I forgot to do is convert each frame to floating point first!

References:
[1] Cutler, Ross, and Larry S. Davis. "Robust real-time periodic motion detection, analysis, and applications." IEEE Transactions on pattern analysis and machine intelligence 22.8 (2002): 781-796.
https://web.archive.org/web/200106060...

[2] Junejo, Imran N., et al. "View-independent action recognition from temporal self-similarities." IEEE transactions on pattern analysis and machine intelligence 33.1 (2010): 172-185.
https://inria.hal.science/hal-0106469...

[3] Huang, Peng, Adrian Hilton, and Jonathan Starck. "Shape similarity for 3D video sequences of people." International Journal of Computer Vision 89.2 (2010): 362-381.
https://www2.cs.sfu.ca/CourseCentral/...

[4] Cooper, Matthew, and Jonathan Foote. "Automatic Music Summarization via Similarity Analysis." International Society for Music Information Retrieval (ISMIR). 2002.
https://archives.ismir.net/ismir2002/...

[5] Evans, Zach, et al. "Long-form music generation with latent diffusion." arXiv preprint arXiv:2404.10301 (2024).
https://arxiv.org/pdf/2404.10301?

[6] Tralie, Christopher J., and Paul Bendich. "Cover Song Identification with Timbral Shape Sequences." International Society for Music Information Retrieval (ISMIR). 2015.
https://archives.ismir.net/ismir2015/...

[7] Tralie, Christopher J. "Geometric multimedia time series." Dissertation. Duke University, 2017.
https://www.ctralie.com/Research/ctra...

[8] Tralie, Christopher J., Matthew, Goodwin S., and Sapiro, Guillermo. "Automated detection of
stereotypical motor movements in children with autism spectrum disorder using geometric feature
fusion." International Society for Autism Research (INSAR), Rotterdam, The Netherlands, 2018.
https://www.ctralie.com/Research/Auti...

[9] Examples of classical 3D shape descriptors (i.e. ways of turning 3D shapes into Euclidean points)
https://github.com/ctralie/Topologica...

[10] http://www.recurrence-plot.tk/