Maximizing the norm of the output of a complex multilinear map twice and obtaining the same result

Описание: In this visualization, we maximize the norm of the output of a multilinear map twice, and we obtain the exact same results.

For this visualization, we assume all vector spaces are over the field of complex numbers.

Consider a multilinear mapping M. The goal is to find unit vectors x_1,...,x_r that maximize the norm of the output ||M(x_1,...,x_r)||. In our case, the vectors x_1,...,x_r are complex vectors of length 2, and the output M(x_1,...,x_r) is also a vector of length 2. The maximum value of ||M(x_1,...,x_r)|| such that x_1,...,x_r are unit vectors is a norm ||M|| for the multilinear mapping, so we are estimating the norm ||M||.

I maximize ||M(x_1,...,x_r)|| using gradient ascent. I did not have to use gradient ascent since I could have used a version of the power iteration technique that one commonly uses to find dominant eigenvectors and dominant singular vectors, but gradient ascent makes a better visualization.

For the visualization, suppose that x is one of the r vectors x_1,...,x_r. Then there is a unique complex number c where cx is a unit vector and cx[1] is a positive real number. We therefore show the coordinate cx[2] for the visualization (this is why the dots move quickly around the edge of the circle but slowly on the interior of the circle). The first instance of gradient ascent is colored red while the second instance of gradient ascent is colored blue.

We obtain the same value for M(x_1,...,x_r) after training twice only some of the time.

For this visualization, we set r=256.
Suppose that x_1,...,x_r are complex vectors each of length 2. Then construct an array of vectors x_{i,j} where x_{i,0}=x_i for all i and where
x_{i,j+1}=x_{2i,j}*_{i,j}x_{2i-1,j} for all i,j. Here, the operation *_{i,j} is a randomly generated bilinear mapping. We set M(x_1,...,x_r)=x_{1,8}. In other words, M is a multilinear mapping constructed from bilinear mappings.

This norm calculation problem is my own, but the notion of the operator norm of a bounded linear mapping between normed spaces generalizes to the norm ||M|| easily, so this particular problem is not very unique. I will follow up this visualization with other visualizations of maximizing the norm ||M(x_1,...,x_r)|| where x_1,...,x_r are still unit vectors but where M is a different construction of a multilinear mapping, and I suspect that the follow up experiments would have similar results. I have made this visualization in order to investigate some AI safety aspects of various fitness functions. This fitness shows that the use of multilinear mappings like M in machine learning should result in safer and more interpretable AI.

Unless otherwise stated, all algorithms featured on this channel are my own. You can go to https://github.com/sponsors/jvanname to support my research on machine learning algorithms. I am also available to consult on the use of safe and interpretable AI for your business. I am designing machine learning algorithms for AI safety such as LSRDRs. In particular, my algorithms are designed to be more predictable and understandable to humans than other machine learning algorithms, and my algorithms can be used to interpret more complex AI systems such as neural networks. With more understandable AI, we can ensure that AI systems will be used responsibly and that we will avoid catastrophic AI scenarios. There is currently nobody else who is working on LSRDRs, so your support will ensure a unique approach to AI safety.