Implementing The Spherical K-Means Dictionary

Описание: To summarize, it works the worst out of the 3 different dictionaries that we tried in the Hopfield Dictionary playlist. The reason for that is most likely that the mutable updates aren't good at disambiguating the cluster assignments, and a much better way of implementing the dictionary would be to do outright insertions like some algorithms do.

We are filled with disappointment, but this is about par the course for AI.

We tried deep RL as well as evolutionary algorithms in the past and they didn't work on poker. Now we'll add shallow unsupervised methods to the mix.

Really, that only leaves hand crafted functions. But do we really want to spend our time making those for a game of poker when we could be doing it to trade the financial markets instead?

---

#spiral #functionalprogramming #machinelearning #reinforcementlearning #programming #cpp #programminglanguage #compiler #parallelprogramming #cuda #gpu

Playlist(HD-CFR):    • Hopfield Dictionary CFR  
Playlist(Staged FP in Spiral):    • Staged Functional Programming In Spiral  
Spiral: https://github.com/mrakgr/The-Spiral-...
Spiral's ML Library: https://github.com/mrakgr/Spiral-s-ML...
Github: https://github.com/mrakgr/

If you have interesting work opportunities and require an expert functional programmer, don't hesitate to get in touch. My email is on my Github profile. Put "Work" as the subject in order to avoid the spam filters.

Music:    • Epic Metal Instrumentals Power for Motivat...  

00:00:00 - Start
00:00:02 - A brief summary of the compiler bug from last video
00:00:52 - Implementing Spherical K-Means using Arrayfire in pure C++
00:02:22 - How to do L2 normalization
00:07:41 - Sketching out the design
00:12:38 - What the Spherical K-Means update is doing (live explanation)
00:17:38 - Calculating the updates
00:33:42 - It's a pretty convoluted implementation, is there an easier one out there?
00:38:28 - A simpler vector implementation
00:42:54 - Spherical K-Means implementation in Spiral
00:49:59 - Why the key update is done at the start
00:51:09 - Getting it to run
00:53:55 - A series of disappointing results
00:58:00 - "I think the main issue is that K-Means is very sensitive to initialization"
01:00:21 - Printing out the cluster id assignments by the Spherical K-Means algorithm
01:01:18 - 90% of the keys are assigned to a single cluster id
01:01:55 - What is the neural gas update?
01:05:03 - Implementing the neural gas update
01:12:21 - Testing the update
01:13:15 - Is the pickler right?
01:16:27 - Is the normalization working correctly?
01:22:31 - It simply doesn't work
01:23:01 - The issue is most likely that the mutable update doesn't work well in highly dimensional spaces
01:23:41 - This is too difficult
01:24:22 - Outright insertions would work much better than mutable updates
01:25:31 - We'll hand craft the clustering function, the perfect hashing pickler will make it much easier
01:25:45 - One last experiment
01:26:21 - Not subtracting the min could be causing these issues
01:27:40 - Things are better but still extremely bad with numerous keys clumped to the same cluster id