This is free educational App that I wrote in response to an educational challange by Andrej Karpathy on X.
It is an App for people interested in machine learning (LLM) and the so-called positional encoding. In the famous Attention is All You Need- paper, integer positions are embedded into vectors by a formula that is quite similar to the discrete Fourier transform (but for small indices the sinoids are denser than they need to be and they are too separated for larger indices).
Play with this App to acquire a deeper intuitive understanding of the above mentioned positional encoder.
The App shows how the correlation between two positional vectors depends only on the distance between the corresponding two integer position, and that the correlation decay with increasing distance.
(If you have learnt something from the App, or if you think others may do so, please review it at App Store)
