A bit advanced topics from ML

To separate in-depth notes from the fairly simple ones.

In no particular order

• Positional Encoding in Transformer Blocks
• Difficulty with Training RNNs
• Vision Transformers
• Long Tailed Visual Recognition, talks about Long Tail Distribution, Contrastive Pairs, von Misher Fisher distribution (gaussian on a curve).
  I know I know, this sounds more like a paper summary, yes it is.
  This is also referred in the paper summary directory
• Weight Initialization in Neural Networks
• What even are RAGs, Saw Sony Research asking for RAGs for undergrad research positions, hence this.

What Next ?

I should also mention what next do I plan to read
The above link also you to the index page of guides, which basically mentions the following pages :
From the index page for /guides

• More on computer science fundamentals, or just a mundane list from YouTube videos I couldn’t understand.

• Advanced Topics in Machine Learning that would feel overwhelming to a sophomore, compiled when I was a sophomore.