A bit advanced topics from DS/ML/DL

To separate in-depth notes from the fairly simple ones.

• A bit advanced topics from DS/ML/DL
  • In no particular order
    • Work in progress
    • Completed, please have a look monsieur
  • What Next ?

In no particular order

Work in progress

• Dead neurons in neural networks
• Batch Norm VS Layer Norm
• Are some directions illegal for the sigmoid activation function ?
• Understanding cross and self attention
• Internal Covariate Shift, yes the reason why Batch Norm is a thing.

Completed, please have a look monsieur

• Positional Encoding in Transformer Blocks
• Difficulty with Training traditional RNNs
• Vision Transformers
• Long Tailed Visual Recognition, talks about Long Tail Distribution, Contrastive Pairs, von Misher Fisher distribution (gaussian on a curve).
  I know I know, this sounds more like a paper summary, yes it is.
  This is also referred in the paper summary directory
• Weight Initialization in Neural Networks
• What even are RAGs, Saw Sony Research asking for RAGs for undergrad research positions, hence this.
• Extreme Classification, which is a topic born out of research at Microsoft.

Ƹ̵̡Ӝ̵̨̄Ʒ Ƹ̵̡Ӝ̵̨̄Ʒ Ƹ̵̡Ӝ̵̨̄Ʒ
butterfly as a line break

What Next ?

I should also mention what next do I plan to read
The above link also you to the index page of guides, which basically mentions the following pages :
From the index page for /guides

• More on computer science fundamentals, or just a mundane list from YouTube videos I couldn’t understand.

• Advanced Topics in Machine Learning that would feel overwhelming to a sophomore, compiled when I was a sophomore.