Argmax – Details, episodes & analysis

We talk about Low Rank Approximation for fine tuning Transformers. We are also on YouTube now! Check out the video here: https://youtu.be/lLzHr0VFi3Y

In this episode we discuss the paper "Training language models to follow instructions with human feedback" by Ouyang et al (2022). We discuss the RLHF paradigm and how important RL is to tuning GPT.

We discuss NeurIPS outstanding paper award winning paper, talking about important topics surrounding metrics and reproducibility.

We talk about QMIX https://arxiv.org/abs/1803.11485 as an example of Deep Multi-agent RL.

Todays paper: Can Neural Nets Learn the Same Model Twice? Investigating Reproducibility
and Double Descent from the Decision Boundary Perspective (https://arxiv.org/pdf/2203.08124.pdf)

Summary:
A discussion of reproducibility and double descent through visualizations of decision boundaries.

Highlights of the discussion:

• Relationship between model performance and reproducibility
• Which models are robust and reproducible
• How they calculate the various scores

Todays paper: VICReg (https://arxiv.org/abs/2105.04906)

Summary of the paper
VICReg prevents representation collapse using a mixture of variance, invariance and covariance when calculating the loss. It does not require negative samples and achieves great performance on downstream tasks.

Highlights of discussion

• The VICReg architecture (Figure 1)
• Sensitivity to hyperparameters (Table 7)
• Top 5 metric usefulness

Todays paper: data2vec (https://arxiv.org/abs/2202.03555)

Summary of the paper
A multimodal SSL algorithm that predicts latent representation of different types of input.

Highlights of discussion

• What are the motivations of SSL and multimodal
• How does the student teacher learning work?
• What are similarities and differences between ViT, BYOL, and Reinforcement Learning algorithms.

This is the first episode of Argmax! We talk about our motivations for doing a podcast, and what we hope listeners will get out of it.

Todays paper: Reward is Enough

Summary of the paper
The authors present the Reward is Enough hypothesis: Intelligence, and its associated abilities, can be understood as subserving the maximisation of reward by an agent acting in its environment.

Highlights of discussion

• High level overview of Reinforcement Learning
• How evolution can be encoded as a reward maximization problem
• What is the one reward signal we are trying to optimize?

We talk about AlphaTensor, and how researchers were able to find a new algorithm for matrix multiplication.