Accueil/ expose
Optimization's Hidden Gift to Learning: Implicit Regularization
mardi 01 octobre 2019

Loading the player...

Exposé de Nathan Srebro dans le cadre du Data Science Colloqiuum de l'ENS. 

It is becoming increasingly clear that implicit regularization afforded by the optimization algorithms play a central role in machine learning, and especially so when using large, deep, neural networks. We have a good understanding of the implicit regularization afforded by stochastic approximation algorithms, such as SGD, for convex problem, and we understand and can characterize the implicit bias of different algorithms, and can design algorithms with specific biases. But in this talk I will focus on implicit biases of local search algorithms for non-convex underdetermined problem, such as deep networks. In an effort to uncover the implicit biases of gradient-based optimization of neural networks, which holds the key to their empirical success, I will discuss recent work on implicit regularization for matrix factorization, linear convolutional networks, and two-layer ReLU networks, as well as a general bottom-up understanding on implicit regularization in terms of optimization geometry.

Voir aussi

  • Aucun exposé du même auteur.
  • Can Big Data cure Cancer?
    Jean-Philippe Vert
  • What physics can tell us about inferenc...
    Cristopher Moore
  • Beyond stochastic gradient descent for l...
    Francis Bach
  • The brain as an optimal efficient adapti...
    Sophie Deneve
  • Cosmostatistics: Tackling Big Data from ...
    Jean-Luc Starck
  • Searching for interaction networks in pr...
    Rémi Monasson
  • Brain-computer interfaces: two concurren...
    Maureen Clerc
  • Machine learning in scientific workflow...
    Balàzs Kégl
  • Learning Graph Inverse Problems with Neu...
    Joan Bruna
  • Towards developmental AI
    Emmanuel Dupoux
  • Prototype-based classifiers and relevanc...
    Michael Biehl
  • Generative modelling with diffusion : th...
    Valentin De Bortoli
Nathan Srebro
Toyota Technological Institute at Chicago

Plus sur cet auteur
Voir la fiche de l'auteur

Cliquer ICI pour fermer
Téléchargements :
   - Télécharger la vidéo
   - Télécharger l'audio (mp3)

Dernière mise à jour : 23/06/2020