Caroline Uhler (MIT), gives a OxCSML Seminar on Friday 2nd July 2021. Abstract: Massive data collection holds the promise of a better understanding of complex phenomena and ultimately, of better decisions. An exciting opportunity in this regard stems from the growing availability of perturbation / intervention data (genomics, advertisement, education, etc.). In order to obtain mechanistic insights from such data, a major challenge is the integration of different data modalities (video, audio, interventional, observational, etc.). Using genomics as an example, I will first discuss our recent work on coupling autoencoders to integrate and translate between data of very different modalities such as sequencing and imaging. I will then present a framework for integrating observational and interventional data for causal structure discovery and characterize the causal relationships that are identifiable from such data. We then provide a theoretical analysis of autoencoders linking overparameterization to memorization. In particular, I will characterize the implicit bias of overparameterized autoencoders and show that such networks trained using standard optimization methods implement associative memory. We end by demonstrating how these ideas can be applied for drug repurposing in the current COVID-19 crisis.

Department of Statistics

Causality and Autoencoders in the Light of Drug Repurposing for COVID-19

Qiang Liu (University of Texas at Austin) gives the OxCSML Seminar on Friday 4th June 2021. Abstract: Stein's method is a powerful technique for deriving fundamental theoretical results on approximating and bounding distances between probability measures, such as central limit theorem. Recently, it was found that the key ideas in Stein's method, despite being originally designed as a pure theoretical technique, can be repurposed to provide a basis for developing practical and scalable computational methods for learning and using large scale, intractable probabilistic models. This talk will give an overview for some of these recent advances of Stein's method in machine learning.

Recent Applications of Stein's Method in Machine Learning

Cynthia Rudin (Duke University) gives a OxCSML Seminar on Friday 14th May 2021. Abstract: While the trend in machine learning has tended towards more complex hypothesis spaces, it is not clear that this extra complexity is always necessary or helpful for many domains. In particular, models and their predictions are often made easier to understand by adding interpretability constraints. These constraints shrink the hypothesis space; that is, they make the model simpler. Statistical learning theory suggests that generalization may be improved as a result as well. However, adding extra constraints can make optimization (exponentially) harder. For instance it is much easier in practice to create an accurate neural network than an accurate and sparse decision tree. We address the following question: Can we show that a simple-but-accurate machine learning model might exist for our problem, before actually finding it? If the answer is promising, it would then be worthwhile to solve the harder constrained optimization problem to find such a model. In this talk, I present an easy calculation to check for the possibility of a simpler model. This calculation indicates that simpler-but-accurate models do exist in practice more often than you might think. This talk is mainly based on the following paper Lesia Semenova, Cynthia Rudin, and Ron Parr. A Study in Rashomon Curves and Volumes: A New Perspective on Generalization and Model Simplicity in Machine Learning. In progress, 2020. https://arxiv.org/abs/1908.01755

Do Simpler Models Exist and How Can We Find Them?

Aki Vehtari (Aalto University) gives the OxCSML Seminar on Friday 7th May 2021 Abstract: I discuss the use of the Pareto-k diagnostic as a simple and practical approach for estimating both the required minimum sample size and empirical pre-asymptotic convergence rate for Monte Carlo estimates. Even when by construction a Monte Carlo estimate has finite variance the pre-asymptotic behaviour and convergence rate can be very different from the asymptotic behaviour following the central limit theorem. I demonstrate with practical examples in importance sampling, stochastic optimization, and variational inference, which are commonly used in Bayesian inference and machine learning.

Practical pre-asymptotic diagnostic of Monte Carlo estimates in Bayesian inference and machine learning

Quan Zhou, Texas A and M University, gives an OxCSML Seminar on Friday 25th June 2021. Abstract: In a model selection problem, the size of the state space typically grows exponentially (or even faster) with p (the number of variables). But MCMC methods for model selection usually rely on local moves which only look at a neighborhood of size polynomial in p. Naturally one may wonder how efficient these sampling methods are at exploring the posterior distribution. Consider variable selection first. Yang, Wainwright and Jordan (2016) proved that the random-walk add-delete-swap sampler is rapidly mixing under mild high-dimensional assumptions. By using an informed proposal scheme, we obtain a new MCMC sampler which achieves a much faster mixing time that is independent of p, under the same assumptions. The mixing time proof relies on a novel approach called "two-stage drift condition", which can be useful for obtaining tight complexity bounds. This result shows that the mixing rate of locally informed MCMC methods can be fast enough to offset the computational cost of local posterior evaluation, and thus such methods scale well to high-dimensional data. Next, we generalize this result to other model selection problems. It turns out that locally informed samplers attain a dimension-free mixing time if the posterior distribution satisfies a unimodal condition. We show that this condition can be established for the high-dimensional structure learning problem even when the ordering of variables is unknown. This talk is based on joint works with H. Chang, J. Yang, D. Vats, G. Roberts and J. Rosenthal. Bio: Quan Zhou is an assistant professor of the Department of Statistics at Texas A&M University (TAMU). Before joining TAMU, he was a postdoctoral research fellow at Rice University. He did his PhD at Baylor College of Medicine.

Complexity of local MCMC methods for high-dimensional model selection

Distinguished Speaker Seminar - Friday 18th June 2021, with Susan Murphy, Professor of Statistics and Computer Science, Harvard John A. Paulson School of Engineering and Applied Sciences. Reinforcement Learning provides an attractive suite of online learning methods for personalizing interventions in a Digital Health. However after a reinforcement learning algorithm has been run in a clinical study, how do we assess whether personalization occurred? We might find users for whom it appears that the algorithm has indeed learned in which contexts the user is more responsive to a particular intervention. But could this have happened completely by chance? We discuss some first approaches to addressing these questions.

Assessing Personalization in Digital Health

Graduate Lecture - Thursday 3rd June 2021, with Dr Fergus Boyles. Department of Statistics, University of Oxford. Drug discovery is a long and laborious process, with ever growing costs and dwindling productivity making it ever more difficult to bring new medicines to the market in an affordable and timely fashion. There is a long history of applying statistical modelling and machine learning to problems in drug discovery, and, as in many fields, there is growing excitement about the potential of modern machine learning techniques to both automate and accelerate time-consuming tasks, and to enable previously unfeasible experiments. In this talk I will describe the drug discovery pipeline and introduce computer-aided drug discovery. Drawing on my own research and that of others, I will explain how machine learning is currently being applied to problems in drug discovery and highlight challenges and pitfalls that remain to be addressed.

Machine Learning in Drug Discovery

OxCSML Seminar - Friday 28th May 2021, presented by Alexandra Carpentier (University of Magdeburg). In this talk we will discuss the thresholding bandit problem, i.e. a sequential learning setting where the learner samples sequentially K unknown distributions for T times, and aims at outputting at the end the set of distributions whose means mu_k are above a threshold tau. We will study this problem under four structural assumptions, i.e. shape constraints: that the sequence of means is monotone, unimodal, concave, or unstructured (vanilla case). We will provide in each case minimax results on the performance of any strategies, as well as matching algorithms. This will highlight the fact that even more than in batch learning, structural assumptions have a huge impact in sequential learning.

Several structured thresholding bandit problems

Benjamin Guedj, University College London, gives a OxCSML Seminar on 26th March 2021. Abstract: PAC-Bayes is a generic and flexible framework to address generalisation abilities of machine learning algorithms. It leverages the power of Bayesian inference and allows to derive new learning strategies. I will briefly present the key concepts of PAC-Bayes and highlight a few recent contributions from my group.

A primer on PAC-Bayesian learning *followed by* News from the PAC-Bayes frontline

Julyan Arbel (Inria Grenoble - Rhône-Alpes), gives an OxCSML Seminar on Friday 30th April 2021, for the Department of Statistics.

Approximate Bayesian computation with surrogate posteriors

Ben Lambert, Department of Computer Science, University of Oxford, gives the Graduate Lecture on Thursday 6th May 2021, for the Department of Statistics.

Introduction to Bayesian inference for Differential Equation Models Using PINTS

Professor Sara Van de Geer, ETH Zürich, gives the Distinguished Speaker Seminar on Thursday 29th April 2021 for the Department of Statistics.

On classification with small Bayes error and the max-margin classifier

Murat Erdogdu gives the OxCSML Seminar on Friday 12th March, 2021, for the Department of Statistics.

Convergence of Online SGD under Infinite Noise Variance, and Non-convexity

Karolina Dziugaite (Element AI), gives the OxCSML Seminar on 26th February 2021. Abstract: Deep learning approaches dominate in many application areas. Our understanding of generalization (relating empirical performance to future expected performance) is however lacking. In some applications, standard algorithms like stochastic gradient descent (SGD) reliably return solutions with low test error. In other applications, these same algorithms rapidly overfit. There is, as yet, no satisfying theory explaining what conditions are required for these common algorithms to work in practice. In this talk, I will discuss standard approaches to explaining generalization in deep learning using tools from statistical learning theory, and present some of the barriers these approaches face to explaining deep learning. I will then discuss my recent work (NeurIPS 2019, 2020) on information-theoretic approaches to understanding generalization of noisy, iterative learning algorithms, such as Stochastic Gradient Langevin Dynamics, a noisy version of SGD.

Distribution-dependent generalization bounds for noisy, iterative learning algorithms

Professor Davina Durgana, award-winning international human rights statistician and professor with almost 15 years of experience developing leading global models to assess risk to modern slavery, gives a talk on their work on modern slavery. Abstract: Dr. Durgana will present her insights on the use of statistics in the global modern slavery vulnerability and prevalence field over the past decade. She will present work on the Global Estimates of Modern Slavery with the United Nations, Global Slavery Index, and on application of Multiple Systems Estimation throughout Europe with the UN Office on Drugs and Crime. She will also discuss compelling developments within leading national governments on prevalence estimation and how this work engages with the global policy community.

Finding Today’s Slaves: Lessons Learned From Over A Decade of Measurement in Modern Slavery

The Department of Statistics at Oxford is a world leader in research including computational statistics and statistical methodology, applied probability, bioinformatics and mathematical genetics. In the 2014 Research Excellence Framework (REF), Oxford's Mathematical Sciences submission was ranked overall best in the UK. This is an exciting time for the Department. We have now moved into our new home on St Giles and we are currently settling in. The new building provides improved lecture and teaching space, a variety of interaction areas, and brings together researchers in Probability and Statistics. It has created a highly visible centre for the Department in Oxford. Since 2010, the Department has been awarded over forty research grants with a total value of £9M, not counting several very large EPSRC and MRC funded awards for Centres for doctoral training.The main sponsors are the European Commission, EPSRC, the Medical Research Council and the Wellcome Trust. We offer an undergraduate degree (BA or MMath) in Mathematics and Statistics, jointly with the Mathematical Institute. At postgraduate level there is an MSc course in Applied Statistics, as well as a lively and stimulating environment for postgraduate research (DPhil or MSc by Research). Our graduates are employed in a wide range of occupational sectors throughout the world, including the university sector. The Department co-hosts the EPSRC and MRC Centre for Doctoral Training (CDT) in Next-Generational Statistical Science- the Oxford-Warwick Statistics Programme OxWaSP.

hubhopper

Department of Statistics

hubhopper