Improving MFVI in Bayesian Neural Networks with Empirical Bayes: a Study with Diabetic Retinopathy Diagnosis

Specifying meaningful weight priors for variational inference in Bayesian deep neural network (DNN) is a challenging problem, particularly for scaling to larger models involving high dimensional weight space. We evaluate the recently proposed, MOdel Priors with Empirical Bayes using DNN (MOPED) method for Bayesian DNNs within the Bayesian Deep Learning (BDL) benchmarking framework. MOPED enables scalable VI in large models by providing a way to choose informed prior and approximate posterior distributions for Bayesian neural network weights using Empirical Bayes framework. We benchmark MOPED with mean field variational inference on a real-world diabetic retinopathy diagnosis task and compare with state-of-the-art BDL techniques. We demonstrate MOPED method provides reliable uncertainty estimates while outperforming state-of-the-art methods, offering a new strong baseline for the BDL community to compare on complex real-world tasks involving larger models.


Ranganath Krishnan

Research Scientist, Anticipatory Computing Lab

View authors bio

Mahesh Subedar

Research Scientist, Anticipatory Computing Lab

View authors bio

Omesh Tickoo

Angelos Filos

Yarin Gal

Related Content

Clone Swarms: Learning to Predict and Control Multi-Robot…

In this paper, we propose SwarmNet -- a neural network architecture that can learn to predict and imitate the behavior…

Triplet-Aware Scene Graph Embeddings

Scene graphs have become an important form of structured knowledge for tasks such as for image generation, visual relation detection,…

View publication

Imitation Learning of Robot Policies by Combining Language,…

In this work we propose a novel end-to-end imitation learning approach which combines natural language, vision, and motion information to…

Evolutionary Reinforcement Learning for Sample-Efficient Multiagent Coordination

A key challenge for Multiagent RL (Reinforcement Learning) is the design of agent-specific, local rewards that are aligned with sparse...

Stay Connected

Keep tabs on all the latest news with our monthly newsletter.