This year’s International Conference on Machine Learning (ICML) will be an in-person event held in Baltimore, Maryland, July 17-23rd, 2022. For those unable to travel, there will be some virtual elements as well. In addition to the main session, the conference will include Tutorials, Workshops, and an Expo day. In its 39th year, ICML continues to be the world’s leading machine learning conference and provides a large gathering of bright minds. Welcoming researchers, engineers, students, and more, ICML boasts a broad and inclusive participant registry. Intel is proud to contribute cutting-edge research in several areas of the machine learning field.
Ensemble of neural networks reduces the variance within models and produces superior predictions. Because of this, ensemble learning is quickly becoming a prominent tool for advancing deep reinforcement learning algorithms. However, training large numbers of neural networks in an ensemble has an exceedingly high computation cost which may become a hindrance in training large-scale systems. In response, Intel Labs’ first work introduces DNS: a Determinantal Point Process based Neural Network Sampler that specifically uses k-dpp to sample a subset of neural networks for back propagation at every training step thus significantly reducing the training time and computation cost. Researchers integrated DNS in a Randomized Ensembled Double-Q (REDQ) learning algorithm for continuous control tasks and evaluated on MuJoCo environments. The experiments showed that DNS augmented REDQ outperforms baseline REDQ in terms of average cumulative reward and achieves this using less than 50% computation when measured in floating point operations per second (FLOPS).
Intel’s next presentation introduces a novel reinforcement learning algorithm. Learning a new skills can be a cumbersome undertaking because it often involves lots of new elements. However, the knowledge obtained while learning this skill will often make it easier to learn future skills. This is known as morphology-agnostic learning. The prototypical approach to reinforcement learning involves training policies from scratch for every new agent morphology. Recent work aims to eliminate the re-training of policies by investigating whether a morphology-agnostic policy, trained on a diverse set of agents with similar task objectives, can be transferred to new agents with unseen morphologies without re-training. This challenging problem previously required hand-designed descriptions of the new agent's morphology. Rather than using this labor intensive approach, Labs developed a data-driven method that learns a representation of morphology directly from the reinforcement learning objective. This is the first reinforcement learning algorithm that can train a policy to generalize to new agent morphologies without requiring a description of the agent's morphology in advance. The method is more flexible and easier to deploy than prior works because it eliminates the need for an engineer to provide morphology information that is difficult to obtain in the real world. The approach also delivers state-of-the-art generalization and robustness for controlling large collections of agents without explicit descriptions of morphology.
Intel Labs’ final work presented at the conference aims to improve upon Gaussian smoothing (GS). GS is a derivative-free optimization (DFO) algorithm that estimates the gradient of an objective using perturbations of the current parameters sampled from a standard normal distribution. Labs researchers generalize it to sampling perturbations from a larger family of distributions. Based on an analysis of DFO for non-convex functions, they propose to choose a distribution for perturbations that minimizes the mean squared error (MSE) of the gradient estimate. They successfully derived three such distributions with provably smaller MSE than Gaussian smoothing. To validate the claims, researchers conducted evaluations of the three sampling distributions on linear regression, reinforcement learning, and DFO benchmarks. The proposal improves on GS with the same computational complexity, and is competitive with and usually outperforms Guided ES and Orthogonal ES, two computationally more expensive algorithms that adapt the covariance matrix of normally distributed perturbations.
These innovations deliver on Intel Labs’ promise to seek answers, solve problems and scale solutions, furthering the future of machine learning. Intel will also host an AI Networking Happy Hour Tuesday, the 19th from 5:30-7:30PM at The Center Club. Add some fun to the conference weekend with food and drinks, entertaining demos, and a survey raffle with giveaways. Intel will cover first 100 ICML attendees, so be sure to reserve your spot ahead of time.
Register for the conference here.