The NASA Frontier Development Lab is a public/private partnership that engages in interdisciplinary research that benefits the space program and all humankind. Each summer, the program outlines broad challenges for researchers to tackle in a competitive, collaborative environment. One of this year’s challenges focused on applying super-resolution techniques to solar magnetograms. A magnetogram is a map of the spatial variation and strength of the solar magnetic field. Several instruments have been developed to produce magnetograms; their quality and sensitivity has improved over time, enabling researchers to better understand the solar magnetic field and geoeffective space weather events. While these instruments have captured decades worth of data, the quality and resolution of the data itself has a significant variance.
Figure 1: variance of magnetogram data
As shown in figure 1, the oldest magnetogram data available, collected from Mt. Wilson Observatory in Los Angeles, spans a timeframe that began in the late 1960s. The same instrument, however, also has the lowest spatial resolution of all the instruments in use today.
Figure 2 shows a range of images captured using different instruments. Moving from left to right, one can see that there is a tremendous improvement in spatial resolution. The first major question in the challenge tackled this issue: is it possible to apply computer vision techniques to observation data that spans a large range of time across a variety of instruments?
Figure 2: Magnetogram images from various sources
The second major question of the challenge relates to the image below:
Figure 3: Magnetic field mapping examples.
Most instruments map the magnetic field of the sun over a full disc, as shown on the left. The newest instrument, known as the Solar Optical Telescope (SOT), cannot survey a full disk, but instead can create an extremely dense mapping of a rectangular patch on the surface of the sun. To put this into perspective, in the image above, the rectangular patch on the right is a magnetogram from the SOT instrument that corresponds to the small rectangle on the image generated by a Helioseismic Magnetic Imager (HMI) on the left. The challenge also asked: is it possible to translate very high-resolution local data produced by the SOT to the full-disc images provided by other instruments?
Why is this challenge important? To date, the lack of homogeneity between magnetograms has prevented their combined use in several applications. This presents a serious obstacle for analyzing and predicting space weather events. If all the data collected over decades could be converted to a homogeneous representation, it could push the boundaries of our understanding of the Sun and also enable scientists to predict important events which may have a direct impact on the earth.
The super-resolution map challenge focused on expanding the availability and use of solar magnetogram data. Specifically, the aim of the challenge was to apply deep learning-based super-resolution techniques that can increase the availability of data by utilizing information from a variety of instruments spanning a range of both time and space resolutions. Super-resolution is a class of techniques that enhances the resolution of an imaging system. Deep learning-based computer vision can be used to estimate images at a higher resolution than available in the individual images. The hypothesis is that these techniques can be applied to train or fine tune a deep network to upsample magnetograms.
With these factors in mind, the super-resolution core mentors and team of researchers got to work, supported by Intel AI mentors Sairam Sundaresan and Santiago Miret and mentors from Google Cloud, element.ai and Lockheed Martin.
Using Startup Sprints to Generate Insights
The team needed a way to capture a diverse set of opinions and ideas while reducing arguments and conflicts - with experts are spread across the world. They chose the Sprint methodology to address the shortcomings of traditional brainstorming sessions. By using focused and timed sessions with clearly defined objectives and silent debates, a stronger consensus was formed within the core team and stakeholders. This method reduced the noise and distractions often found in brainstorming sessions, while highlighting underlying commonalities and insights.
In less than a week’s time, the group brainstormed potential solutions, selected which ones to pursue, and navigated team and mentor dynamics. Experts in the fields of heliophysics, space instruments, and deep learning were also available to provide insights and help map out problems and possible solutions. One of the major insights from the Sprint was to move away from a “traditional” start-to-finish project plan and adopt a pipeline framework, so that each segment of the solution could be modularized and improved upon as the challenge progressed.
Figure 4: The super-resolution team
Figure 5: The Sprint process in action
After the Sprint: Building from A Strong Foundation
Intel AI mentors continued to provide remote support to the team as the project progressed, including training on fast.ai, a PyTorch-based deep learning library, the application of superresolution techniques that can be trained in minutes and deliver state-of-the-art performance, and insights into using registration techniques to reduce hallucinations in upsampled super resolution maps. Intel mentor Sairam Sundaresan also emphasized the importance of having a physics-based component in the loss function of the upsampled images, to enforce functional as well as aesthetic consistency. With thorough experimentation and iterative research, the team converged on using HighResNet, a state of the art deep network for super-resolution. The network was adapted to convert magnetograms from a source survey while preserving the features and systematics of the target survey. Figure 5 shows a sample result of a cross-instrument conversion by taking a magnetogram produced by the GONG instrument and converting it to one that would be produced by the HMI instrument. The GONG magnetogram is on the left and the converted HMI representation of the same magnetogram is on the right.
Figure 6: Result of a cross-instrument magnetogram image conversion
Overall, the NASA FDL challenge was a great opportunity for Intel AI researchers and data scientists to get involved in multi-disciplinary projects that apply deep learning techniques to problems in space science. Throughout the process, Intel AI mentors got to interact with experts from other fields and exchange knowledge in a way that fostered collaboration between scientists and industry experts. For both Santiago and Sairam, this was an eye opening experience that highlights the power of cross disciplinary teams in tackling some of the most challenging problems in the world today. You can view the final presentation by the team here and we invite you to follow Intel AI and NASA FDL on Twitter for more updates - including how to be a part of next year’s challenge.
Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No product or component can be absolutely secure. Check with your system manufacturer or retailer or learn more at intel.com.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.
Mary is the Community Manager for this site. She likes to bike, and do college and career coaching for high school students in her spare time.