A New LINCC to Understanding the Universe
The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) is scheduled to make its first observation in 2024. The 10-year survey will collect 60 petabytes of data, which will contain valuable information about dark energy and dark matter, our galaxy and solar system, and near-Earth objects like asteroids. By scanning all of the visible sky every few nights, the project will yield an astronomical catalog that is thousands of times larger than any of its predecessors.
A catalog this large would be unmanageable using traditional data analysis tools. The LSST Interdisciplinary Network for Collaboration and Computing (LINCC) Frameworks team, led by Rachel Mandelbaum, professor of physics at Carnegie Mellon University, and Andrew Connolly, professor of astronomy at the University of Washington, has been hard at work over the last year building a team and beginning the software development projects that will lead to new frameworks that leverage modern computational methods like artificial intelligence and machine learning to analyze the LSST’s behemoth data sets. LINCC Frameworks is supported by Schmidt Futures.
“The LSST will yield many fundamental discoveries about our universe,” said Mandelbaum. “For example, by viewing the same patch of sky every three nights for 10 years we expect to identify new types of variable objects, in our galaxy and beyond, along with new objects in our solar system like asteroids or Kuiper belt objects.”
The LSST will also provide images of the universe at unprecedented depths and the two-dimensional images can be converted into four dimensions: effectively a color movie that encodes information about the distances to the galaxies and stars we are observing. This data will yield new information about the universe’s structure and expansion — and dark matter and dark energy.
But the challenge will be finding and pulling the information that each researcher using the data wants for their projects, which can be like finding the proverbial needle in a haystack.
To begin their monumental task, LINCC Frameworks assembled a team of researchers from many disciplines to work at the interface between algorithmic development and astrophysics. Jeremy Kubica, an alumnus of Carnegie Mellon’s School of Computer Science, joined the LINCC Frameworks team as its director of engineering. As a doctoral candidate in robotics at CMU, he completed research spanning computer science and astrophysics. He returns to the university with more than 15 years of experience working on large-scale machine learning and software engineering projects at Google. Kubica leads a software engineering team of experts who are located at both CMU and the University of Washington.
“I am very excited about LINCC Frameworks’ potential to enable the next generation of astronomical research. The Rubin telescope will generate data that will push the bounds of both state-of-the-art computer science and astronomy,” said Kubica.
Faculty from across the university, including Statistics and Data Science Professor Chad Schafer, Electrical and Computer Engineering Assistant Professor George Amvrosiadis, postdocs and graduate students in physics and computer science, and collaborators from the Pittsburgh Supercomputing Center and The University of Pittsburgh are all contributing to the project, as are collaborators from around the world.
LINCC Frameworks also spent the last year gathering input from the LSST user community.
“The software we create has to be able to answer our users’ most important scientific questions,” said Mandelbaum.
They hosted a workshop, From Data to Software to Science with the Rubin Observatory LSST, in the spring. More than 300 researchers expressed interest in the workshop. Those who attended gave input on the types of information they hoped to be able to extract from the LSST data and the computational challenges they expected to encounter when analyzing the data set.
The workshop resulted in a community white paper that outlined the software priorities for early LSST science, which included scalable cross-matching, photometric red-shift determination, scalable time-series analysis, and image access, among others.
Astrophysicist Alex Malz is a project scientist for LINCC Frameworks. Malz works with the software engineers to provide them with the context they need to build the tools that will address the community’s needs. As the project moves forward and more project scientists join the team, they’ll begin extragalactic science studies that use LINCC Frameworks resources to analyze early data from the Rubin Observatory.
“I’m excited to investigate how to optimize analysis design decisions for LSST, to explore surprises in early Rubin data, and to do those two things by building widely applicable, robust software tools along the way,” said Malz. “However, my role in LINCC Frameworks situates me to accomplish those goals more effectively than I could in isolation through the opportunity to work closely with a team of more diverse expertise than what I’d encounter elsewhere in academia and thus from whom I have plenty to learn.”
■ Jocelyn Duffy