Big data from new telescopes

I read an article recently on the Guardian's science website called Big universe, big data, astronomical opportunity. The article discussed the rise of large datasets in science, particularly in astronomy, and the need for astronomers to adapt to the challenges faced by this by acquiring new skills for analysing such data.

The Large Synoptic Survey Telescope
(Credit: LSST)

The article made many good points about the rise of big data. We are, after all, on the verge of a number of large projects that will produce more data than we've ever seen before. These include the Gaia satellite (which I've talked about before), new sub-mm and radio telescopes such as ALMA and the Square Kilometre Array, and the Large Synoptic Survey Telescope. The latter is particularly noteworthy because once it enters operation in 2022 it will produce 30TB of data each night, more than most telescopes produce in a year!

Astronomers will need to learn not just to manage these datasets, but to effectively sort through and process them. New algorithms will need to be designed, utilising artificial intelligence and advanced machine learning methodologies, as well as new approaches to visualise and understand the results. Astronomers will have to embrace this because this is the way science is going, and the telescopes that will produce this data are already being built, or in some cases are coming online as we speak.

Despite making these good points the article then appears to suggest that these new telescopes and satellites, designed to answer the most far-reaching questions humanity has ever asked, aren't what we need. "Have we leaned all that we can from the data that we have?", the author asks, suggesting that we should analyse all the data we have before collecting more. The article even suggests that the data we have could "hold answers to some of the fundamental questions of the Universe we are seeking".

I strongly disagree with the author on this point and find this attitude rather short-sighted. The data that we have may provide answers to some questions, but certainly not the big questions we want to answer, and that's where we should be investing our efforts. We won't find answers in our existing data to questions such as how the Sun and the Earth formed, how the Universe began, or whether there is life out there, it's just not possible.

Astronomical research is the continuous pursuit of answers to the greatest questions we can ask. We usually can't answer these questions in one go so instead we answer them step-by-step, performing experiment after experiment, refining the question as we go. For each experiment we design the equipment needed to answer the question, and then when that question has been answered we take the research further. If the same equipment or telescope can be used to answer the next question then we use it, but often the next question involves looking further, deeper and at different wavelengths, requiring new telescopes and equipment.

Just because a lot of data already exists doesn't mean that the answers to our questions can be found there. The great explorers of the past didn't discover new continents by searching around at home, they built great ships and went in search of discovery. Astronomers shouldn't be searching around in old data, but building great telescopes and searching the skies for their discoveries!