Tuesday, 14 July 2015

Big data from new telescopes

I read an article recently on the Guardian's science website called Big universe, big data, astronomical opportunityThe article discussed the rise of large datasets in science, particularly in astronomy, and the need for astronomers to adapt to the challenges faced by this by acquiring new skills for analysing such data.
The Large Synoptic Survey Telescope
(Credit: LSST)

The article made many good points about the rise of big data. We are, after all, on the verge of a number of large projects that will produce more data than we've ever seen before. These include the Gaia satellite (which I've talked about before), new sub-mm and radio telescopes such as ALMA and the Square Kilometre Array, and the Large Synoptic Survey Telescope. The latter is particularly noteworthy because once it enters operation in 2022 it will produce 30TB of data each night, more than most telescopes produce in a year!

Astronomers will need to learn not just to manage these datasets, but to effectively sort through and process them. New algorithms will need to be designed, utilising artificial intelligence and advanced machine learning methodologies, as well as new approaches to visualise and understand the results. Astronomers will have to embrace this because this is the way science is going, and the telescopes that will produce this data are already being built, or in some cases are coming online as we speak.

Despite making these good points the article then appears to suggest that these new telescopes and satellites, designed to answer the most far-reaching questions humanity has ever asked, aren't what we need. "Have we leaned all that we can from the data that we have?", the author asks, suggesting that we should analyse all the data we have before collecting more. The article even suggests that the data we have could "hold answers to some of the fundamental questions of the Universe we are seeking".

I strongly disagree with the author on this point and find this attitude rather short-sighted. The data that we have may provide answers to some questions, but certainly not the big questions we want to answer, and that's where we should be investing our efforts. We won't find answers in our existing data to questions such as how the Sun and the Earth formed, how the Universe began, or whether there is life out there, it's just not possible.

Astronomical research is the continuous pursuit of answers to the greatest questions we can ask. We usually can't answer these questions in one go so instead we answer them step-by-step, performing experiment after experiment, refining the question as we go. For each experiment we design the equipment needed to answer the question, and then when that question has been answered we take the research further. If the same equipment or telescope can be used to answer the next question then we use it, but often the next question involves looking further, deeper and at different wavelengths, requiring new telescopes and equipment.

Just because a lot of data already exists doesn't mean that the answers to our questions can be found there. The great explorers of the past didn't discover new continents by searching around at home, they built great ships and went in search of discovery. Astronomers shouldn't be searching around in old data, but building great telescopes and searching the skies for their discoveries!


  1. Thank you for sharing such a nice and interesting blog with us. Hope it might be much useful for us. keep I have read your blog its very attractive and impressive. I like it your blog... Big Data Training in Chennai
    Big Data Training Institute in Chennai

  2. Infycle Technologies provides the Big Data Hadoop Training in Chennai for students, freshers, and tech professionals. Infycle also offers other professional courses such as Java, DevOps, Artificial Intelligence, Python, Oracle, Java, Digital Marketing, Data Science, Power BI, Cyber Security, Selenium Testing, etc., which will be trained with theory and practical classes. Call 7504633633 and get more info for a free demo.

  3. Infycle Technologies, the to Chennai's No.1 software training institute, Infycle Technologies, provides the best Data Science training in Chennai for freshers, college students, and tech professionals along with other corporate courses such as Cloud computing, DevOps, Data Science, Digital Marketing, Full Stack Development, Python, Big Data, Selenium, iOS, and Android development, Java and Hadoop with 100% hands-on training. Call 7502633633 to get more info and a free demo.

  4. Jump into a great learning experience of learning AWS Training in Chennai from Infycle Technologies, the finest software training Institute in Chennai. Also, a proper place to learn other technical courses like Cyber Security, Graphic Design and Animation, Block Security, Java, Cyber Security, Oracle, Python, Big data, Azure, Python, Manual and Automation Testing, DevOps, Medical Coding etc., and here we provide well-experienced trainers with excellent training to the freshers. And we also provide 100+ Live Practical Sessions with Real-Time scenarios which helps the students in learning the technical stuff easily and they are able to crack interviews in top MNC’s with an amazing package. for more queries approach us on 7504633633, 7502633633.

  5. This comment has been removed by the author.