'Deep Web' search to help decode space data

When you do a simple internet search on a topic, the results that appear are not the whole story. Most of the information is buried in the "Deep Web" in the mysterious online world.

Now, NASA researchers have joined the US Defense Advanced Research Projects Agency (DARPA) to harness the benefits of "Deep Web" searching for science.

The project called "Memex" could help catalog the vast amounts of data NASA spacecraft deliver on a daily basis.

"We are developing next-generation search technologies that understand people, places, things and the connections between them," said Chris Mattmann, principal investigator on Memex at the NASA's Jet Propulsion Laboratory in Pasadena, California.

"Memex" checks not just standard text-based content online but also images, videos, pop-up ads, forms, scripts and other ways information is stored to look at how they are interrelated.

"We are augmenting web crawlers to behave like browsers -- in other words, executing scripts and reading ads in ways that you would when you usually go online. This information is normally not catalogued by search engines," Mattmann explained.

Additionally, a standard web search does not get much information from images and videos but Memex can recognise what is in this content and pair it with searches on the same subjects.

The search tool could identify the same object across many frames of a video or even different videos.

The video and image search capabilities of Memex could one day benefit space missions that take photos, videos and other kinds of imaging data with instruments such as spectrometers.

"Searching visual information about a particular planetary body could greatly facilitate the work of scientists in analysing geological features," the authors noted.

Scientists analysing imaging data from the Earth-based missions that monitor phenomena such as snowfall and soil moisture could similarly benefit.

"We are developing open source, free, mature products and then enhancing them using DARPA investment and easily transitioning them via our roles to the scientific community," Mattmann pointed out.

Memex would also enhance the search for published scientific data, so that scientists can be better aware of what has been released and analysed on their topics, the US space agency said in a statement.