DEX, a mixed reality Earth dictionary

Milan

Local People's Choice

NASA's Space Apps Challenge 2017

Jury's panel - Milan

Global Nominee

NASA's Space Apps Challenge 2017

DEX is a mixed reality peer reviewed platform for indexing and discovering scientific knowledge. Explore the world, collect Knowledge Points and help others understand our planet with the power of crowdsourced based data gathering challenges!

Challenge

A creative way for the public and scientists alike to learn scientific and technical Earth-related terms.

Problem

Earth and space are both beautiful and complex, but scientists sometimes struggle to communicate how every aspect of nature is connected and how they interact with each other.

Themes and trends

Starting from some common themes and trends, such as the will and need to protect Earth’s diversity and sustainability, we set out to create an app that aims to spread knowledge both in the scientific community and among the public alike.

Solution

Introducing DEX, an interactive and collective Earth dictionary based on crowdsourcing and mixed reality.

DEX stands on three main pillars:

1. Mixed reality
An immersive way to experience content, in order to make complexity understandable, graspable and enjoyable by users.

2. Crowdsourcing
A peer reviewed system to collect and share knowledge, where scientists and enthusiasts can collaborate and validate content, while ensuring proper credit to be given through referencing.

3. Gamification
A gamified system where increasing knowledge is awarded with in-app points. Each user’s history is shown in their own personal diary, giving an overall sense of progression with total earned “knowledge points” and current level.

Discovering new content and learning new scientific terms is easy, intuitive and direct. It’s always possible to see the source of definitions, images and augmenting material such as explanatory schemes and visualizations. Additionally, the crowdsourced nature of the uploaded contents allows it to be always monitored and corrected if wrong or incomplete.

The user interface of DEX uses metaphors like paper and diaries to create a tactile experience, that paired with mixed reality has the aim to emulate the act of gathering and saving sheets and photos in a researcher’s journal.


Prototypes

App Prototype

We assembled a first prototype of how the app could work in the final product. You can explore all the app's core features, from the map, to the Explore page, down to the Discovery Diary where all progress is stored.

Computer Vision Prototype

Our working prototype showcases the potential of the mixed reality feature of DEX. The prototype allows you to take a picture, which is then live processed by the Google Cloud Vision API, and returns text printed on screen detailing the recognized subject of the picture along with its description (automatically retrieved from Wikipedia), all of this with the form factor of the visual user interface showed in the video.

2017

Team
Valeria Aufiero
Andrea Benedetti
Lorenzo Bracco
Elena Mongelli
James Simonetta
Alessandro Zotta

Space Apps 2017