Towards the Internet of Augmented Things: An Open-source Framework to Interconnect IoT Devices and Augmented Reality Systems

Abstract

[Abstract] The latest Augmented Reality (AR) and Mixed Reality (MR) systems are able to provide innovative methods for user interaction, but their full potential can only be achieved when they are able to exchange bidirectional information with the physical world that surround them, including the objects that belong to the Internet of Things (IoT). The problem is that elements like AR display devices or IoT sensors/actuators often use heterogeneous technologies that make it difficult to intercommunicate them in an easy way, thus requiring a high degree of specialization to carry out such a task. This paper presents an open-source framework that eases the integration of AR and IoT devices as well as the transfer of information among them, both in real time and in a dynamic way. The proposed framework makes use of widely used standard protocols and open-source tools like MQTT, HTTPS or Node-RED. In order to illustrate the operation of the framework, this paper presents the implementation of a practical home automation example: an AR/MR application for energy consumption monitoring that allows for using a pair of Microsoft HoloLens smart glasses to interact with smart power outlets.