Desenvolvemento dunha aplicación de realidade aumentada espacial para guiar usuarios na ensamblaxe dunha mesa

Abstract

[Resumo]: Este Traballo de Fin de Grao consiste na elaboración dun prototipo de Realidade Aumentada Espacial (SAR, Spatial Augmented Reality) de baixo custo e alto rendemento, para poder guiar ós novos usuarios dunha empresa dedicada ó ensamblaxe de mesas nas súas primeiras semanas. A motivación principal deste proxecto é que as empresas non teñen persoal nin tempo para formar ós novos empregados, debido á tremenda rotación que está acontecendo. Debido a elo, co sistema proposto neste TFG a empresa pode formar aos novos usuarios de forma específica e cun custo moi baixo. O desenvolvemento descrito neste TFG baséase nunha Raspberry Pi 5, a cal permite poñer a proba a nova arquitectura desta SBC (Single Board Computer) para ver a súa integración cos módulos de cámara xa existentes e co procesamento dunha rede neuronal convolucional (CNN). Igualmente, o traballo desenvolto fai uso dun modelo Only Look Once (YOLO) que é un algoritmo de detección de obxectos en tempo real, no que faremos que detecte dous tipos distintos de obxectos: un que representará o lugar onde o operario terá que aplicar un pegamento, e outro onde terá que perforar cun taladro para posteriormente poñer un parafuso. Para a comunicación entre o operario e o prototipo úsase a detección da súa man durante tres segundos nunha parte concreta da imaxe proxectada. Deste xeito o operario poderá ir navegando polos distintos pasos que require o proceso de ensamblaxe. O prototipo é capaz de explicar de forma verbal a tarefa que o operario terá que desenvolver en cada paso polo que este navegue. Para a proxección da imaxe, utilízase un mini proxector conectado vía HDMI co SBC para reducir a latencia, e faise uso dun programa para a calibración e outro para a execución do proceso en si.[Abstract]: This Final Degree Thesis consists in the development of a prototype of Augmented Reality of Spatial (SAR, Spatial Augmented Reality) of low cost and high performance, to power guide the new users of a company dedicated to the assembly of tables in their first work. The main motivation behind this project is that companies don’t have staff or time to train new employees, due to the tremendous turnover that is happening. due therefore, with the system proposed in this TFG, the company can train new users in a way specific and at a very low cost. The development described in this TFG is based on a Raspberry Pi 5, which allows you to put test the new architecture of this SBC (Single Board Computer) to see its integration with already existing camera modules and with the processing of a convolutional neural network (CNN). Likewise, the developed work makes use of an Only Look Once (YOLO) model that is one real-time object detection algorithm, in which we will make it detect two types other than objects: one that will represent the place where the operator will have to apply a glue ment, and another where you will have to drill with a drill and then insert a screw. For the communication between the operator and the prototype, the detection of his hand is used during three seconds in a specific part of the projected image. In this way the operator will be able to go navigating through the various steps required by the assembly process. The prototype is able to verbally explain the task that the operator will have to carry out in each step through which he navigates. For the projection of the image, a mini projector connected via HDMI to the SBC is used reduce latency, and one program is used for calibration and another for the execution of the process itself.