NITRUS is a project focused on enriching human interaction with digital experiences through Artificial Intelligence-based gesture recognition. Its main goal is to bridge the gap between traditional interfaces and XR environments while keeping the solution accessible and computationally efficient.
The project is built on the generation of diverse gesture datasets and the training of AI algorithms specialized in gesture recognition. These models are integrated into optimized pipelines for real-time execution and connected to the Rainbow teleconference platform.
The system works by first locating people in the image feed and creating small, individual video sequences for each person. Each sequence is then analyzed to determine whether a gesture is being performed and, if so, classify it. The solution supports multiple people simultaneously and is designed to run directly in web environments.
The system currently supports 12 gestures, achieves over 90% gesture recognition accuracy, operates with less than 100 ms latency on CPU, and is ready for integration into video call applications and other interactive use cases.
The project has been carried out within the framework of the CORTEX2 project.
