AR for Collaborative Visual Inspection

Short name: AR-CVI

By TUM

Purpose

The component provides the Augmented Reality instructions for a worker who performs assembly or scanning tasks. The AR information is displayed using an overhead projector. The proper position of the AR visuals is estimated based on the depth camera mounted in the system.

This component takes as an input AR visuals, ground tuth (pointclouds of expected objects) and configuration information calculates the pose of objects and displays the visuals accordingly. The otput from the componete is mostly focused on the status of the process.

Data interfaces

Input and output data (but not user interfaces):

NOTE 1: the component is a closed system consisting of a depth camera, projector and PC for calculations therefore the data that is exchanged between these 3 elements is not considered as inputs or outputs. What is described is the info that requested from “outside” and provided to “outside” of the component.

NOTE 2: entire comunication is using Pub/Sub paradigm in ROS (and transformed to Fiware)

1. INPUT: Sequence Description
   • Format: ROS custom message
   • list of names of all AR visuals and ground truth point clouds ordered according to the steps of the process
   • Size depends on the number of steps in the process
2. INPUT: Projection slides (gometric primitives)
   • Format: ROS custom message
   • Containing info like id, name, POI, List of Texts, CUrrent status
3. INPUT: Ground Truth Point Cloud
   • Format: PointCloud2
   • Ground Truth information for object detection (either generated from CAD or taught)
   • Data volume depends highly on the selected sensor and size of the object.
4. INPUT: Control messgae
   • Format: ROS custom message
   • Separate ROS topic to enable configuring of the visuals and pointc clouds of the process.
   • Will contain info like: how many slides, process name, what pointclouds to expect and so on.
   • triggered at the beginning to anable listening to the incomung messages (2. and 3.).
5. OUTPUT: Sequence status
   • Format: ROS custom message
   • Publishing regularly info about current sequence step, slide, and cloud.
6. OUTPUT: Anomaly status
   • Format: ROS custom message
   • If anomaly occurs (wrong object, wrong step) this message will be published mentioning the anomaly type and associated sequence step.
7. OUTPUT: Process report
   • Format: ROS custom message
   • Published at the end of the process describing various info like sequence id, anomaly counter and type etc.

For now on no component provides the data. Especially useful would be to receive the inputs 1, 2 and 3 from the other components. For the prototype those inputs will be generated by TUM