Scene-functional alignment in an industrial environment

Industrial facilities built before modern digital modeling tools often lack a direct connection between their functional schematics and their 3D geometry. Yet, this link is crucial as it paves the way to building a 3D digital twin useful for many industrial applications such as simulation, intervention planning, equipment retrieval and training. For example, this link can be used to train nuclear plant operators in virtual reality.To align functional information with the physical layout of the scene, a strategy consists in acquiring images and 3D point clouds inside the facility. Images provide visual details while point clouds capture the precise geometry and position of the objects. When combined with functional schematics (such as P&IDs, Piping and Instrumentation Diagrams), these data, if properly processed, can be used to obtain a 3D model linked to the functional information. Although it is possible for a human to associate the schematics with the acquired data, this task is time-consuming and labor-intensive, and impractical for large facilities due to the large number of components, sometimes numbering in tens of thousands.This thesis aims to develop tools to accelerate this alignment process. The objective is to locate within the 3D scene each piece of equipment and each pipeline represented in the schematics relying on the images and point clouds and using methods drawn from data science and computer vision.This alignment problem is difficult because the schematics lack distance information between objects, acquisitions may be inconsistent or incomplete, and industrial objects are often complex, occluded, or of variable appearance for the same object type. Also, industrial environments are poorly represented in computer vision datasets.Previous works address the alignment between industrial scenes and functional data, but they mostly tackle isolated subproblems such as object detection from the acquisitions. To date, no end-to-end framework has been proposed and experimentally demonstrated.In this thesis, a multi-stage approach is presented to align the acquisitions with functional schematics and a method is proposed for each step. This approach provides alignment between industrial scenes and their functional information emphasizing robustness to the various difficulties that may appear. In addition, we introduce IRIS, the first dataset combining real 2D, 3D, and functional data in an industrial environment on which experiments have been conducted to demonstrate the feasibility of the proposed approach.