project

Mixed Reality (MR)

  • On site at the Rhodes neoclassical school
    On site at the Rhodes neoclassical school
  • The H-BIM model over the façade
    The H-BIM model over the façade



The building's digital twin on site, exactly where the work happens


In brief

Point a phone or tablet at the building and witness the digital model in place: walls, rooms, building elements, sensors, equipment, and tasks. Technicians and engineers monitor the HVAC performance and maintenance, the inspectors and archaeologists use the SINCERE MR app to mark a crack exactly where it is, describe it, photograph it and send it back to the office in one step. No more "a crack somewhere on the north wall" and photos circulated in web folders.




Overview

The SINCERE MR app is a BIM-based mixed reality application developed by up2metric that connects the physical building with its digital twin, with a focus on historic buildings. It runs on Android and iOS phones and tablets and is fully connected to the SINCERE Building Management Platform: the tasks, models, sensors, and equipment inventory created on the web appear in the app, and everything recorded on site returns to the platform.

The app aligns the model with the real building using printed image anchors. The manager places the anchors on the 3D model in the platform and prints them; on site, the user scans one anchor and the model, the tasks and the markers appear at their real positions. Multiple anchors are calibrated as groups on the platform, so no hardware has to be fixed in the historic fabric, and every later visit uses the same spatial reference.

An optimized Field Mode serves the people who work on site, such as technicians, engineers, inspectors and archaeologists: it shows only their assigned tasks and the MR tools they need. Experts use the full MR mobile application capabilities to review findings on site, add new ones, add equipment and sensors in the actual position.




Main applications

1. Building inspection. The user opens an assigned task, scans the anchor and walks the inspection route. For each defect, one tap places a marker on the real surface, followed by a title, a description and a photo. Each marker becomes an observation linked to the inspection and to its exact 3D position.

2. Maintenance of building services. Step-by-step procedures-based on a maintenance plan for the equipment inventory- for preventive, corrective or contract maintenance (for example HVAC units), with the history of the unit and the task localised at the equipment. Readings outside their acceptable range are flagged as issues and documented with a photo.

3. Review on site. Architects and conservators view all findings of an inspection anchored on the real walls, confirm them in their structural context and add new ones during their visit.

4. Sensor installation. When an IoT sensor is installed, its position is recorded in the model, so its readings are later shown in the right place on the web and in VR.

5. Design and restoration overlay. Restorers and site managers overlay restoration phases or the BIM design on the real building to compare documentation, check deviations and coordinate work on site.


Left to right: Scanning a printed anchor; A marker on the wall; Placing an IoT sensor.



Video: Field inspection with the SINCERE MR app

A technician opens an inspection task, scans the anchor, marks wall damage on the spot and documents it with a photo.




Video: The ECOMONDO booth presentation on the MR/AR app



> Watch the full MR/AR user guide on youtube




Experimentation - SINCERE Pilots

Open Day Demonstrations to all SINCERE pilots are scheduled for March 2027





Under the hood

> Unity 6 with Unity's cross-platform mobile mixed reality framework and the XR Interaction Toolkit, for Android and iOS.

> Registration with printed image anchors and anchor groups calibrated on the platform; plane detection and SLAM-based tracking keep the model in place while the user walks.

> Models stream from the platform as compressed glTF, so building models stay usable on mobile devices; tasks, findings, photos and sensor positions sync through the platform GraphQL API with single sign-on.

> Known challenges that the project addresses: tracking drift in repetitive, thick-walled historic interiors; heavy BIM models on light devices; and users who are not BIM experts, which Field Mode is designed for.




Why it matters

For technicians and inspectors: clear instructions and one-tap documentation, without paper forms.

For managers and conservators: findings with exact location, photo and history, reviewed without a repeat site visit.

For heritage authorities: spatially referenced, auditable records of the condition of a monument over time.

Regulation the MR app supports: The EPBD (Art. 23) requires heating and air-conditioning systems above 70 kW to be inspected at least every five years, and every three years above 290 kW; the MR app guides the inspection step by step at the unit and records the result in its logbook.

Heritage monitoring: Condition surveys with located, time-stamped photos build the record of a monument over time, as encouraged by Commission Recommendation (EU) 2021/1970, and on-site checks against the model keep the information model current during operation (ISO 19650-3).



> Developed by up2metric within SINCERE WP3.