The Vatican, a bastion of tradition, has embraced cutting-edge technology in a groundbreaking collaboration. Earlier this month, they unveiled an AI-powered digital twin of St. Peter’s Basilica, a 400-year-old architectural marvel. This immersive virtual experience, created in just one month through a partnership with Microsoft and Iconem, a Paris-based digital preservation company, will allow millions worldwide to explore the Basilica’s intricate details. The project leveraged drones, 3D cameras, and lasers to capture over 400,000 high-resolution images, securely stored on Microsoft’s Azure Cloud. Microsoft’s AI tools then refined these images into a millimeter-accurate 3D representation, simultaneously identifying structural vulnerabilities like cracks and missing mosaic tiles – a feat impossible through traditional methods.
The creation of digital twins, virtual representations of physical entities, has been ongoing for two decades. Techniques like LiDAR scanning, 3D mapping, and photogrammetry have been utilized to create static models of everything from manufacturing plants to ancient temples. However, the integration of AI marks a significant leap forward. Heritage Building Information Modelling (H-BIM) enhances these twins, providing incredibly precise 3D models for informed decision-making. For example, the Aga Khan Trust for Culture (AKTCI) and Tata Trusts partnered with the Archaeological Survey of India (ASI) to restore Delhi’s Humayun Tomb using similar techniques, demonstrating the power of these tools in large-scale restoration projects. The Indian Ministry of Tourism even collaborated with Google’s ‘Incredible India’ initiative to create virtual walkthroughs of famous historical sites. But these earlier iterations often lacked the dynamic capabilities of AI-enhanced twins.
The integration of AI transforms digital twins from static representations into powerful tools for proactive conservation. These AI-powered twins are capable of detailed documentation, continuous monitoring, and predictive analysis. By processing data through computer-aided design (CAD) and geographical information systems (GIS), and integrating internet of things (IoT) data from sensors, they can simulate various scenarios to predict vulnerabilities and guide restoration efforts. The ancient Roman city of Pompeii, digitally captured multiple times in the past, now boasts an AI-powered twin (part of the EU-funded RePAIR project) that uses machine learning to reconstruct broken artifacts by simulating missing pieces based on historical patterns and material analysis. Similarly, UNESCO is leveraging AI-powered digital twins to restore the war-damaged Mosul Al-Nuri Mosque in Iraq, utilizing drone and satellite data to create precise reconstructions and assess structural stability, even simulating the mosque’s original acoustic properties and sunlight patterns. Even the Notre-Dame Cathedral, following the devastating 2019 fire, has benefited from an AI-powered twin to guide its intricate restoration.
This innovation extends beyond the realm of ancient history and religious sites. The Voice of Art project, a partnership between IBM, Ogilvy, and Pinacoteca (Sao Paulo’s oldest museum), employs natural language processing (NLP), cognitive computing, and machine learning (ML) to create personalized visitor experiences, allowing interaction with specific artwork. Closer to home, the Virtual Experiential Museum Ajanta (VEMA) showcases Ajanta’s UNESCO-listed cave paintings using digital technology, while Vizara Technologies, funded by India’s Department of Science and Technology, has created 3D digital models of numerous Indian heritage sites, including the Taj Mahal, Kashi Vishwanath Temple, and the ruins of Hampi. The AI-powered enhancements of these models allow for the simulation of weathering effects, facilitating targeted restoration efforts and predictive maintenance. For instance, the Konark Sun Temple’s digital twin simulates the damage caused by salt-laden winds and rainfall, while in Hampi, AI analyzes the impact of foot traffic and vibrations from modern activities, aiding in the preservation of these fragile monuments. Even the tomb of Mohammed Quli Qutb Shah in Hyderabad has received the digital twin treatment, showcasing the global reach of this technology.
Recent research from Lindenwood University demonstrates that advances in 3D AI generative content and photogrammetric scanning are democratizing the creation of digital twins, empowering non-specialists to contribute to historical research. However, significant challenges remain. The European Union highlights concerns about data protection and copyright infringement, hindering widespread adoption by museums and cultural heritage institutions. A lack of clarity regarding commercial opportunities further dampens enthusiasm. The Lindenwood University researchers also point out the lack of standardized technologies for international collaboration, with diverse software and hardware hindering interoperability and reproducibility. The discontinuation of HoloLens 2, for example, creates uncertainty for AR projects, and VR development faces similar fragmentation. The absence of global standards for methodology and archival longevity, coupled with a reliance on specialized technical expertise, presents significant hurdles to cohesive collaboration and sustainable development in this rapidly evolving field. The immense potential of AI-powered digital twins in heritage preservation demands that these challenges be proactively addressed to ensure its widespread and effective implementation.
