Digital Restoration as a Digital Twin Process
This resource is based on the presentation by Georgios Papaioannou (Athens University of Economics and Business) on digital restoration, virtual anastylosis and Heritage Digital Twins developed for the ARTEMIS Summer School 2026 in Hof, Germany. It builds on the concepts introduced in the session and uses examples from cultural heritage restoration to explore how 3D digitization, computational reassembly and human expertise can support the analysis, reconstruction and interpretation of fragmented objects.
Introduction
Digital restoration brings together cultural heritage expertise, 3D digitization, and computational methods to support the documentation, analysis, and reconstruction of cultural heritage objects and sites. Within this context, a Heritage Digital Twin can provide a digital environment for working with 3D data, annotations, measurements, and different restoration hypotheses.
This resource focuses in particular on virtual anastylosis and reassembly: the computational reconstruction of fragmented or disassembled objects. It introduces the main challenges of this process, from uncertain fragment configurations and large search spaces to the different geometric and material properties of heritage objects. It also explores how 3D data is prepared for computational restoration, how fragments can be aligned and assembled, and how missing geometry can be predicted.
A central theme is the relationship between computational methods and human expertise. Rather than replacing the restoration expert, interactive approaches allow computational processes and expert decisions to work together. The resource also considers how services such as ARTEMIS can support this workflow and how web-based, accessible, and collaborative environments can make digital restoration more practical for the cultural heritage community.
Learning Objectives
By the end of the resource, learners will be able to:
- explain the role of Heritage Digital Twins in digital restoration, including how they support documentation, analysis, and the exploration of reconstruction hypotheses;
- describe the main challenges and data requirements of virtual anastylosis and computational reassembly, including fragment uncertainty, large search spaces, and the preparation of suitable 3D data;
- distinguish between manual, automatic, and interactive restoration approaches, and explain how human expertise contributes to “user-in-the-loop” workflows;
- understand key computational concepts and collaborative challenges in digital restoration, including geometric alignment, shape prediction, prosthetic geometry, accessibility, and web-based collaboration.
Target Audience
The resource is aimed at students, researchers, and professionals in cultural heritage, archaeology, conservation, restoration, and digital humanities, as well as learners interested in 3D and computational methods. No advanced technical background is required.
Intended Impact
The resource aims to provide a clear understanding of how digital and computational methods can support restoration. It highlights the importance of task-oriented digitization, human–computer collaboration, and accessible digital tools, while showing how Digital Twins can support the exploration, analysis, and reconstruction of cultural heritage objects.
Requirements
No prior technical knowledge is required. A basic familiarity with archaeological objects, cultural heritage restoration, or digital documentation is helpful but not essential.
Suggested Learning Path
For learners who are new to digital restoration and Heritage Digital Twins, it is recommended that they first watch the video resource Reconstruction of Archaeological Objects as a Digital Twin. The video introduces how the physical restoration and reassembly of archaeological objects can be transformed into a digital environment where computational methods can be used to explore and test restoration hypotheses in a noninvasive way. The present text resource builds on these concepts by providing a broader introduction to digital restoration, virtual anastylosis, 3D data, computational reassembly, and the role of human expertise.
Heritage Digital Twins in Restoration
Heritage Digital Twins often emerge from restoration projects of considerable complexity. Within such projects, a Digital Twin can support project monitoring and asset management while also bringing together visualization, measurements, spatial annotations, and metadata. These functions can, in turn, contribute to decision support.
Renderings of Notre-Dame Cathedral from Iconem provide one example. Another important example is the n-Dame_Heritage Advanced ERC Grant program at CNRS (2021–2027), led by Livio De Luca. Here, collaborative documentation of media and 3D assets, 3D annotation, and the integration of photographic and 3D data illustrate how different forms of information can be brought together within a digital heritage environment.
Simulation in Heritage Digital Twins
Simulation adds another dimension to the Digital Twin. In cultural heritage, teams use computer simulations to construct plausible “what if” scenarios. These simulations help them estimate problem parameters, predict the future state of an object or site, assess the impact of an intervention, and explore hypotheses.
Reliability of Simulations
The reliability of such simulations depends first on the quality and completeness of the input. How accurately are the input data modeled, and how completely have the parameters affecting the process been captured? It also depends on the simulation model itself: whether the relevant control parameters have been accounted for and how well the actual process has been mapped to an algorithm.
Potential Simulations for HDTs
Several types of simulation can be relevant for Heritage Digital Twins, depending on the object, site, or conservation question at hand. Mechanical simulations, for instance, can help researchers and conservation teams assess the structural stability of archaeological and architectural sites, including their response to seismic activity. Chemical and erosion simulations may be used to estimate how materials deteriorate over time, while radiative and heat-transfer simulations are useful for studying environmental conditions in museums, exhibitions, and outdoor heritage settings.
Other simulation approaches address more specific tasks. Digital anastylosis supports the reconstruction of fragmented monuments or objects by helping to identify possible original arrangements. Fluid and particle simulations can contribute to disaster preparedness and risk assessment, for example, by modeling floods, fire dynamics, or debris accumulation. In addition, specialized simulations may be used to analyze visitor movement and crowd flow, particularly in complex or heavily visited heritage spaces.
Mechanical Simulations
Civil engineers commonly use mechanical simulation, and heritage teams can also apply it to heritage sites. Commercial FEM solvers are available, established, and validated, and BIM integration pipelines already exist. At the same time, stress and chemical processes often alter heritage-site materials over time. These changes can make simulations less reliable when they rely on assumptions about material behavior.
Chemical Simulations
Chemical simulation presents a distinct set of challenges. The processes involved are often highly material-specific, making it difficult to generalize them into a broadly applicable simulator. At the same time, their temporal progression is usually too slow to observe and validate in vivo, even under accelerated aging conditions. In addition, detailed spectrographic data may be required to model and verify such processes accurately. Within the context discussed here, however, such approaches had not yet been successfully applied to real cultural heritage objects. The PRESIOUS project is an example of research in predictive digitization, restoration, and degradation assessment of cultural heritage objects.

Laboratory experiments for assessing material erosion and degradation under acid rain and freeze–thaw conditions, including the monitoring of environmental parameters.
Image 1: Erosion Measurement Experiments – Acid Rain Effects. Source: PRESIOUS Project, Norwegian University of Science and Technology (NTNU), Image Gallery.
Image 2: Erosion Measurement Experiments – Freeze-Thaw-Rain Effects. Source: PRESIOUS Project, Norwegian University of Science and Technology (NTNU), Image Gallery.
Radiative/Heat Transfer
Radiative and heat-transfer simulation is a comparatively mature computational field. It has been extensively studied in computer graphics and finite element modeling, and both open-source and commercial tools are available. In many cases, the required input data are also relatively well understood and can be obtained from freely available sources.
Nevertheless, some limitations remain. Dedicated software solutions for cultural heritage applications are still relatively rare, and reliable simulations depend on high-quality input data, such as photometric measurements and accurate material properties. Related work in this area includes the LOTUS and GLIDE projects, while ARTEMIS aims to provide lighting simulation as an open service.

Digital 3D environments for lighting simulation and optimization, illustrating the integration of 3D models, virtual visualization, and illumination analysis. Sources: LOTUS (2022–2024) and GLIDE (2014–2015).
Image 2 & 3: Vitsas, N., Evangelou, I., Papaioannou, G., & Gkaravelis, A. (2023). Opening design using Bayesian optimization. Virtual Reality & Intelligent Hardware, 5(6), 550–564. https://doi.org/10.1016/j.vrih.2023.06.001
Computed Digital Anastylosis
Computed Digital Anastylosis deals with the virtual restoration of fragmentary and disassembled material from objects and archaeological sites. It is a well-studied area in computer graphics and an important restoration task. A minimum requirement is partial geometry scans. At the same time, there are relatively few niche open dedicated software solutions and computational approaches vary considerably. ARTEMIS plans to provide part-alignment simulation as an open service.

3D digitization and virtual reconstruction of fragmented cultural heritage objects for documentation, restoration, and degradation assessment. Source: PRESIOUS project (2013–2016).
Fluid and Particle Simulations
Fluid and particle simulations provide another application area for Heritage Digital Twins. They are driven by technologies developed in the CGI and visual-effects industry, with established high-quality commercial simulators, GPU-accelerated solutions, and open alternatives available. At the same time, fire simulation remains challenging because of its unpredictability and is not necessarily intended to reproduce fully realistic behavior.
The Virtual Reassembly and Anastylosis Problem
At an abstract level, reassembly is a 2D or 3D puzzle. In practice, however, many of the elements that would normally define a puzzle are uncertain. The target objects may not be obvious, the number of missing pieces may be unknown, and even the total number of fragments may be uncertain. The shape and number of contact sides are not fixed, and in some cases, the final shape itself is not well defined.
Constraints and shape priors can reduce this uncertainty. Useful information includes planarity, axial symmetry, textural and geometric landmarks such as tool marks, bracket holes, and patterns, as well as the overall expected shape. Even with such constraints, the search space can grow explosively. Uncertainty and the dispersion of fragmentary material can therefore become significant obstacles for restoration work.
Types of Material
The suitability of virtual reconstruction varies with the material. Rigid materials include potsherds, masonry and statuary blocks, as well as other thin fragments such as bone, metallic sherds, and blades. These materials may be broken, intact but disassembled, or both. Deformable materials include textiles, wood, hardened leather, ivory, papyri, and metallic foils.
Materials such as papyrus, paper, cloth, and metallic sheets can be particularly difficult because they may bend substantially. Reconstruction may require “unwrapping,” which introduces significant human or computational error. Not every deformation is recoverable, especially where unknown plastic effects are involved, and specialized digitization such as CT may be required. Chemical damage or fusion, for example, through burning, creates further complications.
Materials that are normally considered hard or rigid can also deform over time as a result of mechanical stress or chemical alteration. Objects made of wood, bone, or solid metal may nevertheless retain recognizable landmarks, even after such changes have occurred. In these cases, a combination of rule-based or manual pre-deformation and nonrigid alignment can provide an effective strategy for matching and reconstruction. The Vesuvius Challenge is a relevant example in this context, as it involves the digital analysis and virtual unwrapping of deformed ancient scrolls.
Why Go Digital?
Digital restoration changes what can be manipulated and tested. A virtual environment allows conservators to work with material that is difficult to handle or physically inaccessible. The virtual setting also allows restoration hypotheses to be explored without necessarily being constrained by the physical behavior of the material. Most importantly, digital restoration is nondestructive: assemblies can be undone, and even extreme hypotheses can be tested without changing the physical object.
This also makes it possible to retain several hypotheses and versions rather than committing to a single reconstruction. Digital methods can process and compare artifacts at a level of detail that is difficult to attain manually, and they allow observations to be considered at larger scales. Researchers can perform precise measurements, results can be visualized, and virtual access can be provided to assemblies that would be impossible or very difficult to inspect physically.
The digital environment also supports communication and collaboration. Teams can share research results, work together in virtual workspaces, and use digital representations for presentation and education. They can access remote resources, including assets owned by third parties.
A Digital Restoration Pipeline
A computational restoration Digital Twin can be organized as a pipeline beginning with 3D digitization. The resulting data then undergo geometry processing, followed by multi-part assembly and, where appropriate, object completion. The process connects physical objects with digitized geometry such as point clouds, meshes, and volume data, simplified geometry, features, metadata, and segmentation information. Once fragments have been assembled, the resulting geometry can support measurements and renderings, and augmented geometry can potentially be returned to physical space, for example through 3D printing.
Digital Surrogates
A digital surrogate represents the actual object as faithfully as possible in digital form. From this surrogate, teams can derive more theoretical representations for specific purposes. Digital surrogates support preservation through re-mediation and enable access, information processing and analysis.
For a restoration Digital Twin, digital data are a means to an end. The relevant information therefore needs to be sampled and retained according to the purpose of the analysis. In this computational context, human perception factors are not the criterion for deciding what information is necessary.
Digital 3D Asset Requirements
Different versions of a 3D asset can serve different purposes. One version may be suitable for documentation and physical reproduction, another for geometry analysis and high-fidelity visualization, and another for fast preview and web-based presentation.
Any digitized 3D model can potentially be used. Meshes are preferred, while point clouds can also be used. 3D Gaussian splats are not considered suitable here because they are volumetric data. Researchers may use volumetric data, but these data often create problems.
Very high resolution is not necessarily beneficial. Medium-quality models can be sufficient, particularly because many fragments are already deteriorated. Additional points may simply introduce noise, smooth out relevant surface features and make computations unnecessarily difficult. Surfaces can therefore be sub-sampled so that detectable relief variations are retained. Simplification methods commonly collapse low-curvature polygon edges while preserving geometric detail. MeshLab is a freely available tool for this purpose.
Completeness is likewise not always required. A 3D model does not need to be complete or watertight if it sufficiently represents the critical surfaces, including contacts, volume-defining corners and salient features.
Surface Classification
Automatic reassembly depends on distinguishing useful correspondences from trivial ones. Not all surface regions are equally important because typical solvers minimize distances based on geometry, texture, saliency or related criteria. Fragmentary material can easily produce misleading correspondences, whereas silhouettes and landmarks are particularly important for intact parts.
One strategy is to identify surfaces that are potentially fractured and to exclude trivially smooth surfaces, which are typical failure cases. The classification can use conservative thresholds because false-positive fractures are not considered a major problem: feature-rich intact surfaces will not align with corresponding fractured regions. ARTEMIS alignment services can perform the associated preprocessing behind the scenes.
Manual vs Automatic Anastylosis
Manual anastylosis begins with a virtual workspace in which fragments can be manipulated directly. This makes experimentation possible, but manual work can be imprecise and time-consuming.
Automatic clustering can produce precise recommendations, but the number of possible combinations creates a very large search space. Manual or automatic constraints therefore become important. The computational process can also be slow, and in a fully offline workflow experts mainly verify or reject the recommendations rather than continuously directing the computation.
The offline workflow consists of facet extraction, classification, pairwise alignment and multi-part reassembly. It aims to avoid trivial matches and reduce the search space. For each facet pair, error can be estimated and a pose computed, followed by global optimization, fragment clustering and alignment. Preemptive pair-wise alignment and pre-segmentation can further reduce the computational burden, although global registration remains necessary and expensive.
What About the Human Expert?
A more interactive approach places the human expert directly within the computational process. Restoration becomes user-driven: the user provides constraints, manipulates fragments, inspects recommendations and performs integrity checks while computation continues progressively.
The computer contributes pairwise matching, combinatorial optimization and bundle adjustment, while the user remains involved in the decisions that shape the reconstruction. In the interactive workflow, facet extraction and classification are followed by pair-wise alignment and incremental multi-part reassembly. For each facet pair, error can be estimated and a pose computed, after which greedy optimization, fragment clustering and alignment can take place.
Users can request pairwise alignment when needed. Users can combine local registration with step-by-step interaction. The goal is therefore not to remove the expert from the process, but to combine human experimentation and judgment with computational assistance.
Shape Prediction and Prosthesis
Once fragments have been assembled, missing or complementary parts can be generated from shape priors. The expected overall shape of the object can provide one source of information, while self-similarities such as symmetries and repetitive patterns provide others. Additional priors include axial and planar symmetry, contour continuity, statistical similarity, similar objects, partial-object retrieval and non-rigid fitting.
Symmetry provides the most reliable source of geometric information. For cylindrical objects, shape can be computed through revolution after detecting an axis; for non-cylindrical objects, self-registration can be used.
Prosthetic geometry can then be derived by subtracting the assembled fragments from the predicted shape. In this context, volume subtraction reliably produces geometry suitable for 3D printing. In this way, the Digital Twin cycle can return from digital reconstruction to physical space.

Digital reassembly and virtual reconstruction of fragmented cultural heritage elements, illustrating the alignment of individual fragments and the reconstruction of missing sections.

ARTEMIS Geometric Alignment Services
ARTEMIS provides geometric alignment services for 3D models representing partial objects, complementary scans, fragments, or components that need to be assembled. These services support both local and global registration and include hierarchical clustering and bundle adjustment to refine geometry that has already been approximately aligned. Optional tether points or anchors can also be used, with adjustable influence on the alignment process. To support efficient processing, the server automatically simplifies and preprocesses the uploaded models. Server-side caching and synchronization further enable collaborative workflows.
The services are exposed through REST API calls and are primarily intended for application developers. They can be integrated into web-based, mobile, or desktop applications. The ARTEMIS infrastructure includes a dedicated server environment hosted at the Athens University of Economics and Business (AUEB), a Node.js application server, and native C++/CUDA backend servers. Authentication and Digital Twin transactions remain centralized through the main ARTEMIS infrastructure.
A Typical Workflow
A typical session begins by initializing a project and adding Digital Twin parts such as F1 and F2. Once the Digital Twin session has been initialized, the user can manipulate the parts in the client application. Their poses are updated, and an alignment of F1 and F2 is requested. The client then polls for the alignment result and can group the two parts.
The same cycle can be extended by adding F3. The user again interacts with the parts, updates the poses of the existing group and F3, and requests an alignment of F1–F2 and F3. After the server returns the result, the client can group the three parts, and a bundle adjustment can refine the assembled geometry. The workflow therefore alternates between user interaction and on-demand computational services.
Parts may already reside on the server when they are reused, and additional services can be called for tasks such as mesh simplification and annotation. In collaborative work, synchronization calls can lock transformations while tasks performed by other users are still pending.
What Is Next? Priorities for Accessible Digital Restoration
The next step is to make the digital restoration pipeline more accessible and to remove significant bottlenecks. A central priority is a user-centric approach in which cultural heritage experts remain at the forefront while unnecessary computational complexity is kept out of the user experience. Focused, intuitive, and usable tools are preferred over all-encompassing solutions that are difficult to use and maintain.
This direction includes a transition from desktop applications to the web, greater use of cloud-based infrastructure, access across different platforms, including mobile devices, open access, and extensibility for specialized needs. Paid services and applications are explicitly regarded as unsuitable for the cultural heritage community because they can create financial, institutional, and technical barriers for smaller museums, research projects, conservation teams, and heritage organizations.
Digitization as a Bottleneck
Digitization is a major bottleneck because computational restoration requires digital data. System and interface complexity can add another barrier. The appropriate response is not necessarily to digitize everything at maximum resolution. The purpose of digitization needs to be considered in relation to the task.
For artifact documentation, detailed 3D digitization can be high-resolution, high-precision, and complete, but this approach is slow. Restoration can instead work with approximate 3D digitization using variable resolution, moderate precision, and partial geometry. The aim is then to capture restoration-specific geometric details, prioritize important surfaces such as fractured or characteristic regions, predict or omit missing geometry, capture multiple fragments simultaneously, and automate cleanup.
Partial geometry is therefore acceptable for digital restoration tasks. The specific digitization technology is less important than how scanning is performed and processed. Affordable and accessible approaches such as structure-from-motion (SfM) photogrammetry can be useful when combined with appropriate post-processing, including automatic segmentation of multiple fragments and automatic cleanup.
The Role of AI
Deep learning has proven useful for specific tasks, but its role in geometric object restoration is described as overrated. The reasons include the need for large training collections, which are not generally available for cultural heritage objects, the risk of overfitting to specific categories, and the computational resources required.
Procedural methods are described as still vastly outperforming deep-learning approaches in this context. At the same time, successful methods from computer vision may contribute to restoration through object-based retrieval, automatic image annotation, automatic shape completion using geometric transformers, and graph-based combinatorial optimization for puzzling.
Collaboration
Restoration requires collective work, and the Digital Twin needs to support the same kind of collaboration. A digital restoration workspace should allow several experts to work independently and jointly in real time, share and comment on results, represent multiple interpretations and versions, and share both the experience and the data.
The web provides a suitable environment for this kind of shared workspace. At the same time, digital restoration does not need to reproduce every physical action literally. The physical experience can be transferred more subtly, with a virtual metaphor for a physical task being sufficient.
Conclusion
Digital restoration combines digital representation with computational analysis, simulation, virtual reassembly, and collaboration. The Digital Twin provides a basis for working with cultural heritage objects beyond what is possible through physical manipulation alone. It can support the exploration of hypotheses, the comparison of possible assemblies, and the generation of complementary geometry.
The quality of the result depends not simply on how much data is collected, but on whether the digital representation contains the information required for the restoration task. Critical surfaces, contacts, and features may matter more than complete, high-resolution geometry. This makes task-specific digitization and preprocessing central parts of the workflow.
At the same time, computational restoration benefits from keeping the expert involved. Interactive, user-driven approaches combine human constraints and experimentation with pairwise matching, optimization, alignment, and refinement. The broader direction is toward accessible, web-based, and collaborative tools in which the computational complexity supports the work without becoming the focus of the user’s experience.


