Javier Alba-Tercedor1, Pierre Buffin2, Jeremy Robert2, Christina Wise2, Philippe Lepine2
1. Department of Zoology. Faculty of Sciences. University of Granada. Campus de Fuentenueva, 18071-Granada. Spain. email@example.com
2. BUF Canada. 3575 Boulevard Saint-Laurent. Montreal, QC H2X2T6 Canada, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com,
Javier Alba-Tercedor, received his MSc in 1977 and PhD in zoology in 1981 from the University of Granada, Spain. He is now a full professor of zoology (since 1989) at the university and responsible for a research group on the biology and ecology of lotic freshwater ecosystems, with over 200 publications. His interests have been mainly focused on freshwater ecology, especially on aquatic macroinvertebrates and the biomonitoring of running water. Now he is an active enthusiast of the potential of X-ray tomography, in both research and education. Some of his creative micro-CT videos can be visualised at: youtube.com/user/albatercedor
. More details in: mendeley.com/profiles/javier-alba-tercedor/
Micro-CT and the industry of visual effects have collaborated to create animated morphs of real beetles for the film Blade Runner 2049 (directed by Denis Villeneuve and released in October 2017). It went on to gain the Oscar for Best Visual Effects in 2018.
In this paper we have summarized and illustrated all the necessary process, from the micro-CT scan, and final volume rendered images of beetles to the animation process developed and performed by the BUF Company
The director of Blade Runner 2049, Denis Villeneuve; the Visual Effects Supervisor, John Nelson; the Visual Effects Producer Karen Murphy; and Alcon Entertainment. Dr. Eduard Vives (University of Barcelona), a well-known coleopteran specialist of the family Cerambycidae, who identified our specimens; and to Dr. Javier Pérez López (Parque de las Ciencias, Granada) who lend us specimens of Zophobas morio. To Bruker-microCT (formerly Skyscan) staff for fast and effective support, their patience and effectiveness, and for their constant improvements to the software and in implementing new options we requested.
Also for their kindness in providing the senior author fast and effective suggestions and answers to queries. In this respect, we are especially indebted to: Alexander Sasov, Stephan Boons, Xuan Liu, Vladimir Kharitonov, and Phil Salmon. Dr. Nikolaus Nestle from BASF, SE, Ludwigshafen am Rhein in Germany let to senior author to know about the utility for micro-CT of the Basotec® material, providing us with the first sample of it, and Luisa Kari and Stephanie Thum from the BASF Company who kindly sent us additional material.
This paper benefitted from subaward agreement S15192.01 between Kansas State University (KSU) and the University of Granada, as a part of the USDA-NIFA Award 2014-70016-23028 to Susan J. Brown (KSU), Developing an Infrastructure and Product Test Pipeline to Deliver Novel Therapies for Citrus Greening Disease (2015–2020).
The collaboration with the Buff Company to show the possibilities of Micro-CT as a valuable base tool for the industry of visual effects was presented at the Bruker Micro-CT User Meeting, April 16–19, 2018, Ghent, Belgium.
Prof. Dr. Javier Alba-Tercedor,
Departamento de Zoología
Facultad de Ciencias
Universidad de Granada
During the 2018 Oscar awards ceremony in Hollywood Blade Runner 2049, directed by Denis Villeneuve, and released in October 20175 received the award for Best Visual Effects.
In one scene some beetle morphs can be seen. The purpose of those particular visual effects of the beetle morphs was to illustrate how much control the character Dr. Ana Stelline has over the images she creates for the artificial memories that are implanted into “replicants”. She uses a hand-held device that can quickly morph images to her desired level of expectation. In fact, she is the best in her field at memory creation.
BUF’s technical team was requested to create a computer-generated (CG) accurate model of a beetle which then morphs into a second separate beetle in a natural way. Due to Dr. Stelline’s skill level, the morphs needed to happen in a fast and fluid manner. For that purpose, BUF had to recreate extremely detailed and photo-realistic models, with micrometric details, such as microscopic hair or the detail of the facets of their eyes, of real beetle specimens. BUF contacted the senior author at the University of Granada in Spain to perform and reconstruct images of high-resolution micro-CT scans of different beetle species.
Figure 1. Schematic explanation of the micro-CT procedure starting with the mounting of the samples and finishing with the final volume rendering reconstruction image.
MATERIALS AND METHODS
Different pinned beetles species from entomological collections (Figs.2: a, b,c) were used for a micro-CT study (two exotic Cerambycidae, known as longhorned beetle species, living in Sumatra and Java: Cerosterna pollinosa Buquet, 1859; and Anoplophora amoena, Jordan, 1850) and a common Tenebrionidae, known as a darkling beetle species: Zophobas morio Fabricius, 1776, used as live food for rearing and maintaining different animals species in captivity.
Figure 2. Pinned beetle used for scanning, as the base for film animations Cerosterna pollinosa (a,e), Anoplophora amoena (b, d) and Zophobas morio (c). Beetles mounted to be scanned inside a plastic container and fixed with Basotect® foam to avoid any movement during the scans (d, e)”
Before scanning, the pin was removed from the specimens. Samples were mounted in a piece of Basotect® (a melamine resin foam, created by BASF), inside a plastic container (figs. 2: d, e). The Basotect’s material has very low density, and therefore is very transparent to an X-ray. It can be easily eliminated during the segmentation procedure, as was reported in a previous paper2.
A SkyScan 1172 high-resolution microtomograph, upgraded to have a Hamamatsu 100/250 source and a SHT 11Mp camera was used. The scanning parameters were setup as it follows: a) for C. pollinosa, in Figs. 3 a, b, e and Fig. 5a, and for A. amoena in Fig. 4c (Isotropic voxel size = 13.54µm per pixel; source voltage=59KV, source current=167µA, image rotation step=0.5º, 360º of rotation scan, and no filter. Resulting four connected scans); b) for Z. morio, in Figs. 3 c, d and 4a, b, c left, e and f (Isotropic voxel size = 7.72 µm per pixel; source voltage=59KV, source current=131µA, image. Rotation step=0.5º, 360º of rotation scan, and no filter. Resulting three connected scans).
Figure 3. CTVox’s volume renderings and reconstructions of the beetles animated by BUF for the film Blade Runner 2049 directed by Denis Villeneuve. Cerosterna pollinosa: dorsal (a) and ventral (b) views, and detail of the head (e). Zophobas morio: dorsal (c) and ventral (d) views.
For primary reconstructions and the “cleaning” process to obtain the datasets of “slices” (Fig. 4a) the up-to-date last versions of the Bruker micro-CT’s Skyscan software (NRecon, DataViewer, CTAnalyser) were used. Volume renderings of Figs. 3e and 4a were obtained with Amira’s software v.6.3.03. The free Skyscan’s software CTVox was used for volume rendering reconstructions images in Fig. 3a-d. A schematic explanation of the procedure is shown in Fig. 1. For a more detailed explanation of the process see the previous paper1.
Figure 4. Summary of the animation process: reconstructed image from the cross-section datasets used to recreate the model (a); each slice is placed in a 3D space (using particles, then voxels converted to mesh), a very precise model is created (b); the models are then shaded and texturized (c); a virtual skeleton is built for animation purposes (d); CG beetle is placed next to the real beetle for comparison of animation, behavior, movement (e); at last, the animated model is placed in context (f); and the final work where one beetle transforms into another (g). This visual effect work was done by BUF Company (buf.com)
BUF Creative Visual Effects work
(see summary in Fig. 4)
To facilitate the position of the insects, tracking markers on the practical leaf (the leaf on set) were placed. Several HDRIs (High Dynamic Range Images) were shot to help the integration of digital insects. Dead and living insects were used as a reference. Many photographs were taken, first on the film set, for questions of scale, light and texture. Then several more photographs were taken from different angles in order to reconstruct the insects digitally. Video reference was also shot on set to record how insects move. In fact they were very demanding actors!
To define the look of the transition between the two insects, BUF did a lot of 3D visual research and a multitude of tests. These tests were introduced to the VFX Supervisor John Nelson and Director Denis Villeneuve to define the desired transition effect. The shots of the insects in the film ranged from wide to macro.
The data obtained from the scans resulted in a sequence of images close to the level of an ultrasound image (i.e. Fig. 4a). Approximately 3,000 images were mapped onto plates, sampled with particles, converted into voxels and then converted into a 3D model. The 3D model obtained with this process, after voxelization, was converted to mesh (Fig. 4b), and then shaded and texturized (Fig. 4d). BUF’s “neuro” proprietary tool, based on “B.L.A.S.T” language, also developed internally at BUF4, allowed us to achieve an extremely precise reconstruction using simple black and white images.
After sectioning the model into several pieces the construction of a skeleton for animation was started and each part of the beetle was reworked separately. Simple objects were attached to the virtual skeleton permitting the artist to quickly manipulate and animate the beetles (Fig. 4d).
Mapping and shading work based on the reference photographs taken during the shoot then followed: animation was reapplied on the high resolution model and lit with a virtual light setup (Fig. 4e). Finally, the animated model was placed in context (Fig. 4f). The transition between insect morphs (Fig. 4g) had gone through several design phases. First, a 3D morph of each part of the insects migrating from one bug to the other, then different types of wipes, ranging from simple scanning to more complex temporal transitions. The final idea of the transition from one beetle to another is based on light projection, wherein different “spotlight” flashes make the holographic matter appear or disappear.
Figure 5. Amira’s volume rendering of a frontal detailed view of the beetle Cerosterna pollinosa (a), and the final visual effect work done by BUF Company using BUF proprietary software: bufsoftware.com/products/bsuite/ (b).
RESULTS AND CONCLUSION
The micro-CT results are shown in Figs: 3, 4a, b and Fig.5a. The visual effects results are summarised in Figs 4 and final results are in Figs 4g and 5b. Also they can be viewed in the film Blade Runner 2049 when the character K meets Dr. Ana Stelline for the first time in her laboratory. It is clear that micro-CT represents a very accurate and valuable tool, to help to reconstruct, and to produce CG animations of small living animals with microscopic structural details like the beetle insect species used for this work.
To our knowledge, this study represents the first symbiosis between the micro-CT technique and that of the visual effects industry. In short, between academic-science and the entertainment industry. Many hours of labor went into the creation of the beetles, which represent barely twenty seconds of screen time in the final film. However, the accuracy of the final effect is well worth the efforts of both the senior author and BUF company.