Mitchelville XR:
Bringing History Back to Life
Fall 2019 - Fall 2020
Mitchelville XR: Bringing History Back to Life is a project lead by professor Christopher Maraffi who received a National Endowment for the Humanities (NEH) Digital project for the public "Discovery" grant and a Walter & Lalita Janke Emerging Technologies Fund seed grant to work on the early design and development of the XR immersive and interactive historical tour around the cultural Mitchelville site on Hilton Head Island SC, which was the first Freedman's town in the US during the Civil War, and a Fullah-Geechee heritage site today.
This project helps to create a pipeline to teach students how to start, and create a digital humanities project using the latest technologies in performance capture, real-time rendering, and XR tools from concept idea, to technology proof of concept, and production.
Early research and results of this project by MFA students James Jean Pierre and Ledis Molina
We used a historical photography taken by Henry P Moore in 1862 of Mitchelville inhabitants and houses to create a 3D simulation of the town in the Unity game engine.
Mitchelville game engine simulation of 1862 historical photograph (Library of Congress), Unity, Oculus Quest by Ledis Molina. 3D models by Ledis Molina and James Jean-Pierre, Spring 2020.
The characters are created based on historical reference found online, museums, books, and every other source that can provide accurate and reliable information about the person that we are going to recreate using a 3D software. On each character we have to focus on the clothing, style, materials, etc., to be appropriate to their time. This adds more realism and lets the player immerse themselves more into a real historical experience. We are still in the prototyping phase of the project, we are going to continue developing more technology research for extended reality projects, characters, environments, and the overall performance to provide the best experience for this digital humanities project.
Homebound: The Interactive Immigrant Experience
Spring 2020
Homebound is a project lead by the graduate students Alberto Alvarez and Brandon Martinez who received the Walter & Lalita Janke Emerging Technologies Fund seed grant, to work on the final version of the prototype experience at Florida Atlantic University. This project was completed as part of their master thesis research on an interactive immigrant experience using Oculus Rift S and Unreal Engine 4.
For this project, I served as the motion capture artist for the characters "Maria" and "Sicario" using the Rokoko Smartsuit Pro and Apple face-capture system for a complete performance capture. This research experience helped me understand the pipeline used in Unreal Engine to develop characters with facial blendShapes that are optimized for performance capture as well as the level of impact that full body animation can have on immersive storytelling projects. This experience as well as other side research on motion capture has helped me develop an efficient pipeline for an online live motion capture session for students to understand the process, and benefits to apply it towards their projects using Real-Time applications.
Virtual Production
Fall 2019
On the Mitchelville XR project as part of the first proof of concept, funded graduate students Alberto Alvarez, Brandon Martinez and Ledis Molina along with other Graduate and Undergraduate students of the School of Communication and Multimedia Studies have developed a demo using the emerging technology of Virtual Production in Unreal Engine 4. The video below provides an introduction to Virtual Production and some render test scenes of a work in progress of the digital humanities project, Mitchelville XR.