Skip to main content
SearchLoginLogin or Signup

RESOURCE: virtual reality for ISRU

Capability Analysis Exercise

Published onFeb 24, 2021
RESOURCE: virtual reality for ISRU

Assignment #1: Capability Analysis Exercise

Topic: Enhancing real-time decision-making capabilities for lunar polar volatile exploration missions with a virtual reality mission simulation system (Artemis III Science Finding 6.3.7-1, Recommendation 6.3.7-1b)

Scientific Rationale:

As is stated in the Artemis III Science Findings: “In situ instrumentation will be greatly beneficial in addressing a number of Artemis III science investigations, including instrumentation to support sampling [and] volatile monitoring”. The recommendations specify the need for real-time transmission of data from the science instruments which will allow the science support teams to provide real-time feedback to the crew, and processed data which can help to convert raw data into tactical decision-making. While the final goal will be to provide these capabilities for human exploration missions, pre-cursor rover missions will serve to establish the necessary bandwidth communication capabilities and establish protocols for use of the different tools. The topic suggested here will be implemented using a virtual reality platform capable of providing decision-making visualizations of instrument data of lunar polar volatiles.

In order to support this topic, specific questions related to both the on-board instrument data, current technical capabilities of both a lunar rover and VR hardware, as well as concept of operations will need to be addressed. These will include:

  • What are the data types and sizes expected for the on-board instruments? How frequently will this data be transmitted? What is the required refresh rate that will impact decision-making?

  • How is the data currently displayed and used to make decisions for traverse planning? Are changes made to traverses based on this data in real-time? Who needs to understand this data (stakeholders)?

  • What are the limitations on VR hardware use (i.e. is there a time limit)? What are the VR hardware capability requirements (memory, graphical resolution, standalone vs. desktop, etc.)?

  • How do the concept of operations impact the platform needs – would this be useful for the entire duration of the mission or only for specific use cases (traverses, waypoints, science stations, drill locations, etc.)

  • What are measurement criteria for assessing improvements to decision making capabilities? How can this be done (analog field mission, in-lab)?

The future of space exploration requires a paradigm shift. Mission complexity is increasing and with the advent of the Lunar Gateway, frequency can expect to gain momentum as well. In order to ensure we are achieving the most science possible within these missions, human-computer interaction needs to take a front-seat. By treating machines as collaboration tools, we can improve cross-discipline communication, improve real-time decision-making processes, reduce task loads and provide flexibility in both temporal and spatial planning. Volatiles prospecting missions in particular will stand to benefit given the specificity of the knowledge required to make decisions around geological data. Providing naturalistic visualization tools in which multiple team members can analyse, discuss and interpret real-time data, can dramatically improve the scientific return on both prospecting missions, and later human exploration missions.

Virtual reality (VR) has become synonymous with video games and entertainment primarily due to both the dramatic increase in power and resolution of commercially available systems as well as the significant decrease in price of these systems. Despite the common perception of these systems, their use in industry and academic contexts for research, design and training have demonstrated the broad applicability of the platforms [1]. In particular, VR has proven useful for storytelling, abstract data visualisation and multi-modal communication across disciplines; three key components of lunar rover missions. Among the industries which have begun to use VR in practice is the aerospace industry for the design decision making process, including Boeing and Lockheed Martin Space Systems. NASA has been using VR systems for astronaut training at the Virtual Reality Training Lab (VRL) at the NASA Johnson Space Center for decades [2]. Despite the use of VR at NASA, proving new technology is one of the main challenges for this project. Due to the complex and high-risk nature of space missions, minimal risk can be taken for technological implementation. Another key challenge will be assessing the value of the technology. Thus, precursor analog missions will be needed. These will test the VR platform at various levels of development and for increasingly complex scenarios.

Technology Analysis:

The software development will be a part of the resource exploration and science of our cosmic environment (RESOURCE) project [3]. The primary mission for which RESOURCE will provide the technology to close the knowledge gap is the Volatiles Investigating Polar Exploration Rover (VIPER) mission. There are also two analog missions which can feed into this development by providing ground data and mission-critical design requirements; the Mojave Volatiles Prospecting (MVP) mission [4] and the Biological Analog Science Associated with Lava Terrains (BASALT) mission [5]. We will focus on the two primary onboard instruments common to all three missions. These are the near-infrared and visible spectrometry system (NIRVSS) and the neutron spectrometry system (NSS), both of which are used to assess volatiles on the lunar surface, and for NSS, up to 1 m below the surface. NIRVSS also has camera capabilities and can provide black and white, multi-colour spectral and panorama imagery of the surface below the rover. The data collected on the analog missions is representative of the data types, sizes and expected results of what will be collected on the lunar mission. For both MVP and BASALT data was collected using the Exploration Ground Data System (xGDS) which collects the raw data and translates it to normalized graphical displays over time and geospatial displays on a map. MVP and BASALT were used to assess the rover capabilities, the onboard instruments and the concepts of operations of real-time traverse monitoring for flexible mission execution. VIPER will require all of the functionality of the MVP and BASALT missions but with the added challenge of bandwidth restrictions and minimal situational awareness. While the instruments and rover need to withstand the harsh lunar environment, we will focus instead on the design of the VR system to have robust data transmission capabilities and a flexible platform design to enable tailoring to specific decision checkpoints and on-demand scenarios.

Technology Readiness Level Assessment:

Because a preliminary VR mission simulation system has already been developed for RESOURCE, the TRL of this platform is at a TRL 2 according the NASA TRL definitions (technology concept and/or application formulated). We would aim to have the platform and associated instrument visualizations at a TRL 5: Component and/or breadboard validation in relevant environment or TRL 6: System/subsystem model or prototype demonstration in a relevant environment. We would aim to prove the VR platform on a demonstration rover in a simple field test. The difference between 5 and 6 being the capabilities achievable in VR with the analog data we have from previous experiments. If additional data is needed to advance the VR platform from validation level to prototype level, this would limit us to TRL 5.


[1] L. P. Berg and J. M. Vance, “Industry use of virtual reality in product design and manufacturing: a survey,” Virtual Real., vol. 21, no. 1, pp. 1–17, 2017, doi: 10.1007/s10055-016-0293-9.[2] A. D. Garcia, J. Schlueter, and E. Paddock, “Training astronauts using hardware-in-the-loop simulations and virtual reality,” AIAA Scitech 2020 Forum, vol. 1 PartF, no. January, pp. 1–13, 2020, doi: 10.2514/6.2020-0167.[3] J. L. Heldmann, M. C. Deans, and D. Newman, “Resource Exploration and Science of OUR Cosmic Environmnet,” 2017.[4] J. L. Heldmann et al., “Lunar polar rover science operations: Lessons learned and mission architecture implications derived from the Mojave Volatiles Prospector (MVP) terrestrial field campaign.,” Adv. Sp. Res., vol. 58, no. 4, pp. 545–559, 2016.[5] D. S. S. Lim et al., “The BASALT Research Program: Designing and Developing Mission Elements in Support of Human Scientific Exploration of Mars,” Astrobiology, vol. 19, no. 3, Mar. 2019.

No comments here
Why not start the discussion?