LVC Simulation for the Indian Army

​This blog post covers how emergent gaming tools and technologies have been used to make high-fidelity defence simulation, including LVC (Live, Virtual and Constructive) simulation which is the way ahead for most military simulators.
 
The LVC Simulation was done atop a high-fidelity terrain reconstruction which was mapped in-house via a Drone operation. The high-fidelity, one-to-one scale terrain already had about 30times the resolution available from Google earth, and was then made interactive and compliant with an LVC Simulation architecture where troops and tanks could train simultaneously. The concept was successfully demonstrated and highly appreciated, as part of CII & Indian Army’s Seminar on Simulation, conducted by the Army Design Bureau.
 
This is part 2 and concluding post on the entire exercise, and part 1 covers the aerial mapping of the terrain via a drone here. 

High fidelity synthetic terrain recreation

The attached graphic illustrates the entire pipeline of constructing the high-fidelity terrain for the simulation exercise. Major parts are further explained below.

1. Primary processing
   1. The first level of processing is to take the drone camera payload imagery and drone flight data. By syncing the clocks of both, we have accurate position and orientation of each imagery, and then based on SFM (structure from motion), we can recreate the terrain below (please refer to blog post 1)
   2. The output data which we receive then, is a height map (Digital elevation map), a triangulated mesh, and an orthomosaic texture.
   3. These, by themselves, need further processing to make them game/simulation ready.
2. Secondary processing
   1. Since game engines have their own terrain features, then either using the height map data from above, we can create the game engine’s terrain, or import that triangulated mesh as a substitute for ground, including LOD (level of detail support). An intermediate processing requires the ground to be separated from elements like vegetation, civil structures, or any element which grows/stands atop the terrain.
   2. Builtup and vegetation elements then need to be reconstructed atop the terrain – if the same is available in the library (which is increasing getting populated) of geo-typical assets like builtup structures of urban and rural localities of the area, and vegetation, which again is geo-typical of the terrain type.
3. Tertiary processing
   1. Finally, all the dynamics are added – example, collision bodies on terrain and superstructures, including wind/weather simulation.
   2. Lighting is setup to reflect the time of day and weather.
   3. Characters, and vehicles are added, including AI for automatic navigation and movement.
   4.  And finally, application and user specific elements like networking, graphical user interfaces etc are added.  

The image below, shows the synthetic terrain (virtual) constructed via the above pipeline.

LVC Architecture and its enablers

​LVC stands for Live, Virtual and Constructive Training.

• In Live training – real humans operate real systems. In Virtual training, real humans operate virtual systems, and in constructive, virtual operators drive virtual systems as almost complete AI. The mix-and match gives us the LVC system.
• Live training, is off course the best training for realism, but many scenarios, for example nuclear, chemical and biological warfare can’t be practiced live, and hence need to be simulated.  Live training is also costly, and has inherent risks to life and property in a military environment. Virtual training overcomes these aspects but can often lack proper realism. Finally, Constructive training is almost completely AI driven – where based on rules, the entities and needs to interface with live and virtual.
  • LVC training architecture thus requires various systems to be able integrate with each other and Compose and Operate together.

This is the major problem - This is a major problem as right now, how do various systems – say a human and a tank, operate together, especially when one may be live, and the other virtual, located hundreds of kilometres apart ? And that too in a cost-effective manner ? ​
 
This is where the rapidly transforming gaming technology comes in handy.

​Gaming technology is making rapid technological strides, with Virtual Reality and networked multiplayer support becoming very accessible. With a chosen terrain already mapped one a one-to-one scale in high-fidelity, it is possible then to simulate the virtual and constructive part on the same. Virtual vehicle simulators, including tank simulators are already being made, and constructive simulation via AI is also happening.  

​The challenge now shifts to integrate the live operator. The virtual participants need to see the updated position and movements of the live operator while the live operator needs to see updated virtual imagery of the virtual / constructed assets in his live imagery – superimposed on the real terrain.  The solution, is Augmented Reality. And more specifically, cost effective networked augmented reality kits, which will convert a human and his stores (weapons, assets) into AR systems showing virtual imagery over real field of view. Indian army has been constructed similar concepts and Threye Interactive has also developed its Threye Smart Augmented Smart Helmet, a networked portable AR device. 

​This completes all the hooks, which are needed to interface between live and virtual participants, and can then operate jointly over a mapped terrain, with the game engine acting as the glue which holds everything together. The below diagram illustrates the interfaces. 

Demonstration of an LVC Exercise

​An ounce of practice, is worth a ton in theory, and thus the above proposed scheme needed to be put to test, so we got on with it. A proposed setup was made, where geographically distributed participants from different systems could engage in a joint networked exercise. Here is how one such activity unfolded. 

1. Tasking: A Command and control cetre (Based in New Delhi) assigns objectives and details to participants.
2. The Virtual participants, a VR tank driver, from Meerut (75 KMs east of Delhi), advance over a synthetic terrain mapped from a real one.
3. The 'Real' participants, soldiers, with ATGM (Anti Tank Guided Missiles) get positioned in the real terrain (175 KMs West of Delhi), and through the modified scope of the ATGMs, see the virtual tanks, and aim to ambush them.
4. The Command and control centre monitors the exercise as it goes along, and introduces elements – e.g weather change conditions, or bring in other elements, like artillery fire, and also analyses and compares the performance of participants.

Reception and Conclusion.

​The above mentioned exercise was prepared and demonstrated live, at SEMSIM 2015, a joint symposium by Indian Army and CII, organised by the Army Design Bureau. Two participants, one hosting the networked synthetic environment drove the tank via a PC, where as a mockup of a ATGM, once again networked via a kit participated with the join exercise and was again able to ambush the tank. A detailed paper on the entire exercise, and as an extract from the SEMSIM compendium, is attached alongside for download. ​

On the whole, the entire exercise demonstrated that existing gaming tools and technologies, and the rapidly evolving mediums of AR and VR, are able to deliver the LVC training as required.

​The technology demonstrator proved the live, virtual and the constructive elements were able to perform as per their objectives, and most importantly, able to co-operate over a synthetic terrain. 

​​The complete activity, from mapping the terrain, to make conversion kits to integrate live and virtual counterparts, as well as the simulation exercise was done in-house. We’ve been able to provide

• An easy absorption roadmap,
• A robust underlying architecture, and
• A proven production pipeline, to create high-fidelity and immersive training simulation systems for the Indian Army,

And thus, it is a great pleasure when our efforts are found well suited and appreciated, and kindly acknowledged by Design Bureau of the Indian Army.