The Training & Simulation Journal has a short article about a project I've been working on lately: “PC-based JTAC sim will train Air Force airstrike controllers”.
In many ways, the JTAC [joint terminal attack controller] sim will resemble a first-person-shooter video game. The user will see a simulated battlefield. He’ll have an array of virtual tools, such as binoculars, a laser designator and Global Positioning System gear. It’s how the JTAC will direct aircraft that makes this sim different. The interface is almost totally voice-driven. The user directs the computer-controlled pilots using the same commands given to real pilots. For extra authenticity and convenience, users can plug the headsets from their real-life communications gear into their laptops.
The virtual pilot first checks in with the JTAC, who describes the target to the pilot. The computer pilot reads back the information, and then the JTAC uses commands to direct the simulated pilot’s eyes to the target. This requires a voice-recognition system clear enough to recognize the commands and artificial intelligence (AI) sophisticated enough to direct the virtual pilot in response to those commands.
I've primarily been working on the natural language dialog aspect of the system, the most challenging part of which might be where “the JTAC uses commands to direct the simulated pilot’s eyes to the target.”
The problem is that the pilot is flying high and fast and can see a huge amount of territory all at once, and the JTAC needs to be able to direct the pilot's vision to a very particular location. He typically does that by referencing landmarks within a visual context that is at least partially shared between the two, i.e. they're both looking at basically the same region but from very different perspectives, so the JTAC needs to choose landmarks that will be obvious from the air. And then there are many strategies for making use of those landmarks to guide the pilot's vision—e.g., defining a unit of measure based on the extent of some geographic feature and doing a kind of dead reckoning from an anchor point (“Go two units of measure to the west of the radio tower and then one north.”) or an iterative refinement (“In the northwest corner of the city there's a river, and there's a bridge going over the river, and there's a tank on the west side of the bridge.”).
It's an interesting challenge.
Posted by jjwiseman at January 05, 2007 09:44 PM