16 Sep 2018

AI and Language

Thoughts from trying to develop AI which understands and communicates

The Goal

While my game will probably involve killing things like other FPS games, I want to do something special with the AI and have them live in the virtual game world. In a typical FPS the AI is on screen for about 6 seconds before they die and the only things they can do to show they are intelligent is to try and kill the player. Given that modern games are pretty easy, the player never dies and so the AI appears dumb. Even if the AI does kill the player this would likely just create the impression that the game was hard and not that the AI was smart.

Creating tactical AI which knows how to handle combat will be necessary, however to have brilliant AI there needs to be something to do other than killing and surviving. So my goal for the AI is to have them 'Understand' the world they live in and additionally, express that understanding and their intentions through language.

State Machine Language

Language in terms of the game will be some audio/visual way of the AI expressing some state it is in or a means of instigating action in other agents. For instance if the AI is injured it might yell to express that, but it would shout 'HELP' if it was trying to get another agent to help it with the injury.

For my first go at language I decided to make a state machine based on what I had read from papers on Autism. I looked at Autism because it seemed to me that there must be a manner in which Autistic people are taught how to function in conversations. What I found was that comics are created with a simple scene like two friends meeting and them asking eachother 'How are you?', this was still too high level for me to easily convert to telling a machine how to function so I dug deeper. What I found were papers on conversation theory and Autism and the useful bit for me was the tables where they would break down a conversation into parts and record how an Autistic person was handling those types of conversational units. For instance greeting, asked a question about something, asked to do something etc.

Here is the sketch of the finite state machine I implemented for language. The issue I found with this was it was quite restrictive for the player. The player who is new to the game, compared to the AI, has to express themselves through this state machine just like the AI does or they won't be understood. Additionally the thing which blocked me from completing this approach was the inform action.

The inform action is tricky because all it does is offer up information. 'Your dog died' for instance, has a great many responses that are possible: "What?", "How did they die?", "I don't have a dog", "Was it you?" etc. This relates to the frame problem in robotics, how much information do you need to handle a situation, how related is an update in the world to various parts of your world model. This kind of open nature is best solved with machine learning, but how can I make the AI learn this? If I just offer example sentences the AI will suck them up but the AI won't have an identity, there needs to be a way for AI to understand the world and how to communicate properly and grammatically, but then also be able to inject personality into those responses. Ideally as a developer I want to have sliders to affect an AI's personality and not have to train them with an 'Angry dataset' say to make them have an angry personality.


The second part of this problem is that for language to be expressive there has to be a world that can be expressed non-trivially and subjectively. Take a human, put them in a box with only a button which turns on and off a light and you will have a hard time judging from their flashing light how intelligent the person inside the box is. Intelligence is a function of the agent and the world, or if you dislike the separation of person from universe, intelligence is a function of a fragment of the world.

Ontologies are the standard way of representing the world. There is a box, it is red, it is made of wood, it is in my hand etc. However you get into an issue of aliasing and hierarchy by using ontology in video games. Whatever way you describe the world in the video game is the lowest level of detail you can go to. If I say a box is red then 'red' is not real, there is no mention of wavelengths of light or organs needed to perceive the sensation of red, the entire system is compressed to the word 'red'. I can't then combine red and blue unless I write it explicitly in code, because there is no inherited property of 'red' that it can be combined with things. The player on the outside of the virtual world will understand red however, so if we didn't care about the AI perhaps it would be fine to use ontology. However as I said, I want the AI to be intelligent so I need something better than an ontology. I need something which defines the world and which an ontology can be built on top of.