papers
Amazon Alexa Prize Competition Chatbot - EVE
Structured Knowledge Graph Traversal and Conversational Scaffolding
EVE is a socialbot architecture that combines the strengths of hand-crafted rules, deep learning, and structured knowledge graph traversal. She uses conversational scaffolding as a response strategy, and bases her conversational flow on thousands of conversations she has analyzed for structure. Her self model (as well as her perception of the user) are part of her knowledge graph, enabling her to discern points of common interest while simultaneously identifying areas of ambiguity or cognitive dissonance.
read the paper
The Skyrim Paper
A Semantically-Informed Algorithm For Extracting Interaction Modes
We took images from the video game Skyrim, ran them through image captioning systems (Clarifai vs. CaptionBot vs. a human-generated caption), and then used skip-thought vectors to categorize those generated captions into four distinct interaction modes ('threat', 'barter', 'explore', and 'puzzle'). This would allow an autonomous agent in a 3D world to take appropriate actions in given situations.
read the paper
Directing agent behavior
Using Natural Language
Humans approach games with some high-level knowledge of what needs to be accomplished and how those things can be done. We present a method for an agent's actions to be guided by several natural language phrases using the implicit semantic information in embedding spaces. By using short commands chosen for each particular game, we can prime the agent for success by directing his action selection using simple distance comparisons in high-dimensional space.
read the paper
What Can You Do With a Rock?
Performing affordance extraction via word embeddings
Autonomous agents in text-based environments must often detect affordances (see James Gibson's work): the set of behaviors, enabled by the situation. We present a method for affordance extraction via word embeddings trained on a part-of-speech tagged Wikipedia corpus. The resulting word vectors are treated as a common knowledge database which can be queried using linear algebra. We apply this method to a reinforcement learning agent in a text-only environment and show that affordance-based action selection improves performance in most cases. Our method increases the computational complexity of each learning step but significantly reduces the total number of steps needed. In addition, the agent's action selections start to resemble those a human would choose.
read the paper
Boeing's Multi-User CAD Software
Model Consistency and Conflict Resolution
Simultaneous multi-user computer aided design (CAD) allows multiple designers to contribute to the same model at the same time. The resulting parallel design workflow shortens product development cycles. In a replicated, simultaneous multi-user CAD system, modeling data must be kept consistent between clients. We present a method that keeps independent copies of the models in sync between distributed CAD clients. This is accomplished by enforcing modeling operations to occur in the same order on all the clients. In case of conflict, a resolution method preserves conflicting operations locally for later reuse or resolution by the user. These methods are implemented in a commercial CAD system which has been enhanced to enable simultaneous multi-user. Validation tests are run to demonstrate that the methods implemented ensure model consistency and resolve conflicts while preserving conflicting operation data.
read the paper