GoodAI

Badger architecture is the unifying framework for our research defined by its key principle modular life-long learning.

The modular aspect is expressed in the architecture through a network of identical agents. The life-long learning means that the network will be capable of adapting to a growing (open-ended) range of new and unseen tasks while being able to reuse knowledge acquired in previous tasks. The algorithm that is run by individual Badger modules (a.k.a. experts) will be discovered through meta-learning.

We expect the design principles of Badger architecture to be its key advantages. The modular approach should enable scaling beyond what is possible for a monolithic system and the focus on life-long learning will allow for incremental, piece-wise learning, driving down the demand for training data.

Below you can find a taster of some of our latest work.

Badger is an architecture and a learning procedure where:

• **An agent **is made up of many experts
• All experts share the same communication policy (expert policy), but have different internal memory states
• There are two levels of learning, an inner loop (with a communication stage) and an outer loop
• Inner loop – Agent’s behavior and adaptation emerges as a result of experts communicating between each other. Experts send messages (of any complexity) to each other and update their internal memories/states based on observations/messages and their internal state from the previous time-step. Expert policy is fixed and does not change during the inner loop.
• Inner loop loss need not even be a proper loss function. It can be any kind of structured feedback so long as it eventually relates to the outer loop performance.
• Outer loop – An expert policy is discovered over generations of agents, ensuring that strategies that find solutions to problems in diverse environments can quickly emerge in the inner loop.
• Agent’s objective is to adapt fast to novel tasks
• Open-ended inner loop learning needs to be enabled by a suitable design of the outer loop, for instance through the support of agent self-reference and by using curiosity as an implicit agent goal creation mechanism. An open-ended agent should be able to come up with novel and creative solutions to problems it faces. The environment it operates in needs to be open-ended too – it must enable creation of novel and unforeseen tasks that match the current skill level of the agent, to support its further improvement.

• Roles of experts and connectivity among them assigned dynamically at inference time
• Learned communication protocol with context-dependent messages of varied complexity
• Generalizes to different numbers and types of inputs/outputs
• Can be trained to handle variations in architecture during both training and testing

The animation above shows 10 layers of a cellular automaton evolving over time. The task is to copy the pattern from the middle to one of the dots – not both. Thus, the distributed agent has to learn a coordination strategy to determine which dot will be active. The second animation demonstrates a “transform” task, where the pattern has to be modified during the copying process. Additionally, regularization in the form of cell activation zeroing is present. The policies unfolding above have been found by SGD.

The aim of this project is to develop a new game - code name AI Game - where freeform dialogs between player and game characters take a major role.

In the game, players interact with game characters controlled by large-language models, where the language models simulate the behavior of people in both responses and actions, displaying complex personalities with goals, emotions, and memories.

Large language models are neural networks trained on internet-scale text corpora with one objective: to predict the next word. During gameplay, the game feeds the game character’s personality (described by designers in plain text), observations, recent events, and dialogues to a large language model, which then predicts what this person in this situation would do, think, and say (emulating their personality). This output is then translated to game actions. The current system is hybrid, relying on large language models, but also on traditional planner technology. In the future, we expect that the large language models will do planning too. This new approach to AI-driven characters leads to **non-scripted behavior, complex and richer personalities, and more emergent gameplay.