Dual-system learning model “figures out” how to use a tool

This is an 8 year passion project on attempting to create a control system for a purely autonomous virtual agent.

I wanted to put a model together that could fully control an agent with typical human drives (hunger, play/exploration, control). The full model is comprised of interconnected simple neural network modules. The application is written in C# and implemented in Unity.

The model uses Reward-modulated Hebbian learning in modules associated with value processing (e.g. amygdala, ventral striatum), and Contrastive Hebbian learning in all other modules.

The design is influenced by selected published research on the prefrontal cortex and basal ganglia in executive function/decision-making. But the main inspiration was the following article:

O’Reilly, R. C. (2010). The What and How of prefrontal cortical organization. Trends in Neurosciences

I’d love any feedback!