Value and money are often equated in daily language, but the value of a currency is derived entirely from the collective beliefs of many people, as reflected through their spending patterns. Even the items on the “dollar menu” of your favorite fast food restaurant are rarely a dollar for long. So what is the value of a dollar really? As we explore adaptable AI, could a deeper understanding of how humans determine value inform systems that better align with human-like decision-making?
As autonomous systems become more advanced, we must consider the role of human values in decision making and evaluate if this value system is matched by our technology. Imagine an autonomous emergency medical drone that has been developed to assist in the rescue of survivors after a building collapse. Human rescue teams may risk becoming trapped themselves, while remotely piloted drones can face connectivity issues. Yet, typical deep learning algorithms often fall short when matching the nuanced human valuations that guide decisions, such as prioritizing one patient’s needs over another based on urgency. Humans make judgments rooted in fundamental values, not solely data-driven associations.
The human brain constantly re-evaluates and updates values based on new information, dynamically weighing options. The PVLV model from Dr. Randall O’Reilly’s Emergent framework illustrates this process. Standing for Primary Values (PVs) and Learned Values (LVs), PVLV represents how the brain differentiates between innately valuable stimuli and those that gain value through learned association using biologically plausible learning algorithms from the Emergent framework. Bodily states trigger signals in the brain that highlight the urgency of various PVs—like hunger, thirst, or even the well-being of a loved one. The brain then estimates the effort required to meet these needs, using LVs as references. The difference between calculated value and effort is reflected in our motivations and decisions.
Unlike traditional reinforcement learning systems, the PVLV model can weigh multiple competing needs and determine the most urgent. Separate regions process the receipt of PV versus LV stimuli and handle discrete forms of learning. This division of labor guarantees that the motivational system doesn’t lose sight of the PVs, so to speak, and that LVs can be learned, extinguished, transfer contexts, and support other, more complicated classical conditioning phenomena. It learns abstract goals—such as “money leads to food”—which it uses to gauge progress, generating internal reward signals when it advances toward these goals. The model ensures that current goals are maintained unless stronger evidence prompts a shift. For instance, a medical drone might realize that another patient’s condition has worsened and decide whether to complete treatment for the current patient or shift attention to the more critical one. Such decisions are handled with a flexibility akin to human judgment, rather than by a rigid heuristic algorithm.
As the research of autonomous AI agents advances, we should prefer them to share our human values and motivations. Referring back to the autonomous emergency drone, the system could be designed with primary values (PVs) for multiple vital tasks such as delivering life-saving equipment and searching unstable terrain for survivors. Meanwhile, it would autonomously develop learned values (LVs) through various real and simulated experiences that allow it to generate better judgment calls—such as recognizing the most efficient route in dangerous terrain or determining the likelihood of a specific medical intervention being successful under various conditions. The PVLV model would allow the drone to continuously update its decisions, weighing the urgency of helping a critical patient against its capacity to reach others or return to base to recharge for future missions. In military or disaster scenarios, such a system could save lives by not only completing missions efficiently but adapting to unforeseen challenges in ways current autonomous systems struggle with. Most importantly, engineers could construct the agent’s primary value system to steer its decision-making, better aligning with human priorities.
Ultimately, the question of value has far-reaching implications. As autonomous systems take on an increasingly prominent role in our lives, it becomes essential to reconsider how we define value and ensure that our technology is aligned with a shared, thoughtful set of fundamental human values.