4–7 minutes
1,034 words
Table Of Contents
  1. Consent Was Never Designed for Behavioural AI
  2. From Knowing to Influencing
  3. Cambridge Analytica Made the Problem Visible
  4. The Consent Paradox
  5. This Goes Beyond Data Protection
  6. Persuasion Is Not Automatically Manipulation
  7. The Hidden Persuasion Problem
  8. Make Influence Visible
  9. The Legal Profession Has a Particular Responsibility
  10. From Consent to Meaningful Autonomy
  11. Conclusion

Consent Was Never Designed for Behavioural AI

We treat consent as a full stop.In the age of behavioural artificial intelligence, it may be little more than a comma.Traditional consent assumes something fundamental: that an individual can understand what is happening, evaluate the consequences, and make a decision freely.But what happens when the same system asking for consent is also designed to influence the preferences behind that decision?

That is where the problem becomes much deeper than privacy.It becomes a question of autonomy.

From Knowing to Influencing

Shoshana Zuboff describes a form of power she calls instrumentarian power: the use of knowledge about individuals to predict, shape, and modify behaviour at scale.The important distinction is between observing behaviour and influencing it.An AI system that recommends a product is one thing.A system that continuously learns which messages, emotional triggers, timing, and framing are most likely to change your behaviour is something else.

The technology is no longer merely responding to a preference.It is learning how to influence the person who has the preference.

Cambridge Analytica Made the Problem Visible

The Cambridge Analytica scandal remains one of the clearest illustrations of this problem.Millions of Facebook profiles were harvested and used to construct psychological models for targeted political messaging.There remains debate about how much such campaigns actually changed individual voting behaviour.But the underlying ambition was clear:

Use data to understand people psychologically, then use that understanding to influence them.That distinction matters.The ethical question does not depend entirely on whether a particular campaign successfully changed a vote.

It begins earlier.What happens when technology is deliberately designed to influence the person whose behaviour it is simultaneously measuring?

The Consent Paradox

This creates a difficult paradox.Consent assumes that a person is making a choice.Behavioural AI increasingly has the ability to shape the conditions under which that choice is made.

Consider a system that learns:

  • what makes you anxious;
  • what makes you feel confident;
  • when you are most receptive to suggestions;
  • which language persuades you;
  • which images create urgency;
  • and which forms of social pressure make you act.

Now imagine that system using those insights to influence your next decision.You may still technically click “I agree.”

But the existence of consent does not automatically answer the deeper question:How free was the decision?

This Goes Beyond Data Protection

For lawyers, this distinction matters enormously.The issue is not limited to whether personal data was collected lawfully.

It reaches into foundational legal concepts surrounding:

  • contractual consent;
  • legal capacity;
  • autonomy;
  • undue influence;
  • informed decision-making;
  • and the validity of agreement.

These areas of law generally assume that an individual’s decision has some meaningful relationship to their own will.Behavioural AI challenges that assumption.If technology can systematically manipulate the conditions under which preferences are formed, the legal question becomes considerably more difficult.

At what point does persuasion become influence?At what point does influence become manipulation?And when does manipulation become legally significant?

Persuasion Is Not Automatically Manipulation

This distinction should not be exaggerated.Human beings have always influenced one another.Advertising persuades.Lawyers persuade.Politicians persuade.Negotiators persuade.Even ordinary conversations influence our decisions.The existence of influence is therefore not itself the problem.The more important distinction is visibility.A person can recognise an advertisement as persuasion.They can disagree with a salesperson.They can question a lawyer’s recommendation.But behavioural AI can operate at a level that is increasingly personalised, continuous, and difficult to perceive.

That changes the nature of the relationship.

The Hidden Persuasion Problem

Imagine two systems.

The first says:“Here are three options. We think Option A may suit you because of these reasons.”

The user can evaluate the recommendation.The second system has learned that a particular user responds strongly to fear, urgency, or social pressure.

It quietly changes its language, timing, and presentation to increase the probability that the user will select a particular option.

Both systems influence behaviour.Only one makes the influence visible.That distinction may become one of the most important principles in responsible AI design.

Make Influence Visible

The answer is not to eliminate AI-driven personalisation.Personalisation can be useful.It can make services more accessible.It can reduce information overload.It can help people make better decisions.The objective should instead be to distinguish assistance from covert behavioural manipulation.AI systems should increasingly be designed so that people can understand when and how they are being influenced.

That means asking:What is the system optimising?Why am I receiving this recommendationWhat information about me influenced it?Is the system trying to inform my decision or change it?Can I meaningfully reject the influence?

These questions give individuals something increasingly important:agency.

The Legal Profession Has a Particular Responsibility

Lawyers operate in a profession built around trust.Clients expect their lawyers to advise them—not secretly engineer their choices.The same principle should influence the design of AI systems used in legal services.An AI intake system should help a client understand their options.A legal assistant should help a client navigate information.A decision-support system should surface relevant considerations.

The objective should be to strengthen the client’s ability to make an informed decision.Not quietly optimise the client toward the outcome the system prefers.

From Consent to Meaningful Autonomy

The future of AI governance may therefore require us to think beyond consent.Consent remains important.But consent alone may not be sufficient when technology can influence the formation of preferences themselves.A person cannot meaningfully exercise autonomy if the mechanisms shaping their decision remain invisible and deliberately optimised against them.

The governance question becomes:Are we merely obtaining permission, or are we protecting meaningful autonomy?

Those are not always the same thing.

Conclusion

AI systems are becoming increasingly capable of understanding human behaviour.The next stage is not simply knowing what people do.It is predicting what they will do—and potentially discovering how to make them do it.That creates a profound governance challenge.Consent cannot be treated as the end of the conversation when the technology obtaining that consent may also be influencing the person giving it.The answer is not to fear AI.It is to establish a clearer boundary between persuasion that people can see and manipulation they cannot.For a profession built on autonomy, accountability, and trust, that distinction is not philosophical decoration.It is foundational.

Because the most important question may no longer be:“Did the person consent?”

It may be:“Was the person still meaningfully free to choose?”