AI is learning to anticipate our choices, from battlefield decisions to mortgages, spending and relationships. Alex Karp’s warning raises a question of power: when machines understand our habits and vulnerabilities, will they serve our interests or subtly steer us towards someone else’s objectives?
The AI-assisted raid on South Korea's banks never went near the vault. It slipped in through loan portals, sales platforms and a staff mobile app. Now Seoul is ordering every bank to count its exposed doors, and regulators in Washington, London, Frankfurt and Sydney are asking the same question.
A bipartisan AI committee wants independent evaluators inside OpenAI and Anthropic and early access for security agencies. Read slowly, its letter to Albanese is a ransom note written in reverse. Australia is not being held hostage. It is offering itself, with its creative archive as collateral.
AI is learning to anticipate our choices, from battlefield decisions to mortgages, spending and relationships. Alex Karp’s warning raises a question of power: when machines understand our habits and vulnerabilities, will they serve our interests or subtly steer us towards someone else’s objectives?
The most consequential question in Alex Karp’s warning about artificial intelligence is who gets to act on what it predicts. Reporting on the Palantir chief’s interview with Louis Robson describes systems that could eventually anticipate people’s thoughts as well as their behaviour. That remains a claim about where the technology might lead.
Predicting a decision is not reading a mind. Nevertheless, governments, businesses and households have reason to examine what happens when increasingly intimate knowledge of a person becomes available to someone else.
Research offers a glimpse of that possibility. A Stanford-led study used two-hour interviews to build AI simulations of 1,052 people. On a major social survey, those simulations reproduced participants’ responses 85 per cent as accurately as the participants reproduced their own answers a fortnight later. The result does not establish that machines can reliably forecast a person’s life. It does demonstrate how a relatively brief conversation can support a detailed model of someone’s attitudes. Behavioural intelligence is becoming more sophisticated, drawing meaning from what people disclose alongside the digital traces they leave.
On the battlefield, the stakes are already substantial. NATO’s acquisition of Palantir’s Maven Smart System places AI within operational decision-making, where combining information can help identify patterns, assess threats and prioritise action. Speed has military value, but it also compresses the time available to question a conclusion. A pattern may suggest hostile intent without establishing it.
The danger arises when the apparent precision of a machine’s assessment obscures the uncertainty beneath it, turning an inference into grounds for action.
In civilian markets, specialist AI agents could bring similar predictive methods to spending, investment, mortgages, housing and personal relationships. An assistant with access to financial records and private conversations might recognise an emerging need before its user articulates it.
Google’s opt-in Personal Intelligence already connects information across services including Gmail and Photos. Meta has announced that AI interactions will inform content and advertising recommendations, subject to regional and account conditions. These developments extend the commercial model built around search and social media into the conversations through which people work out what they want.
There is no established, all-seeing AI cloud observing everyone. The more credible concern is that connected services could assemble increasingly detailed accounts of people’s circumstances, vulnerabilities and ambitions.
Research has demonstrated that language models can infer personal attributes from ordinary text. Separate experimental work found personalised GPT-4 more persuasive than human opponents in controlled debates. Neither finding establishes unlimited predictive or persuasive power. Together, they raise a commercial question: when an agent recommends a loan, a purchase or a course of action, whose objective determines the advice?
The cultural consequences deserve equal attention. UCL-led research found sharply increasing exposure to misogynistic material on simulated TikTok accounts, illustrating how recommendation systems can concentrate a narrow worldview. Conversational agents could make such reinforcement more personal, responding to loneliness, resentment or financial anxiety with advice that sustains dependence. They could also help people challenge assumptions and make better decisions.
The distinction rests on design, incentives and accountability. Commercial interests should be disclosed, consequential decisions remain contestable, and cultural differences respected. An assistant entrusted with a person’s private life should answer to that person’s interests. Otherwise, the power to anticipate our choices may become the power to quietly arrange them.
Get the stories that matter to you.
Subscribe to Cyber News Centre and update your preferences to follow our Daily 4min Cyber Update, Innovative AI Startups, The AI Diplomat series, or the main Cyber News Centre newsletter — featuring in-depth analysis on major cyber incidents, tech breakthroughs, global policy, and AI developments.
A bipartisan AI committee wants independent evaluators inside OpenAI and Anthropic and early access for security agencies. Read slowly, its letter to Albanese is a ransom note written in reverse. Australia is not being held hostage. It is offering itself, with its creative archive as collateral.
Beam and Mistral are challenging China’s lead in open-weight AI, promising far greater efficiency, choice and sovereign control. For Australia and the Global South, the opportunity is to host intelligence on their own terms, even as unverified benchmarks and cyber risks demand closer scrutiny.
Washington renamed AI super intelligence, and six tech leaders signed a voluntary accord with no enforcer. In the same week OpenAI shelved a model over safety, AMD bought World Labs and US grids strained. Atoms for Peace in 1953 led to a global agency. Super intelligence is still waiting for one.
The scientist whose ImageNet dataset launched Nvidia's AI empire has sold her company to its chief challenger. AMD's $8.2bn all-stock bid for Fei-Fei Li's World Labs is a bet on world models, inference and the race up the AI stack, where chipmakers now buy the labs that define tomorrow's workloads.
Where cybersecurity meets innovation, the CNC team delivers AI and tech breakthroughs for our digital future. We analyze incidents, data, and insights to keep you informed, secure, and ahead. Sign up for free!
