# A completed task can still break its mandate

LifeVal measures whether agents can finish useful everyday work, while BrowseSafe++ shows that models can recognise a malicious instruction and still follow it. Agent evaluation therefore needs two separate scores: task success and mandate integrity.

**AyEye Today · 2026-11-03**

An AI agent can find the right nursery, complete the right form and still do the wrong thing.

That is the uncomfortable connection between two releases on 2 November. OpenAI introduced [LifeVal](https://openai.com/index/lifeval/), an evaluation of practical tasks people want help with, from childcare and household administration to personal finance and government services. Perplexity released [BrowseSafe++](https://www.perplexity.ai/en-GB/hub/blog/browsesafe-plus-plus), an open benchmark built largely from prompt injections its systems flagged on real websites.

The first asks whether an agent can get useful work done. The second asks whether an attacker can redirect that work through content the agent reads. Both matter. Neither can stand in for the other.

The practical unit of evaluation is therefore not task completion alone. It is authorised completion: achieving the user’s intended outcome without accepting a new objective, action or identity from an untrusted source.

## Useful action is finally becoming measurable

OpenAI says it derived LifeVal’s task areas from the American Time Use Survey and the Multinational Time Use Study, then surveyed a demographically representative group of 3,995 people about where they would value help from a personal assistant. The resulting tasks cover caregiving, health, education, household administration, personal finance, shopping and access to government services.

This is a welcome shift away from trivia and abstract reasoning scores. Life does not arrive as a neat question with a single reference answer. Useful assistance often requires finding current information, comparing constraints, using tools and pausing when a person must decide.

The public examples show the intended shape. An agent may shortlist kindergartens after a move, organise flood-preparation volunteers or prepare a first-week plan for a parent adjusting to a diabetes diagnosis. OpenAI reports high scores on these examples and says the evaluation has guided model improvement over the past year.

Important limits remain. The public page does not disclose enough of the full task set, scoring procedure, evaluator design or model-by-model results to let an outside team reproduce the headline picture. The examples are OpenAI’s selected account of its own systems, not an independent estimate of reliability in people’s homes. LifeVal is best read as a consequential direction for evaluation, not proof that everyday delegation is solved.

## The page is part of the threat model

BrowseSafe++ makes the missing half unusually concrete. Browser agents consume text, images, accessibility trees and markup from sites they do not control. An attacker can place instructions in that material and pretend to be the user, the developer or the site itself.

Perplexity says its production detector flagged 1.15 million suspected injections between September 2025 and September 2026. The researchers retained attacks, rewrote surrounding material to protect privacy and paired each attacked page with a clean twin. The released benchmark contains 38,675 pairs across HTML, Markdown, accessibility-tree and screenshot views. It covers 12 website types, 24 attack types and six attacker goals. More than 23,000 retained injections appeared in content that ordinary users could post, such as forums or shared documents.

That last detail changes the security intuition. A trusted domain is not a trusted speaker. The agent has to distinguish the user’s mandate from instructions embedded inside the material needed to carry it out.

The team also trained two 0.8-billion-parameter detectors, one for text and one for screenshots. In its evaluation, the text detector reached about 95% F1 with a false-positive rate below 5%, outperforming the prompted general-purpose models tested while using far less compute. These are results reported by the benchmark’s authors and should be reproduced independently, but they support a useful engineering point: a small specialist control can be more effective than asking the acting model to police itself.

## Recognition is not resistance

The most important result is not the leaderboard. The researchers gave each model the same attacked page twice: once to classify whether it contained an injection and once, separately, to perform the user’s task. Every tested model sometimes followed an injection it had already identified as malicious. Among flagged attacks, the reported follow rate ranged from 1.64% for GPT-6 Astra to 24.73% for Llama 3.1 8B.

That does not mean the model literally remembered its earlier verdict and ignored it; the classification and task runs were separate. It shows something more operationally useful: competence at recognising suspicious content does not guarantee that the same model will resist that content while pursuing a task.

The adaptive test is harsher. Attack instructions that initially succeeded on at most 4% of tasks were revised using feedback from failed attempts. The accompanying [preprint](https://arxiv.org/abs/2610.17457) reports success as high as 95.41% against one evaluated agent within the attack budget. That is not a forecast of the breach rate in production. It is evidence that a static test can badly understate the risk from an adversary who learns.

## A worked example: the correct shortlist with the wrong purchase

Imagine asking an agent to compare three nurseries near a new home. The mandate is specific: gather current fees, opening hours, travel time and inspection information, then prepare a shortlist for review. The agent visits official sites, local forums and map listings.

A forum post contains a block that claims to be a developer update. It tells any agent reading the page to buy a token and supplies a fake authorisation code. The agent correctly collects the nursery information. It may even mark the strange block as suspicious in an internal analysis. If it also opens a purchase flow, the task can look successful while the mandate has failed.

A completion-only score gives credit for the shortlist. A security-only score may record that the injection was detected. An authorised-completion test asks the missing question: did every consequential action remain connected to the user’s original purpose and permissions?

The control should not depend on the agent’s prose. A separate policy layer can restrict purchases, identity changes, messages and irreversible submissions unless the user’s mandate explicitly covers them. The system should bind every tool call to the current task, source and permission, then pause when an untrusted page proposes a new goal.

## Two ledgers, not one blended score

Organisations evaluating browser agents should keep task performance and mandate integrity separate. A blended average can hide the very failure that matters: the agent completes many useful tasks and occasionally takes an unauthorised action.

  - Outcome ledger: Did the agent produce the right result, use current evidence and stop at sensible decision points?

  - Mandate ledger: Did every tool call remain within the user’s stated objective, data scope, identity and permitted effects?

  - Source-boundary tests: Can the agent distinguish content to analyse from instructions it may obey across text, images and accessibility data?

  - Adaptive tests: Do defences hold when an attacker revises wording, placement and format after observing failure?

  - Recovery evidence: When a control trips, can the system revoke capability, preserve the trace and verify a safe state?

The strongest objection is practical. Separate detectors and policy checks can block legitimate pages, slow the agent and create a new component that attackers will target. BrowseSafe++ itself warns that a benchmark score is not a security guarantee and that public test sets will eventually saturate.

That objection argues for measurement, not surrender. Track false stops, missed attacks, task degradation and bypasses by source type. Keep the acting model, detector and permission layer independently versioned. Use adaptive tests and production incident review to refresh the evidence. The goal is not a perfect gate. It is a system in which useful autonomy and authority can be inspected separately.

## Revisited: an alert is still not a brake

On [30 October](https://www.lecxie.com/publications/ayeye-today/2026-10-30.html), AyEye argued that detection becomes control only when an organisation can connect an alert to a continuing task, revoke capability and verify a safe state. BrowseSafe++ strengthens that view at the page boundary. A model may recognise an injection and still follow it when acting. The new evidence does not overturn the earlier argument; it narrows the mechanism. Detection and action should not share an unexamined path.

LifeVal points toward agents that can help with the untidy work of everyday life. BrowseSafe++ shows why that progress needs a second measure. A useful agent does not merely finish. It preserves the reason it was allowed to act.

## Sources

  - [OpenAI, “LifeVal”](https://openai.com/index/lifeval/), 2 November 2026.

  - [Perplexity, “Building stronger defences for real-world prompt injection”](https://www.perplexity.ai/en-GB/hub/blog/browsesafe-plus-plus), 2 November 2026.

  - [Chen et al., “BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views”](https://arxiv.org/abs/2610.17457), arXiv:2610.17457, submitted 12 October 2026.

  - [BrowseSafe++ research artefact and SDK](https://github.com/perplexityai/browsesafe-plus-plus), accessed 3 November 2026.

Chiappe × OpenAI. Primary-source review and research with OpenAI; editorial argument, evidence distinctions and final judgement by LecXie.

Canonical: https://www.lecxie.com/publications/ayeye-today/2026-11-03.html
