Account managers keep and grow existing accounts under a commercial target. They now prepare customer work with AI and read AI analysis of their accounts. Many also sell products with AI features of their own. AI Fluency for Account Managers measures how well they work with AI inside that work. To measure the account work itself, use the Account Managers test alongside it. Both serve candidates and current employees alike.
We built it in two parts. The first decides what to measure: which skills earn a place, and what evidence backs them. The second decides how to measure them. Each scenario tests one skill, follows the same set of rules and passes a set of checks before it’s used. The assessment is built on peer-reviewed and subject-matter expert research, and validated by I/O psychologists.
Part one: deciding what to measure
The evidence base
The skills draw on two kinds of evidence: published research on how people work with automated and AI systems, and a job analysis of account work.
For each way AI can make the work go wrong, we looked for two kinds of study. One shows the human tendency behind the problem, such as relying on a decision aid without noticing the errors it brings in. Studies of this kind can come from any period of research. The other shows the same behavior in AI systems, such as a model giving a confident answer when its input lacks the facts. Studies of this kind date from 2023 or later. Every way of going wrong that the assessment uses is backed by both kinds of study.
The job analysis
The job analysis set out the work account managers produce and the tasks where AI now takes part. That work runs from quotes and proposals to account plans, business reviews and renewal notices. The tasks include preparing customer work with an AI assistant, reading AI summaries and scores of accounts, and selling products with AI features.
For each piece of work where AI takes part, we set out two things.
Strong performance. What good work looks like when AI is part of it. For a proposal prepared with AI, that means deciding what the AI drafts and what comes from company systems, and briefing it with the terms the proposal depends on.
Failure modes. A failure mode is a specific way AI makes that work go wrong. Each one records what goes wrong, why nobody notices, how the AI behaves to produce it, and what a capable account manager without AI fluency does instead.
The exclusion test
Every failure mode then faces one test: take the AI out of the work. If the problem still happens, it belongs to account work in general, and we exclude it. We keep only the failure modes that exist because AI is involved. That keeps the assessment measuring AI fluency inside the role, rather than the role itself.
Five gates a skill has to pass
Next, we group the failure modes into candidate skills. Every candidate skill has to pass the same five gates, in order. If it fails any one of them, it’s cut.
Gate | The test |
|---|---|
Specific to the role | The problem shows up in an account manager’s work, in a form that needs both the AI and the role. If it would read the same for any professional, it belongs in the general AI Fluency Assessment instead. |
Not already covered | The general AI Fluency Assessment doesn’t already measure it, or doesn’t measure it at this depth. A role skill has to add something new. |
Measurable in a work scenario | A single scenario from the role’s work can measure it, with a graded score from the best response to the weakest. |
Needs AI fluency to answer | A capable account manager with little AI experience chooses a weaker option. If ordinary account sense gets it right, the skill measures the job, not AI fluency. |
Enough evidence to measure | The skill holds enough distinct failure modes for several separate scenarios, so no score depends on a single example. |
We grouped the failure modes by what the AI is doing in the work. In account work, AI does one of three things: it prepares work for the account manager, it analyzes their accounts, or it is part of the product they sell. Each group holds enough distinct failure modes for several separate scenarios, and all three passed all five gates.
The three dimensions of AI fluency in account work
Each of the three groups became a dimension: a distinct skill the assessment measures and scores on its own. Together, with their definitions and the evidence behind each, they make up the rubric. Every scenario is written against this rubric, and every score traces back to it.
Preparing customer deliverables with AI. Directs AI on the work prepared for customers, such as quotes, proposals, renewal emails and business reviews. It covers deciding what the AI prepares and what comes from company systems, briefing the AI with the facts, terms and goals each deliverable depends on, and presenting each figure and commitment as it really applies to the customer.
Analyzing account data with AI. Turns AI analysis of accounts and customer conversations into sound account decisions. It covers knowing what each AI summary, score or forecast is based on and what it leaves out, and weighing it against what the account manager knows of the customer.
Advising customers on AI products. Advises customers on the AI features of the product being sold, before buying, during rollout and at renewal. It covers setting expectations the customer’s own results will bear out, and explaining what the feature’s results mean for the customer’s decisions.
Part two: how each scenario is built
Specifications set before writing
Before we write a scenario, we fix its specifications. They apply to the scenario and to every one of its options.
Difficulty. The assessment is set at the intermediate level, for people who already use AI at work. A scenario at this level asks the candidate to apply one fact about how AI behaves, and to fit the effort to what’s at stake.
Plain language. US English and short sentences, so the scenario reads the same for every candidate.
Tool-agnostic. Each scenario describes the AI in plain, vendor-neutral words, and no scenario names a product. The best response depends on how AI works in general.
Goods and services. The scenarios are spread between selling goods and selling services, close to half each.
Weighted options
Each scenario comes from an account manager’s own work, with AI in the picture, and offers four options. Each option carries a weight, and the weights follow the same logic in every scenario.
The best option makes the move that stops the problem, given how the AI behaves. It carries the full weight.
A workable option gets to an acceptable result, but at a real cost, like a wasted round trip or more checking than the stakes call for. It carries part of the weight.
A plausible option sounds reasonable, but rests on a misunderstanding of how AI works. It carries a small part of the weight.
The AI-naive option is what a capable account manager without AI fluency would do. It sounds sensible, and it carries no weight at all.
The best option and the AI-naive option differ on one fact about how AI behaves. That might be information the AI was never given, what a summary or score was built on, or how the AI inside a product behaves. Every option comes with a written reason for its weight.
What earns the full weight also changes from one scenario to the next. Sometimes it’s changing what the AI is given, rather than checking what it returns. Sometimes it’s checking a different thing from the one that looks risky. And sometimes it’s not using AI for that step at all. So no single habit, like always picking the most cautious option, earns the full weight every time.
Gates for every scenario and option
Every scenario and every option goes through the same gates. Anything that fails is rewritten or cut.
Gate | The test |
|---|---|
Needs the AI | Take the AI out, and the scenario stops working. |
The best move works | The best option stops the problem. |
The AI-naive move falls short | The AI-naive option leaves the problem in place. |
Every option acts on the AI’s work | Each option is a move on what the AI is given, what it returns, or how it’s used. |
No clue in the options | The best option is never the longest, and each option is a single action. |
What each scenario measures, and how it’s scored
Each scenario drops the candidate into a real moment from account work, where AI is already part of the job. An AI assistant might have drafted a renewal email. An AI score might be about to shape an account decision. A customer might be asking how the product’s AI feature will perform for them. The candidate decides what happens next.
The assessment doesn’t test definitions or technical knowledge of how AI models are built. It scores the decision an account manager makes when AI is part of the work.
Each option’s weight counts toward its skill, and each of the three skills has its own score. Together the three skills also make an overall score. There’s no pass mark. Each scenario keeps a record of the difficulty it was set at and, once candidates take it, the difficulty observed and the share of candidates choosing each option. We use those figures to keep revising the scenarios.
Where the assessment fits
AI Fluency for Account Managers adds to the general AI Fluency Assessment, and each has its own score. The general assessment shows how someone works with AI in any role. This one shows whether they can do it inside account work. Bryq measures both alongside cognitive ability, personality traits, soft and hard skills in one integrated candidate profile. For where it fits in a full hiring process, see How to hire account managers.
Frequently asked questions
How was AI Fluency for Account Managers built? From published research on how people work with AI systems and a job analysis of account work. Every skill passed five gates, and every scenario passed its own checks before use. It’s built on peer-reviewed and subject-matter expert research, and validated by I/O psychologists.
What does it measure? Three skills: preparing customer deliverables with AI, analyzing account data with AI, and advising customers on AI products. Each has its own score, and together they make an overall score.
Does it test knowledge of AI? No. It scores the decisions an account manager makes when AI is part of the work, not definitions or how AI models are built.
Is it the same for candidates and current employees? Yes. The same assessment works for hiring, internal moves and development planning.
