Record the current failure, affected population, baseline, target, measurement source, accountable owner, adoption threshold, and decision date. Product preference and feature volume are not substitutes for an observable result.
Six scenarios to run with every finalist
1. Multi-channel identity and duplicate tickets
Prepare realistic source data, name the actor and owner, include a failure or exception, and set a measurable completion threshold. Preserve the output and every manual correction.
2. Priority and entitlement routing
Use the proposed permission role and actual integration boundary. Ask an ordinary user to complete the work, then repeat it after a changed owner or invalid input.
3. Engineering escalation and status communication
Measure elapsed work, handoffs, duplicate entry, missing context, and recovery effort. Distinguish native capability from configuration, integration, custom work, roadmap, or workaround.
4. Knowledge suggestion and review
Inspect audit history, notifications, approvals, reports, and exports. A successful screen demonstration is insufficient if the team cannot operate or evidence the result later.
5. After-hours and surge handling
Stress the expected volume and an adverse case. Record the plan or add-on required, the accountable administrator, and the consequence if the scenario fails in production.
6. Complete case and audit export
Export the finished record with relationships, attachments, history, permissions, and identifiers. Estimate the work required to reconcile it and move to another service.
Give vendors the same script, roles, data, integrations, constraints, and time. Reviewers should score independently before discussion and cite a retained artifact. Mandatory legal, privacy, security, data, and critical-workflow gates are pass or fail; they cannot disappear inside an average score.
Stakeholder decision table
| Accountable stakeholder | Decision and evidence required |
|---|---|
| Support operations owns routing and service levels | must define the pass condition and approve any operating compromise |
| Agents validate daily work and exception recovery | must supply dated evidence and identify the unresolved dependency |
| Engineering owns defect handoff | must test normal work without accepting vendor coaching as product capability |
| Security and privacy review customer data | must confirm the control, configuration, integration, and recovery owner |
| Finance and procurement model agents, ai, and add-ons | must reconcile the quote, complete cost, term, renewal, reduction, and exit exposure |
Record dissent and conditions instead of forcing false consensus. The final approver should know which claims were observed, which were documented, which remain assumptions, and who accepts any residual exposure.
Normalize the complete cost
Model the billable user or usage population at launch, year one, and renewal. Add edition, minimum quantity, guests or collaborators, consumption, AI, storage, integrations, support, services, migration, data preparation, internal administration, training, parallel operation, tax, renewal uplift, and exit. Keep vendor charges and internal labor separate. Compare a likely case with a stress case that assumes slower adoption, more billable volume, a premium control, and delayed implementation.
Retain the public price page, dated quote, assumptions, calculation, and contract position. Reconcile plan names, currency, term, discounts, add-ons, allowances, overages, implementation scope, service levels, start date, and renewal protection. An unknown cost should be a range with an owner and due date—not zero.
Implementation and first 90 days
Require owners for design, identity, permissions, data preparation, migration, validation, integrations, testing, training, support transition, acceptance, and decommissioning. Define launch gates and rollback authority. During the first billing cycle, reconcile provisioned access to the invoice. At 30, 60, and 90 days, review adoption by required workflow, unresolved incidents, administrator effort, outcome measures, and any tool that was supposed to be retired.
Renewal and exit
Record term, auto-renewal, notice deadline and delivery method, quantity reduction, uplift, price protection, true-up, suspension, service levels, support, export, transition help, post-termination access, deletion, and retained copies. Test a representative export before signature and again early enough before renewal to switch. A renewal decision should be retain, resize, renegotiate, replace, or cancel, supported by usage, outcome, risk, invoice, support, and portability evidence.
Run the same test and retain the decision
Use the demo scorecard for observed workflows and the pricing worksheet for a three-year commercial model.
Support operations tests that expose real service fit
1. Multi-channel identity and duplicate tickets
Send one issue through email, chat, and a forwarded message. Verify requester identity, organization, language, consent, prior context, duplicate detection, merging, attachments, and the audit trail. Agents should not have to search three queues or accidentally expose one customer's message to another.
2. Priority and entitlement routing
Create two similar requests from customers with different service entitlements, business hours, regions, and impact. Confirm priority is derived from evidence, not only requester choice. Test queue ownership, reassignment, pause conditions, breached targets, and the supervisor's ability to explain every timing calculation.
3. Engineering escalation and status communication
Escalate a reproducible defect with environment, logs, severity, customer impact, workaround, and requested update cadence. Verify linked engineering work can change without losing support ownership. Test customer-safe status updates and closure only after the resolution and knowledge record are validated.
4. Knowledge suggestion and review
Ask a common question, an ambiguous question, and a question affected by a recent product change. Inspect which article or source supports the suggested answer, how confidence and human handoff work, who owns outdated content, and whether AI usage creates a separate billable event.
5. After-hours and surge handling
Simulate an urgent issue outside normal hours followed by a volume spike. Check on-call routing, acknowledgment, language coverage, collision prevention, queue aging, mass communication, temporary staffing, and recovery reporting. Price day passes, extra agents, automation, or AI capacity used during the event.
6. Complete case and audit export
Export a case with every message, internal note, attachment, requester, organization, status change, SLA event, satisfaction response, linked problem, and custom field. Confirm identities and timestamps remain usable and that redaction, retention, legal hold, and deletion rules are understood.
Failure signals to challenge before approval
- Omnichannel branding is purchased before channel identity and routing behavior are proven.
- AI resolution claims are evaluated without source quality, handoff, consumption, and error review.
- Light agents or collaborators gain more customer access than the operating role requires.
- Service reports hide paused time, reopened cases, merged tickets, or excluded hours.
- The exit plan exports message text but loses attachments, audit history, SLA events, or knowledge links.
Measures to carry into the first renewal
- time to accountable ownership by entitlement and severity
- reopen and transfer rate for representative request types
- backlog age distribution rather than average age alone
- knowledge-assisted resolution with human quality sampling
- percentage of sampled cases exported with complete audit evidence
Assign one named owner and evidence source to each measure. Record the baseline before configuration, review it during the first 90 days, and preserve the result beside invoice and support evidence. If a measure cannot influence retain, resize, renegotiate, replace, or cancel, it is not yet a useful renewal control.