Serving Probe v1
Generated 2026-06-11T23:11:28Z - seed 20260611 - status **pass**
In-distribution sample: 20 real titles - direct serve 100.0%, soft fallback 0.0%, generic fallback 0.0%
SOC agreement on direct serves: 100.0% of 20
Perturbed variants resolved or honestly fell back: 40/40 (100.0%)
Out-of-distribution: 34/34 behave correctly
Packet path: 5/5 packets served
Title-expansion exact round-trip: 20/20 (100.0%)
Title-expansion perturbed ok: 10/10 (100.0%)
OOD case results
| input | kind | tier | matched | pass |
|---|---|---|---|---|
| I run facebook ads for small businesses | description | function_archetype | Marketing Managers | pass |
| I do the books for a dental practice | description | function_archetype | Financial and Investment Analysts | pass |
| I do bookkeeping for dentists | description | function_archetype | Financial and Investment Analysts | pass |
| I manage our warehouse and shipping | description | fuzzy_credible | warehouse manager | pass |
| I write seo articles for saas companies | description | function_archetype | Marketing Managers | pass |
| I handle payroll and invoices | description | function_archetype | Financial and Investment Analysts | pass |
| we do brand campaigns and social media | description | function_archetype | Marketing Managers | pass |
| For fifteen years I have handled calendar triage, inbox routing, travel logistics, board deck prep, vendor follow-up, and discreet founder support across time zones for a venture-backed startup CEO. | description | generic_fallback | Management Analysts | pass |
| acountant | typo | fuzzy_credible | accountant | pass |
| marketting manager | typo | fuzzy_credible | marketing manager | pass |
| softwear developer | typo | fuzzy_credible | software developer | pass |
| projct manager | typo | fuzzy_credible | project manager | pass |
| oprations manager | typo | fuzzy_credible | operations manager | pass |
| wizard of light bulb moments | gibberish | generic_fallback | Management Analysts | pass |
| chief vibes officer | gibberish | generic_fallback | Management Analysts | pass |
| asdkjh qwerty | gibberish | generic_fallback | Management Analysts | pass |
| ninja rockstar guru | gibberish | generic_fallback | Management Analysts | pass |
| consultant | vague | exact | consultant | pass |
| manager | vague | exact | manager | pass |
| analyst | vague | exact | analyst | pass |
| founder | alias | alias | General and Operations Managers | pass |
| ceo | alias | alias | General and Operations Managers | pass |
| chief executive | alias | alias | General and Operations Managers | pass |
| executive assistant to CEO | alias_context | exact | executive assistant to ceo | pass |
| senior executive assistant to founder | alias_context | fuzzy_credible | executive assistant | pass |
| chief executive assistant | alias_context | fuzzy_credible | executive assistant | pass |
| founders executive assistant | alias_context | fuzzy_credible | executive assistant | pass |
| founder's executive assistant | alias_context | fuzzy_credible | executive assistant | pass |
| forklift operator night shift | blue_collar | exact | forklift operator night shift | pass |
| I drive a forklift | blue_collar | fuzzy_credible | forklift driver | pass |
| I drive a taxi at night | blue_collar | fuzzy_credible | Taxi Drivers | pass |
| I split nights between taxi driving and warehouse forklift work | mixed_blue_collar | fuzzy_credible | forklift operator warehouse | pass |
| long haul truck driver | blue_collar | exact | Long Haul Truck Driver | pass |
| barista | blue_collar | exact | barista | pass |