{"collectionById":{"b61e0587-0af2-45c4-ba45-8ba53c936964":{"id":"b61e0587-0af2-45c4-ba45-8ba53c936964","name":"Case Study","fieldSchemas":[{"id":"966d46d8-b05c-426b-a399-400bbf1bbf45","name":"Title","type":"plain_text","role":"primary"},{"id":"0b7188b9-0cf9-47e7-81a2-52b99f2b03fa","name":"Slug","type":"slug","role":"slug"},{"id":"db44eed7-6ecd-4707-aff9-f0b5768bf179","name":"Your role and ownership","type":"rich_text"},{"id":"c55211d3-de56-4b70-8016-3f2de559a383","name":"Opportunity Solution Tree Image","type":"image"},{"id":"953ca41e-4a0e-47c7-8cf7-ce8b6b53449d","name":"Opportunity Solution Tree","type":"rich_text"},{"id":"86e0bc77-b438-4f93-9f49-cb09a0a84b00","name":"Success Metrics","type":"rich_text"},{"id":"51353f4e-5480-462a-8e43-7a7136c62c6e","name":"Problem Statement","type":"rich_text"},{"id":"4aeea936-d0cd-4693-a4f5-e00209e48bdc","name":"The Solution Chosen to Pursue","type":"rich_text"},{"id":"be606071-40a9-47d0-9c6a-82f37259d614","name":"Before solution image","type":"image"},{"id":"0cdad3fc-6620-447e-a695-38a5201ab5bf","name":"After solution caption","type":"plain_text"},{"id":"16a1752f-6ab5-403c-9872-bf0a3d89d091","name":"Before solution caption","type":"plain_text"},{"id":"db360f0d-ae97-489f-8769-bd2477d8aca0","name":"After solution image","type":"image"},{"id":"4e859e9e-8106-462b-b25d-e9a319db7462","name":"Before solution URL","type":"link"},{"id":"9ce96517-489d-43a3-8ea1-e3af49530ba9","name":"After solution URL","type":"link"},{"id":"253aa052-5178-4756-90cd-5ee5227756e2","name":"Is it feasible to for engineering to build?","type":"plain_text"},{"id":"e590cf8f-54e3-487f-bf3e-817163c9e9fe","name":"Is it valuable to the customer?","type":"plain_text"},{"id":"21f12812-6ab6-431e-9cc9-7b1c5bbdf21b","name":"Is it viable for the business?","type":"plain_text"},{"id":"95ec0a2e-799c-4500-bd8e-fa8b0dcfa833","name":"Is it usable for the end-user?","type":"plain_text"},{"id":"3c997a22-893c-4398-ba5b-d7e357cb6e85","name":"Core solution responsibility","type":"plain_text"},{"id":"2b27f77f-2329-449d-a30e-7b8f6964a53d","name":"Evidence of impact responsibility","type":"rich_text"},{"id":"2d996378-4e01-4e42-98ef-a061114cac04","name":"Status and next steps caption","type":"rich_text"},{"id":"a09ef021-195d-47e2-ad18-66ce841d37c3","name":"Constraints and tradeoffs caption","type":"rich_text"},{"id":"9c43be24-b69a-4d69-9f36-dc4b88f6afe6","name":"Final Result Preview","type":"link"}],"itemById":{"5c9c161c-6f6e-48fa-b5eb-2681d139d8df":{"id":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","index":"\"","collectionId":"b61e0587-0af2-45c4-ba45-8ba53c936964","fields":[{"id":"0ad60b60-1d9b-49a1-aa29-0b9736e67bb6","value":"{\"root\":{\"children\":[{\"children\":[{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\"Within the time on task comparision bar chart, the actual pilot result of 6 min 45 sec not only beat the 9 min 23 sec baseline, but also surpassed the 7 min 30 sec success target by 45 seconds, representing a 28% reduction in time on task.\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":0,\"textStyle\":\"\"}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"root\",\"version\":1}}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"2b27f77f-2329-449d-a30e-7b8f6964a53d"},{"id":"a65dbf53-cfc1-4b4d-8e58-b0fa85d9fba8","value":"{\"root\":{\"children\":[{\"children\":[{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\"With App Form Suggestions now shipped into the pilot environment, the focus moves to ensuring the feature reaches its full potential in the hands of real brokers at scale.\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":0,\"textStyle\":\"\"},{\"children\":[],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":0,\"textStyle\":\"\"},{\"children\":[{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\"To drive that, the next steps center on three priorities: first, \",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"launching a lightweight product marketing push\",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\" — a short email campaign and in-app onboarding moment — so brokers arrive with the right mental model and the 20% adoption target is reachable from day one rather than left to chance; second, \",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"tracking all three success metrics in production\",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\" over a 30–60 day window to confirm that the 28% time-on-task reduction, ≥20% adoption rate, and ≥40% retention rate hold beyond the Adelie cohort and across a broader range of broker types and submission profiles; and third, \",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"using production telemetry to close the 27% non-Strong-Match gap\",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\" by identifying which field types and NAICS codes are generating weaker confidence scores, then improving the MQS and CQS agent resolution logic to push that 73% Strong Match rate higher over successive iterations.\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":0,\"textStyle\":\"\"}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"root\",\"version\":1}}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"2d996378-4e01-4e42-98ef-a061114cac04"},{"id":"380a7342-6691-4519-8529-1e7194bc495e","value":"{\"root\":{\"children\":[{\"children\":[{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\"The Opportunity Solution Tree connected brokers’ pain in Terminal to a focused solution by clarifying that the desired outcome, faster and more trustworthy completion of complex applications, was blocked by five root opportunities: inaccurate answers, missing source data, cognitive overload, low confidence in Terminal for complex work, and an unsafe way to pause without losing progress. Based on this map, the team chose App Form Suggestions, an AI-powered side panel that provides suggested answers with confidence tiers and transparent source references, keeping brokers in control while reducing verification effort. The approach was guided by assumptions about reducing abandonment through deferral states, driving habitual use, cutting completion time via one-click acceptance, increasing repeat usage through visible time savings, and shipping safely on existing infrastructure without performance or architectural risk. Together, this created a clear, prioritized set of validations to run before and after launch.\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":0,\"textStyle\":\"\"},{\"children\":[],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":0,\"textStyle\":\"\"},{\"children\":[{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"I built the opportunity solution tree to connect the core bottleneck to opportunities and testable assumptions the team could align on.\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":1,\"textStyle\":\"\"}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"root\",\"version\":1}}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"953ca41e-4a0e-47c7-8cf7-ce8b6b53449d"},{"id":"ba977f38-8247-42ee-a8ce-2d7176838bad","value":"The current state of Terminal lets you create a new application and jump straight into a manual flow for each question.","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"16a1752f-6ab5-403c-9872-bf0a3d89d091"},{"id":"057ae96c-6ea8-4421-9b36-436e774837c0","value":"The new Terminal flow lets you choose several ways to start a new application, including the AI form suggestion option.","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"0cdad3fc-6620-447e-a695-38a5201ab5bf"},{"id":"259486d7-793d-4d66-8625-5c7df6b29521","value":"{\"image\":\"2ea3c31b648962690ba4644e3070ff15594da748\",\"imageThumbnail\":\"da679d25e5ede774f033514050719158afa51f8f\",\"originalImageHeight\":900,\"originalImageWidth\":1440,\"altText\":\"AI-solution-after\",\"fileName\":\"How would you like to start_ 1.png\"}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"db360f0d-ae97-489f-8769-bd2477d8aca0"},{"id":"c28ce473-1243-4f07-9f86-444b45ee3277","value":"{\"root\":{\"children\":[{\"children\":[{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"We aligned cross-functionally to pursue App Form Suggestions as the best path to reduce verification effort while keeping broker judgment in control.\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":1,\"textStyle\":\"\"}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"root\",\"version\":1}}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"4aeea936-d0cd-4693-a4f5-e00209e48bdc"},{"id":"6b388210-838a-4254-b889-5e452a3926e0","value":"How an AI-powered side panel helped eliminate hours of manual data verification for commercial insurance brokers","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"966d46d8-b05c-426b-a399-400bbf1bbf45"},{"id":"d0ce3a70-5b83-4050-8091-efe5238a3fdc","value":"{\"root\":{\"children\":[{\"children\":[{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"Role:\",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\" Product Design Manager (senior IC design leadership in executing design product direction)\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":1,\"textStyle\":\"\"},{\"children\":[{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"Ownership:\",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\" Full end-to-end ownership of discovery, definition, and design. You operated as the strategic design lead across research, synthesis, experimentation, and delivery handoff.\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":1,\"textStyle\":\"\"},{\"children\":[{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"Team:\",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\" Product Owner, engineering partners, a Product Analytics team, and Adelie Risk Advisors (an in-house agency of 4 brokers who served as the primary usability testing cohort), alongside stakeholders who aligned on direction and experiment outcomes.\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":1,\"textStyle\":\"\"},{\"children\":[{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"Timeline:\",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\" 6 weeks - Q2 2026\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":1,\"textStyle\":\"\"}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"root\",\"version\":1}}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"db44eed7-6ecd-4707-aff9-f0b5768bf179"},{"id":"83b69550-1bcc-4ed3-b077-f03f7b94aea8","value":"{\"url\":\"https://bp-prototype-playground.netlify.app/app-form-suggestions/home\",\"openInNewTab\":false}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"9c43be24-b69a-4d69-9f36-dc4b88f6afe6"},{"id":"2f879ebf-4595-48d4-a284-50e565e40bf7","value":"{\"image\":\"956dbd272ff4fe7e4de37ca03ba6a1dfa01f31da\",\"imageThumbnail\":\"6747040c6efe152e8027833f7cec71c72d607139\",\"originalImageHeight\":3820,\"originalImageWidth\":8164,\"altText\":\"AI-OST\",\"fileName\":\"OST.png\"}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"c55211d3-de56-4b70-8016-3f2de559a383"},{"id":"d5e5c16b-80d2-45e0-b679-b3fd36851edc","value":"{\"image\":\"7993719baab8ea6f0cacc01dd5d87a9a0e2520fb\",\"imageThumbnail\":\"778ea38ca93227a4315434727ca05439fc23adab\",\"originalImageHeight\":900,\"originalImageWidth\":1462,\"altText\":\"AI-solution-before\",\"fileName\":\"Home - Bold Penguin Terminal 1.png\"}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"be606071-40a9-47d0-9c6a-82f37259d614"},{"id":"a1114dd9-c9bc-465b-85dc-e50832d814b8","value":"{\"root\":{\"children\":[{\"children\":[{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\"The Bold Penguin Terminal is a cloud-based commercial insurance platform that lets agents, brokers, and agencies quickly triage risks, then quote, compare, and bind small business policies from multiple carriers through a single, streamlined application interface.\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":0,\"textStyle\":\"\"},{\"children\":[],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":0,\"textStyle\":\"\"},{\"children\":[{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\"Commercial insurance brokers using Bold Penguin’s Terminal \",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"struggle to complete complex applications\",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\" because verifying data in the Multi‑location and Buildings sections still depends on manual back‑and‑forth with clients. As a result, high‑value submissions stall, brokers lose time chasing details, and many abandon the workflow to finish applications directly in carrier portals instead of staying in Terminal.\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":0,\"textStyle\":\"\"},{\"children\":[],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":0,\"textStyle\":\"\"},{\"children\":[{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"I scoped and authored the problem statement by synthesizing user feedback and funnel data to clarify the highest-friction moments (Multi-location + Buildings verification) and the business impact (abandonment to carrier portals).\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":1,\"textStyle\":\"\"}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"root\",\"version\":1}}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"51353f4e-5480-462a-8e43-7a7136c62c6e"},{"id":"20131a21-c601-4f09-be96-d640f3f65cae","value":"{\"root\":{\"children\":[{\"children\":[{\"children\":[{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\"Time on task for the entire application ****(including Multi-location + Buildings) should take less than \",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"7 mins, 30 secs\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"listitem\",\"version\":1,\"value\":1},{\"children\":[{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\"Adoption rate of new App Form Suggestions should be \",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"20% or \u003e\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"listitem\",\"version\":1,\"value\":2},{\"children\":[{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\"Retention Rate of users using App Form Suggestions a second time should be \",\"type\":\"text\",\"version\":1},{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"40% or \u003e\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"listitem\",\"version\":1,\"value\":3}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"list\",\"version\":1,\"listType\":\"bullet\",\"start\":1,\"tag\":\"ul\"},{\"children\":[],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":0,\"textStyle\":\"\"},{\"children\":[{\"detail\":0,\"format\":1,\"mode\":\"normal\",\"style\":\"\",\"text\":\"I partnered with Product Analytics to define the instrumentation and the success-metric set we would use to evaluate adoption, speed, and trust.\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":1,\"textStyle\":\"\"}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"root\",\"version\":1}}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"86e0bc77-b438-4f93-9f49-cb09a0a84b00"},{"id":"b97cfaa6-6189-4617-86fa-803ec4d492b0","value":"{\"url\":\"https://www.awesomescreenshot.com/video/52718296?key=ac177c556a43f55702ae3214203c2417\",\"openInNewTab\":false}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"4e859e9e-8106-462b-b25d-e9a319db7462"},{"id":"3ec18792-b6ed-4ac7-bb90-ac325f11a432","value":"{\"root\":{\"children\":[{\"children\":[{\"detail\":0,\"format\":0,\"mode\":\"normal\",\"style\":\"\",\"text\":\"The core constraints and trade-offs on this project centered on balancing speed, scope, and broker accountability. The pre-release study was limited to four brokers from a single internal cohort (Adelie Risk Advisors), which provided strong directional evidence but left open questions about performance across independent and enterprise brokers with different NAICS codes and risk profiles. The decision to exclude AI suggestions from Carrier questions sections and to block auto-apply on ratable pricing fields (revenue, payroll) intentionally traded maximum automation for broker trust and E\u0026O accountability. The interaction pattern also had to balance field-level precision with cognitive load, which led to rejecting per-question and page-level designs in favor of a side panel that preserved the existing form mental model without disrupting broker workflow. Finally, token cost containment within a ~$0.232/submission budget required building on existing LangGraph infrastructure rather than pursuing a new architecture, maintaining feasibility while setting boundaries on how far AI capability could be extended at launch.\",\"type\":\"text\",\"version\":1}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"paragraph\",\"version\":1,\"textFormat\":0,\"textStyle\":\"\"}],\"direction\":\"ltr\",\"format\":\"\",\"indent\":0,\"type\":\"root\",\"version\":1}}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"a09ef021-195d-47e2-ad18-66ce841d37c3"},{"id":"d7834aa4-235d-4161-bfde-040052f5914f","value":"{\"url\":\"https://bp-prototype-playground.netlify.app/app-form-suggestions/home\",\"openInNewTab\":false}","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"9ce96517-489d-43a3-8ea1-e3af49530ba9"},{"id":"723e8b76-cf10-48d0-ab29-4e537ce1a3d4","value":"In-session monitoring of 12 broker sessions showed complex application completion time fell from an average of 9 mins, 23 secs to 6 mins, 45 secs - A 28% reduction in time on task completion time.","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"e590cf8f-54e3-487f-bf3e-817163c9e9fe"},{"id":"47df3364-9f4a-4a82-abcb-f6b19d02c22e","value":"In-session monitoring of 12 broker sessions, out of 264 application question, 192 we’re answered with a “Strong Match” selection - 73% Strong Match AI confidence rate","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"95ec0a2e-799c-4500-bd8e-fa8b0dcfa833"},{"id":"327b3d9a-a072-40df-9358-3a7bc399bc8f","value":"The biggest hurdle was estimating the AI token cost for a typical full application submission. We approximated the cost at ~$0.57 = full submission.  Confidence via cascading resolution (API \u003e RAG \u003e defaults) with Critique Agent adversarial validation.","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"253aa052-5178-4756-90cd-5ee5227756e2"},{"id":"3b532b1a-4560-4645-b5d3-f6e3af67cdfa","value":"Pre-release already shows movement toward 7:30 target. Ongoing pilot measures real-world adoption. LangSmith observability enables cost-per-completion monitoring.","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"21f12812-6ab6-431e-9cc9-7b1c5bbdf21b"},{"id":"af989368-21a9-4343-8a4a-12b247c1216e","value":"To verify the risks for this feature, I initiated in-session monitoring across 12 broker sessions to capture behavioral evidence of time reduction. I led analysis of AI suggestion confidence patterns across 264 application questions to validate broker comprehension and interaction quality. I partnered with engineering to model AI token costs and align on the architectural assumptions needed to ship sustainably. I worked with Product Analytics to define success criteria and set up the observability framework used to monitor adoption and cost in production.","itemId":"5c9c161c-6f6e-48fa-b5eb-2681d139d8df","fieldSchemaId":"3c997a22-893c-4398-ba5b-d7e357cb6e85"}]}}}},"slugByItemId":{"c62c6313-526a-497f-a060-89da1a35beaa":"how-reducing-onboarding-steps-improved-new-agent-conversion-in-30-days","5c9c161c-6f6e-48fa-b5eb-2681d139d8df":"how-an-ai-powered-side-panel-helped-eliminate-hours-of-manual-data-verification-for-commercial-insur"}}