Page 2 · Jobs
Discovering inefficiencies and turning user data into product direction.
Three jobs: a Gen-AI/analytics product management internship at IDeaS, an operations management internship at Daikin, and a summer as a deckhand in Alaska.

AI Chatbot Product Discovery & Alignment
- Product Roadmap
- RAG & Multi-Agent
- Requirements
- Team Alignment
- AI Evaluation
Resume highlights
- Aligned product and engineering on an AI evaluation pipeline and PM communication system. Cost Savings: $300k
- Authored Gen-AI system product roadmap for development prioritization. Improves competitor positioning.
- Authored 2 business requirements documents for an analytics explaining agent. Addresses $10 million lost sales.
- Synthesized 8000 Salesforce cases, 20 user interviews and 3 Datadog user flow dashboards for product roadmap.
- Developed V0 UI mock-ups to communicate product vision to UX designers and PM directors.
- Aided SAS analytics developers by delivering a Python statistical analysis for new marketing analytics product outputs.
Main project
Business requirements and the AI roadmap
Situation
- At IDeaS I needed to figure out what users would find valuable for an AI chat bot in our revenue management system.
- Align product on the requirements needed from the engineering teams.
- Align the SAS analytics team with the IDeaS AI engineers on the RAG and multi agent architecture.
Task
- Make an AI product roadmap and supporting business requirements documents.
- Lead alignment meetings.
Action
- Read Inspired by Marty Cagan to learn about product discovery and product alignment.
- Aligned teams on an evaluation pipeline to increase operational efficiency and quality assurance.
- Analyzed 8000 customer case files along with other user data.
- Made an AI system requirements document and an Analytics requirements document.
- Made a prioritized AI capabilities roadmap.
Results
- The company used my roadmap to prioritize a massive engineering effort.
- This project was at the top of the company's executive product strategy.
- Addresses 10 million in lost sales over 2 years.
9 steps
AI product roadmap.
2
3-phase business requirements documents, 51 pages total.
8,000
Customer support cases analyzed, in addition to other data.
Studied up

Inspired
Marty Cagan
Product discovery and alignment, used directly at IDeaS.
Self-driven project
Evaluation-driven AI development
Business problem: outdated PM procedures
Step 4Current state
Back to the start every time, by hand
Future state
- PM
- Test cases
- Evaluation rubrics
- Clear acceptance criteria
First step: learn about AI evaluation
Step 4
AI Engineering · Chip Huyen
Best practice:
- Evaluation pipeline
- Prompt engineering
- RAG / agent systems
Solution: team alignment
Step 4Proposal: evaluation pipeline
Solution design recommendations
Jack Callinan · 7-15-2026
PMs
- Evaluation test cases (linked to BRD)
- Rubrics for evaluation metrics (Confluence folder)
Engineers
- AI as a judge (future roadmap)
- System prompt filing
Conservative ROI = 20×
*assuming 3 engineers and 1 PM
Situation
- AI testing was slow, inefficient and not standardized.
- The AI engineering team and AI product team were separate from my SAS analytics product team.
Task
- Even though this wasn't my team, I knew this would become an issue for the execution of my project. Therefore, I decided I would research and align the team on the best evaluation driven product process.
Action
- Read AI Engineering with Foundation Models by Chip Huyen.
- I researched some of the recent literature.
- Created a pipeline proposal document.
- Aligned the AI engineers and product team on the process.
Results
- The company shifted resources to build the evaluation system.
- The company shifted resources to create human SME data for model context and evaluation.
20×
Conservative ROI, assuming 3 engineers and 1 PM.
Resource shift
Company resources moved onto the evaluation system.
32 pages
Evaluation pipeline proposal document.
What I delivered
AI roadmap
Prioritized gen-AI capabilities for development.
2
Business requirements documents.
Statistical analysis
For SAS on emerging solutions product outputs.