Are you an AI Analyst ready to translate raw model output into a reliable, high-quality production system? Do you excel at the craft of Prompt Engineering and thrive on the challenge of ensuring Generative AI output meets rigorous quality standards in mission-critical applications? Join our R&D Operations Team. You will be the AI analyst responsible for the end-to-end quality, performance, and operational tuning of our Generative AI-driven support system (Co-Pilot). Your mission is to actively shape the model's intelligence, govern the data it uses, and implement the mechanisms that guarantee its accuracy, directly accelerating our customer support engineering velocity.
Key Responsibilities
Support & Customer Advocacy: Champion the Support Journey in Engineering. Develop in-product support instructions that reflect and address real incoming cases, enhancing the quality and effectiveness of AI feature solutions.
Model Quality Validation: Use existing evaluation platforms and methodologies to validate production models. Monitor quality metrics to continuously assess and rank AI answers for accuracy and reliability.
Prompt Engineering & CT Loop: Drive the Continuous Training loop through systematic prompt engineering (refining and versioning inputs). Analyze failures to define R&D actions or features needed to close model performance gaps.
AI Knowledge Governance: Act as AI Content Governor, implementing controls to verify and ingest compliant knowledge base content, ensuring a quality data source.
Cross-Functional SME: Serve as the AI Support Co-pilot Subject Matter Expert, partnering with R&D and Support Enablement to translate quality issues into core model logic improvements and feature development.
Key Responsibilities
Support & Customer Advocacy: Champion the Support Journey in Engineering. Develop in-product support instructions that reflect and address real incoming cases, enhancing the quality and effectiveness of AI feature solutions.
Model Quality Validation: Use existing evaluation platforms and methodologies to validate production models. Monitor quality metrics to continuously assess and rank AI answers for accuracy and reliability.
Prompt Engineering & CT Loop: Drive the Continuous Training loop through systematic prompt engineering (refining and versioning inputs). Analyze failures to define R&D actions or features needed to close model performance gaps.
AI Knowledge Governance: Act as AI Content Governor, implementing controls to verify and ingest compliant knowledge base content, ensuring a quality data source.
Cross-Functional SME: Serve as the AI Support Co-pilot Subject Matter Expert, partnering with R&D and Support Enablement to translate quality issues into core model logic improvements and feature development.
Requirements:
Required Qualifications
Minimum of 5+ years of professional experience in a blend of technical and analytical roles (e.g., Automated QA, Support Enablement, Data Analysis,AI Research. Prompt Engineering, MLOps), with a proven track record operating at the critical intersection of customer operations, data management, and AI/ML systems
AI/ML/LLM Foundation: Possesses a high-level understanding of AI tools, LLMs, machine learning, and applicative AI principles.
Python Proficiency: Proficient in Python for practical applications, including scripting, data processing, and building automation solutions.
Customer Domain Mastery: Demonstrated experience in customer-facing roles with a strong operational understanding of the Customer Support domain (workflows, knowledge base management, optimization).
Technical Communication: Strong skills in translating observed model performance issues and Agentic Action requirements into clear, prioritized technical requirements for R&D teams.
Ownership: A dedicated individual who takes full responsibility for their work, driving projects to successful completion.
Data/Content Governance: Experience with data validation, content management, and implementing data governance standards, especially for knowledge bases feeding AI systems.
Preferred Qualifications
Prompt Engineering: Proven expertise in systematically authoring, testing, and refining instructional inputs to drive specific model behavior.
Cloud Technologies: Experience with Cloud platforms like GCP, Kubernetes and containers
CI Pipelines using Gitlab
Experience with BigQuery
Experience working with APIs.
Required Qualifications
Minimum of 5+ years of professional experience in a blend of technical and analytical roles (e.g., Automated QA, Support Enablement, Data Analysis,AI Research. Prompt Engineering, MLOps), with a proven track record operating at the critical intersection of customer operations, data management, and AI/ML systems
AI/ML/LLM Foundation: Possesses a high-level understanding of AI tools, LLMs, machine learning, and applicative AI principles.
Python Proficiency: Proficient in Python for practical applications, including scripting, data processing, and building automation solutions.
Customer Domain Mastery: Demonstrated experience in customer-facing roles with a strong operational understanding of the Customer Support domain (workflows, knowledge base management, optimization).
Technical Communication: Strong skills in translating observed model performance issues and Agentic Action requirements into clear, prioritized technical requirements for R&D teams.
Ownership: A dedicated individual who takes full responsibility for their work, driving projects to successful completion.
Data/Content Governance: Experience with data validation, content management, and implementing data governance standards, especially for knowledge bases feeding AI systems.
Preferred Qualifications
Prompt Engineering: Proven expertise in systematically authoring, testing, and refining instructional inputs to drive specific model behavior.
Cloud Technologies: Experience with Cloud platforms like GCP, Kubernetes and containers
CI Pipelines using Gitlab
Experience with BigQuery
Experience working with APIs.
This position is open to all candidates.
