Lead Data Scientist

• Apply classification, regression, ranking, forecasting, clustering, and anomaly detection to enterprise AI and agentic use cases.
• Develop and evaluate intent classification, semantic matching, agent/skill/tool ranking, and model-routing techniques; compare learned approaches with rules, embeddings, and LLM-based baselines.
• Use statistical modeling to estimate execution latency, token demand, inference costs, workload trends, and resource requirements.
• Apply clustering and anomaly detection to identify recurring intents, capability gaps, unusual agent behavior, and execution regressions.
• Conduct exploratory analysis, dataset preparation, feature engineering, hypothesis testing, and reproducible experiments.

• Evaluate commercial API and self-hosted open-weight models for accuracy, reasoning, instruction adherence, structured outputs, latency, throughput, and total cost.
• Design and optimize system instructions, prompt templates, few-shot examples, tool-calling prompts, and context assembly strategies.
• Benchmark model alternatives for specific enterprise workloads and document quality/cost/latency tradeoffs.
• Collaborate with AI Engineering and Platform Engineering on inference performance, quantization tradeoffs, and GPU/VRAM capacity assumptions.

• Design and optimize document parsing, chunking, metadata, embedding, retrieval, reranking, and context selection strategies.
• Evaluate vector, keyword/BM25, hybrid, and semantic retrieval, including access-aware retrieval where relevant.
• Diagnose retrieval versus generation failures; measure Recall@K, nDCG, groundedness, faithfulness, and end-to-end answer quality.
• Compare embedding and reranking models using representative enterprise datasets and reproducible benchmarks.

• Assess when prompting, RAG, supervised instruction fine-tuning, or parameter-efficient adaptation is appropriate.
• Curate instruction datasets and holdout evaluations; apply SFT, LoRA, QLoRA, or comparable methods.
• Measure gains over base-model and prompting/RAG baselines, including generalization, regressions, data leakage, and operating costs.
• Collaborate on the packaging and validation of fine-tuned model artifacts and adapters.

• Apply modeling and evaluation expertise to intent routing, agent/skill discovery, MCP tool selection, workflow selection, and planning effectiveness.
• Evaluate task completion, tool-call correctness, argument extraction, delegation quality, and recovery from failures.
• Work with AI Engineering on agentic frameworks and understand MCP tool schemas and A2A Agent Cards, skills, and task delegation.
• Contribute decision models and quality signals without owning agent runtime, MCP server, or A2A protocol implementation.

• Design golden datasets, holdout sets, human review protocols, LLM-as-judge calibration, and statistical comparison methods.
• Automate evaluations and regression checks for prompts, foundation models, RAG, fine-tuned models, routing, and agents.
• Define measurable quality thresholds for accuracy, relevance, groundedness, instruction following, task completion, and cost/performance.
• Investigate regressions and use evidence to guide model, prompt, retrieval, and routing decisions.
• Set cross-team evaluation methodology, benchmark strategy, and acceptance thresholds; resolve tradeoffs among quality, cost, and delivery risk.

• Apply reproducible experiment tracking across datasets, code, model versions, hyperparameters, prompts, retrieval configurations, metrics, and artifacts.
• Develop and maintain automated training, fine-tuning, benchmarking, and evaluation pipelines, with model registry and artifact traceability.
• Define and integrate quality gates, regression tests, and model/prompt/RAG release-readiness criteria with engineering teams.
• Analyze production quality and drift signals, agent routing accuracy, latency, throughput, and token usage; recommend remediation.
• Collaborate with engineering on packaging, deployment handoffs, observability, rollback criteria, and ongoing model lifecycle management.
• Define shared experiment tracking, lifecycle governance, model promotion, and monitoring standards across initiatives; guide platform/tooling investment decisions.
• Define and drive adoption of shared MLOps/LLMOps methodologies, automated evaluation standards, model lifecycle practices, and quality gates across multiple initiatives.

• Define and drive Data Science technical direction across multiple initiatives or a product area.
• Establish shared modeling, benchmarking, evaluation, and MLOps/LLMOps standards adopted across teams.
• Make consequential technical tradeoffs involving model selection, fine-tuning, RAG strategies, evaluation approaches, performance, and cost.
• Prioritize Data Science investments based on business impact, technical feasibility, and available resources.
• Provide technical guidance and mentorship to Senior Data Scientists and other team members.
• Collaborate with AI Engineering, AI Quality, Platform Engineering, Product Management, and senior business stakeholders.
• Drive adoption of reusable AI/ML capabilities and consistent quality standards across teams.
• Remain hands-on in solving complex AI/ML problems and validating critical technical decisions.

• 12+ years of relevant Data Science, Machine Learning, or Applied AI experience.
• Bachelor's degree in Machine Learning, Statistics, Data Science, Physics, Computer Science, or a related field, or equivalent education and industry experience.
• Advanced Python proficiency with Pandas, NumPy, scikit-learn, and PyTorch or TensorFlow; SQL and practical data preparation skills.
• Strong statistics, experimental design, hypothesis testing, ML validation, and applied classification, regression, ranking, clustering, forecasting, and anomaly detection knowledge.
• Hands-on LLM prompting, embeddings, model comparison, and RAG design/optimization, including chunking, retrieval, and reranking.
• Practical instruction fine-tuning or parameter-efficient model adaptation experience, with measurable baseline comparisons.
• Experience designing evaluation datasets, automated quality metrics, and statistical model/LLM/RAG experiments.
• Hands-on MLOps and LLMOps: experiment tracking, model/dataset/prompt versioning, reproducible pipelines, registries/artifacts, evaluation automation, quality gates, and monitoring.
• Working knowledge of agent planning, routing, tool calling, MCP, A2A, and multi-agent patterns.
• Understanding of inference latency, throughput, concurrency, token cost, and on-prem versus provider-hosted model tradeoffs.
• Familiarity with AWS and/or Kubernetes or Red Hat OpenShift (OCP), and collaboration with platform engineering.
• Demonstrated experience setting Data Science technical direction across multiple initiatives or product areas.
• Proven ability to make consequential technical tradeoffs involving AI/ML approaches, model quality, operational performance, and investment priorities.
• Experience establishing reusable modeling, evaluation, and MLOps/LLMOps standards adopted by multiple teams.
• Strong track record of mentoring Senior Data Scientists and influencing technical decisions across engineering, product, and business stakeholders.

• Experience with OpenSearch, Elasticsearch, FAISS, Milvus, pgvector, or similar retrieval platforms.
• Hands-on use of LangGraph, OpenAI Agents SDK, Google ADK, CrewAI, or other agentic frameworks.
• Familiarity with MLflow, Kubeflow, Ragas, DeepEval, OpenJudge, LangSmith, or comparable MLOps/LLMOps and evaluation tooling.
• Experience with vLLM, NVIDIA Triton, quantization, GPU memory sizing, or distributed inference.
• Python backend APIs (FastAPI, Flask, or similar); OpenTelemetry and AI observability.
• AI safety, prompt injection evaluation, privacy, responsible AI, and model governance.

Our commitment to you!

BMC's culture is built around its people. We have 6000+ brilliant minds working together across the globe. You won't be known just by your employee number, but for your true authentic self. BMC lets you be YOU!

If after reading the above, You're unsure if you meet the qualifications of this role but are deeply excited about BMC and this team, we still encourage you to apply! We want to attract talents from diverse backgrounds and experience to ensure we face the world together with the best ideas!

BMC is committed to equal opportunity employment regardless of race, age, sex, creed, color, religion, citizenship status, sexual orientation, gender, gender expression, gender identity, national origin, disability, marital status, pregnancy, disabled veteran or status as a protected veteran. If you need a reasonable accommodation for any part of the application and hiring process, visit the accommodation request page.