Principal Data Scientist
Mountain View, California
Employer: Microsoft
Industry: Research, Applied, & Data Sciences
Salary: $161600 per year
Job type: Full-Time
At Microsoft, our mission is to empower every individual and organization on the planet to achieve more, and we recognize the transformative power of artificial intelligence in realizing this vision. The Bing Metrics and Evaluation team is at the forefront of this journey, meticulously measuring and enhancing the user experience across Bing and Copilot. Our goal is to execute this with precision and efficiency.
This role is available in Mountain View, CA or Redmond, WA.
In this role as a Principal Data Scientist, you will work to improve online metrics that evaluate Product quality in A/B testing and non-A/B testing situations. You will apply sophisticated statistical methods and large language models (LLMs) combined with human labels to conduct thorough evaluations of Bing's Search Product through the lens of metrics. Your expertise will guide Bing's product quality assessments, and user satisfaction measurement.
As part of the rapidly advancing Search and AI domain, our team is a leader in applied machine learning at Microsoft. We are committed to providing an unparalleled search and AI assistant experience to over 500 million monthly active users globally through Bing. Additionally, our work impacts other major search engines like Yahoo, DuckDuckGo, and emerging platforms such as Neeva and You.com.
Join us in shaping the future of search and AI, where your contributions will influence the experiences of users worldwide.
Responsibilities:
General Product quality measurement: Define, invent, and deliver metrics that accurately measure the quality of online information and the satisfaction/success of our customers.
A/B Metrics: Design and implement A/B metrics to evaluate the impact of changes to our products and services. Analyze results to provide actionable insights and drive continuous improvement.
RAG based search experience measurement: Develop and refine metrics to assess the performance and effectiveness of our Generative SERP experience. Ensure these metrics capture user satisfaction, accuracy, and overall interaction quality.
Models: Develop ML/Statistical models to measure/predict the quality of online content and/or user interactions with large-scale AI systems.
Experimental Design: Think critically about experimental design & develop innovative strategies and products in these areas to improve the quality of A/B experimentation
Strategy: Translate strategy into plans that are clear and measurable, with progress shared out to stakeholders.
Cooperation: Partner effectively with program management, engineers, and other areas of the business.
Qualifications:
Required Qualifications:
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay
Microsoft will accept applications for the role until September 27, 2024.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form .
Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.
This role is available in Mountain View, CA or Redmond, WA.
In this role as a Principal Data Scientist, you will work to improve online metrics that evaluate Product quality in A/B testing and non-A/B testing situations. You will apply sophisticated statistical methods and large language models (LLMs) combined with human labels to conduct thorough evaluations of Bing's Search Product through the lens of metrics. Your expertise will guide Bing's product quality assessments, and user satisfaction measurement.
As part of the rapidly advancing Search and AI domain, our team is a leader in applied machine learning at Microsoft. We are committed to providing an unparalleled search and AI assistant experience to over 500 million monthly active users globally through Bing. Additionally, our work impacts other major search engines like Yahoo, DuckDuckGo, and emerging platforms such as Neeva and You.com.
Join us in shaping the future of search and AI, where your contributions will influence the experiences of users worldwide.
Responsibilities:
General Product quality measurement: Define, invent, and deliver metrics that accurately measure the quality of online information and the satisfaction/success of our customers.
A/B Metrics: Design and implement A/B metrics to evaluate the impact of changes to our products and services. Analyze results to provide actionable insights and drive continuous improvement.
RAG based search experience measurement: Develop and refine metrics to assess the performance and effectiveness of our Generative SERP experience. Ensure these metrics capture user satisfaction, accuracy, and overall interaction quality.
Models: Develop ML/Statistical models to measure/predict the quality of online content and/or user interactions with large-scale AI systems.
Experimental Design: Think critically about experimental design & develop innovative strategies and products in these areas to improve the quality of A/B experimentation
Strategy: Translate strategy into plans that are clear and measurable, with progress shared out to stakeholders.
Cooperation: Partner effectively with program management, engineers, and other areas of the business.
Qualifications:
Required Qualifications:
- Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 8+ year(s) data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 10+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 12+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR equivalent experience.
- 5+ years experience programming experience utilizing Python, R or Big Data environments
- Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 10+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 12+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
- OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 15+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results))
- OR equivalent experience.
- Experience with Statistical modeling, A/B experimentation and building metrics for large scale user facing Artificial Intelligence products.
- Experience with human-in-the-loop machine learning.
- Experience with real world system building and data collection, including design, coding and evaluation
- Experience using LLMs (Large Language Models) to build reliable metrics & gather insights
- Customer focused, strategic, drives for results, is self-motivated, and has a propensity for action
- Fantastic problem solver: ability to solve problems that the world has not solved before
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay
Microsoft will accept applications for the role until September 27, 2024.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form .
Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.
Created: 2024-09-24
Reference: 1769247
Country: United States
State: California
City: Mountain View
Similar jobs:
-
Interdisciplinary Scientist (Data Management)
Geological Survey in Santa Cruz, California💸 $74642 per year -
Senior Staff Data Scientist, Product
Google in Sunnyvale, California -
Data Scientist, Analytics
Meta in Menlo Park, California -
Data Scientist, Llama Scalability (Technical Leadership)
Meta in Menlo Park, California -
Data Scientist (TS/SCI)
Piper Companies in San Diego, California -
Senior Data Scientist
State Of California in Sacramento, California -
Language Data Scientist II
Amazon in Santa Clara, California💸 $125500 per year -
REMOTE - INTL - Data Scientist
Insight Global in Livermore, California -
Senior Data Scientist, Ecommerce - USDS
TikTok in Los Angeles, California -
Senior Data Scientist, Ecommerce - USDS
TikTok in Mountain View, California -
AIML - Sr Data Scientist, Siri and Apple Intelligence, Data and ML Innovation
Apple in Cupertino, California -
Data Scientist, Trust and Safety - USDS
TikTok in Mountain View, California -
Senior Data Scientist - ML
Amazon in San Francisco, California💸 $143300 per year -
Data Scientist, Music Search Platform - Science \u0026 Modeling
Amazon in San Francisco, California💸 $97500 per year -
Staff Data Scientist
Intuit in Mountain View, California -
DATA SCIENTIST
Department of the Air Force - Agency Wide in Los Angeles, California -
Data Scientist, Tiktok Ads-Vertical Solutions
TikTok in Mountain View, California -
Data Scientist, Ecommerce - USDS
TikTok in Mountain View, California -
Data Scientist - TikTok E-commerce Governance
TikTok in San Jose, California -
Data Scientist - TikTok Live Streaming & RTC Products
TikTok in San Jose, California