Applied Scientist, Conversational Assistant Services
Boston, Massachusetts
As part of Alexa CAS team, our mission is to provide scalable and reliable evaluation of the state-of-the-art Conversational AI. We are looking for a passionate, talented, and resourceful Applied Scientist in the field of LLM, Artificial Intelligence (AI), Natural Language Processing (NLP), to invent and build end-to-end evaluation of how customers perceive state-of-the-art context-aware conversational AI assistants.
A successful candidate will have strong machine learning background and a desire to push the envelope in one or more of the above areas. The ideal candidate would also have hands-on experiences in building Generative AI solutions with LLMs, including Supervised Fine-Tuning (SFT), In-Context Learning (ICL), Learning from Human Feedback (LHF), etc.
As an Applied Scientist, you will leverage your technical expertise and experience to collaborate with other talented applied scientists and engineers to research and develop novel methods for evaluating conversational assistants. You will analyze and understand user experiences by leveraging Amazon's heterogeneous data sources and build evaluation models using machine learning methods.
Key job responsibilities
- Design, build, test and release predictive ML models using LLMs
- Ensure data quality throughout all stages of acquisition and processing, including such areas as data sourcing/collection, ground truth generation, normalization, and transformation.
- Collaborate with colleagues from science, engineering and business backgrounds.
- Present proposals and results to partner teams in a clear manner backed by data and coupled with actionable conclusions
- Work with engineers to develop efficient data querying and inference infrastructure for both offline and online use cases
About the team
Central Analytics and Research Science (CARS) is an analytics, software, and science team within Amazon's Conversational Assistant Services (CAS) organization. Our mission is to provide an end-to-end understanding of how customers perceive the assistants they interact with - from the metrics themselves to software applications to deep dive on those metrics - allowing assistant developers to improve their services. Learn more about Amazon's approach to customer-obsessed science on the Amazon Science website, which features the latest news and research from scientists across the company. For the latest updates, subscribe to the monthly newsletter, and follow the @AmazonScience handle and #AmazonScience hashtag on LinkedIn, Twitter, Facebook, Instagram, and YouTube.
BASIC QUALIFICATIONS
- PhD, or Master's degree and 4+ years of CS, CE, ML or related field experience
- Experience building machine learning models or developing algorithms for business application
- Experience in patents or publications at top-tier peer-reviewed conferences or journals
- Experience with programming languages such as Python, Java, C++
- Hands-on experience and deep understanding of the Large Language Models architectures
PREFERRED QUALIFICATIONS
- Research experience in conversational assistant or LLM evaluation
- Publications at peer-reviewed NLP/ML conferences (e.g. ACL, EMNLP, NAACL, NeurIPS, ICLR, ICML, AAAI, etc.)
- Hands-on experience with using RLHF models
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
Los Angeles County applicants: Job duties for this position include: work safely and cooperatively with other employees, supervisors, and staff; adhere to standards of excellence despite stressful conditions; communicate effectively and respectfully with employees, supervisors, and staff to ensure exceptional customer service; and follow all federal, state, and local laws and Company policies. Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position. These include the duties and responsibilities listed above, as well as the abilities to adhere to company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the Company's reputation. Pursuant to the Los Angeles County Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $136,000/year in our lowest geographic market up to $222,200/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. This position will remain posted until filled. Applicants should apply via our internal or external career site.
A successful candidate will have strong machine learning background and a desire to push the envelope in one or more of the above areas. The ideal candidate would also have hands-on experiences in building Generative AI solutions with LLMs, including Supervised Fine-Tuning (SFT), In-Context Learning (ICL), Learning from Human Feedback (LHF), etc.
As an Applied Scientist, you will leverage your technical expertise and experience to collaborate with other talented applied scientists and engineers to research and develop novel methods for evaluating conversational assistants. You will analyze and understand user experiences by leveraging Amazon's heterogeneous data sources and build evaluation models using machine learning methods.
Key job responsibilities
- Design, build, test and release predictive ML models using LLMs
- Ensure data quality throughout all stages of acquisition and processing, including such areas as data sourcing/collection, ground truth generation, normalization, and transformation.
- Collaborate with colleagues from science, engineering and business backgrounds.
- Present proposals and results to partner teams in a clear manner backed by data and coupled with actionable conclusions
- Work with engineers to develop efficient data querying and inference infrastructure for both offline and online use cases
About the team
Central Analytics and Research Science (CARS) is an analytics, software, and science team within Amazon's Conversational Assistant Services (CAS) organization. Our mission is to provide an end-to-end understanding of how customers perceive the assistants they interact with - from the metrics themselves to software applications to deep dive on those metrics - allowing assistant developers to improve their services. Learn more about Amazon's approach to customer-obsessed science on the Amazon Science website, which features the latest news and research from scientists across the company. For the latest updates, subscribe to the monthly newsletter, and follow the @AmazonScience handle and #AmazonScience hashtag on LinkedIn, Twitter, Facebook, Instagram, and YouTube.
BASIC QUALIFICATIONS
- PhD, or Master's degree and 4+ years of CS, CE, ML or related field experience
- Experience building machine learning models or developing algorithms for business application
- Experience in patents or publications at top-tier peer-reviewed conferences or journals
- Experience with programming languages such as Python, Java, C++
- Hands-on experience and deep understanding of the Large Language Models architectures
PREFERRED QUALIFICATIONS
- Research experience in conversational assistant or LLM evaluation
- Publications at peer-reviewed NLP/ML conferences (e.g. ACL, EMNLP, NAACL, NeurIPS, ICLR, ICML, AAAI, etc.)
- Hands-on experience with using RLHF models
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
Los Angeles County applicants: Job duties for this position include: work safely and cooperatively with other employees, supervisors, and staff; adhere to standards of excellence despite stressful conditions; communicate effectively and respectfully with employees, supervisors, and staff to ensure exceptional customer service; and follow all federal, state, and local laws and Company policies. Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position. These include the duties and responsibilities listed above, as well as the abilities to adhere to company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the Company's reputation. Pursuant to the Los Angeles County Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $136,000/year in our lowest geographic market up to $222,200/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. This position will remain posted until filled. Applicants should apply via our internal or external career site.
Created: 2024-09-11
Reference: 2742001
Country: United States
State: Massachusetts
City: Boston
ZIP: 02120
About Amazon
Founded in: 1994
Number of Employees: 1600000
Website: https://www.amazon.com/
Career site: https://www.amazon.jobs/en/
Instagram: https://www.instagram.com/amazon/
LinkedIn: https://www.linkedin.com/company/amazon/
Facebook: https://www.facebook.com/Amazon
Similar jobs:
-
Nurse Practitioner I or Physician Assistant Cardiac Arrythmia service Inpatient
Massachusetts General Hospital(MGH) in Boston, Massachusetts -
Food Service Assistant - 15 hrs.
Cooley Dickinson Hospital (CDH) in Northampton, Massachusetts -
EMERGENCY SERVICES ASSISTANT / 32 HOURS / EVENING SHIFT
Brigham & Women's Hospital(BWH) in Boston, Massachusetts -
Assistant Director of Nursing Services (ADNS)
Athena Health Care Systems in Leeds, Massachusetts💸 $12.00 per hour -
Medical Assistant, Physician Services - East Sandwich
Spaulding Hospital Cape Cod(SRN and SCC) in East Sandwich, Massachusetts -
EMERGENCY SERVICE ASSISTANT / 40 HOUR / DAYS- BWH EMERGENCY DEPT
Brigham & Women's Hospital(BWH) in Boston, Massachusetts -
Client Services Assistant Manager - Foster Hospital for Small Animals
Tufts University in N. Grafton, Massachusetts💸 $26.70 per hour -
EMERGENCY SERVICES ASSISTANT / 36 HOURS / EVENING SHIFT
Brigham & Women's Hospital(BWH) in Boston, Massachusetts -
Nurse Practitioner I or Physician Assistant Cardiac Arrythmia service Inpatient
Massachusetts General Hospital(MGH) in Boston, Massachusetts -
EMERGENCY SERVICE ASSISTANT / 36 HOUR / NIGHTS- BWH EMERGENCY DEPT
Brigham & Women's Hospital(BWH) in Boston, Massachusetts -
Medical Assistant MA, Worcester Medical Center Preadmission Services
UnitedHealth Group in Worcester, Massachusetts -
EMERGENCY SERVICES ASSISTANT / 40 HOURS / DAY SHIFT
Brigham & Women's Hospital(BWH) in Boston, Massachusetts -
Medical Assistant- Wellesley College Health Services
Newton-Wellesley Hospital(NWH) in Newton, Massachusetts -
Assistant Director of Nursing Services (ADNS)
Frontier Management in Harwich, Massachusetts -
Referral Services Center Liaison Assistant
Mass General Brigham Healthcare at Home in Waltham, Massachusetts -
EMERGENCY SERVICE ASSISTANT / 36 HOUR / EVENINGS- BWH EMERGENCY DEPT
Brigham & Women's Hospital(BWH) in Boston, Massachusetts -
EMERGENCY SERVICE ASSISTANT / 40 HOUR / EVENINGS- BWH EMERGENCY DEPT
Brigham & Women's Hospital(BWH) in Boston, Massachusetts -
Cardiac Services Assistant (Per Diem)
Cape Cod Healthcare Inc. in Falmouth, Massachusetts -
Assistant Professor, Human Services
MASS in Fitchburg, Massachusetts -
Assistant Director, Part-Time - Career Services, The Fletcher School
Tufts University in Medford, Massachusetts💸 $58750.00 per year