Description

The Shopping Tech Foundation Team is looking for a Language Data Scientist to collaborate in developing solutions for LLM prompt engineering, LLM evaluation/benchmarking, and annotation efficiency. This position is an opportunity to apply your linguistic and data science expertise in a challenging but supportive environment. Do you want to be part of the team developing the future technology that impacts the customer experience of ground-breaking products? Then come join us and make history. Our team works on a variety of projects, including state of the art generative AI, LLM finetuning, alignment, prompt engineering, benchmarking solutions. We are customer obsessed and committed to delivering results with the highest quality and integrity.

Responsibilities

As a Language Data Scientist, you will: * Start by diving deep into a couple of critical LLM related projects. * Collaborate with fellow applied scientists, language data scientists, program managers, as well as stakeholders in engineering, annotation operation teams, and product teams to understand the role data plays in developing models that meet customer needs. * Analyze, follow, and improve processes for collecting, assessing and improving LLM inputs and outputs, and automating where appropriate. * Apply state-of-the-art Generative AI techniques to analyze how well our data represents human language and run experiments to gauge downstream interactions. * Work collaboratively with other language data scientists and scientists to design and implement principled strategies for data optimization.

Key Job Responsibilities

• Source, validate, and deliver high-quality language model artifacts, and linguistic data
• Collaborate with stakeholders to design data collection and LLM development efforts
• Oversee the progress and quality of several data collection, model development and annotation projects at a time
• Advocate for strict adherence to data guidelines and quality thresholds
• Extend existing data collection, annotation, and quality assurance efforts to support feature and language expansion
• Innovate on data collection and LLM finetuning/prompt engineering methodologies, guidelines, quality metrics to support new requests
• Automate repetitive workflows and improve existing processes

Work Location

We are open to hiring candidates to work out of one of the following locations: * Santa Clara, CA, USA * Seattle, WA, USA

Basic Qualifications

• 2+ years of data scientist experience
• 3+ years of data querying languages (e.g. SQL), scripting languages (e.g. Python) or statistical/mathematical software (e.g. R, SAS, Matlab, etc.) experience
• 3+ years of machine learning/statistical modeling data analysis tools and techniques, and parameters that affect their performance experience
• Experience applying theoretical models in an applied environment
• Master's degree in a quantitative field such as statistics, mathematics, data science, business analytics, economics, finance, engineering, or computer science

Preferred Qualifications

• Experience in Python, Perl, or another scripting language
• Experience in a ML or data scientist role with a large technology company
• Knowledge of relevant statistical measures such as confidence intervals, significance of error measurements, development and evaluation data sets, etc.

Amazon Commitment

Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.

Compensation

Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $125,500/year in our lowest geographic market up to $212,800/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits.

Job ID

A2649441