Software Development Engineer, AGI - Amazonbot web crawler

We are looking for an experienced Engineer to join our AI data gathering team and work on supporting cutting-edge generative AI projects.

As a Software Engineer on the Amazon Web Crawling team, you will be responsible for designing, developing and maintaining the systems that empower Amazon's ability to understand and interact with web content. This position requires creativity, passion and experience building innovative solutions to complex technical problems.

We are looking for someone with experience leading technical reviews with senior leadership who is able to present on a monthly basis. This team has high visibility and requires an engineer that is comfortable moving across teams on 1 month sprints. Those with startup experience encouraged to apply.

Key job responsibilities

Design and develop scalable web crawling and data extraction systems to acquire structured data from websites

Build automation data pipelines and insights using big data frameworks(e.g. spark) to acquire petabytes of data visualize important KPI's which would enable technical direction.

Optimize web crawling architecture for performance, scale, resilience and cost efficiency

Implement robust systems to process high volumes of web content and extract meaning

Develop data pipelines and infrastructure to support petabyte-scale datasets

Work closely with scientists and other engineers to rapidly prototype and deploy new algorithms

Write high quality, well-tested production code in languages like Python, spark, Java, Scala.

BASIC QUALIFICATIONS

3+ years of non-internship professional software development experience

2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience

Experience programming with at least one software programming language

PREFERRED QUALIFICATIONS

3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience

Bachelor's degree in computer science or equivalent

Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.

Pursuant to the Los Angeles Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $129,300/year in our lowest geographic market up to $223,600/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. This position will remain posted until filled. Applicants should apply via our internal or external career site.

---