JB057260 - Sr. Data Engineer

Start Date: Interview Types

Skills Teradata Expertise, .. Visa Types Green Card, US Citiz..

Role: Sr. Data Engineer

Location: Atlanta, GA 30354(Hybrid)



Duration: 12+ Months Contract



Client: Delta Airlines



Local only



Overall 10+Years experience required

Qualifications:



Required Skillsets:

Teradata Expertise:



• In-depth knowledge of Teradata architecture and components (e.g., AMP, Nodes, Parsing Engines, etc.).



• Proficiency in Teradata SQL and Teradata utilities.



SQL and Scripting:



• Strong SQL skills for querying and managing relational databases.



• Scripting experience (e.g., Shell, Python) for automating tasks.



• Know tools like Teradata Utilities (e.g., BTEQ, FastLoad, MultiLoad, TPT, etc.).



ETL Tools:



• Experience with ETL tools and technologies used in Teradata environments.



Performance Optimization:



• Knowledge of query optimization techniques, indexing, partitioning, and workload management in Teradata.



Data Warehousing Concepts:



• Strong understanding of data warehousing principles, including star/snowflake schemas, OLAP, and ETL processes.



Cloud Platform / AWS Expertise:



• Proficiency with core AWS services: EC2, S3, RDS, Lambda, CloudWatch, CloudFormation, etc.



• Experience with AWS networking (VPC, subnets, security groups, NACLs) and security management.



• Familiarity with AWS storage options, such as S3, EFS, and Glacier, in the context of large-scale data storage and retrieval.



Teradata and AWS Integration:



• Experience deploying and managing Teradata in AWS cloud environments.



• Knowledge of AWS services that enhance Teradata performance, such as using S3 for staging data or integrating with Redshift for analytics.



Automation and Infrastructure as Code (IaC):



• Hands-on experience with AWS CloudFormation, Terraform, or similar tools for automating infrastructure deployment.



• Proficiency in automating ETL pipelines and data processing workflows using AWS Lambda, Step Functions, or AWS Glue.



Data Security in AWS:



• Understanding of encryption and security protocols for data stored and processed in AWS.



• Experience implementing compliance standards such as GDPR, HIPAA, or other data protection regulations in cloud environments.



Responsibilities:



Key Responsibilities:

Database Management:



• Design, implement, and maintain Teradata databases.



• Optimize and tune queries for performance.



• Monitor and troubleshoot database issues.



ETL Development:



• Develop and maintain ETL (Extract, Transform, Load) processes for data integration and transformation.



• Use tools like Teradata Utilities (e.g., BTEQ, FastLoad, MultiLoad, TPT, etc.).



Data Modeling:



• Create and maintain data models to support business processes.



• Work with stakeholders to understand data requirements and design solutions.



Performance Tuning:



• Identify and resolve performance bottlenecks in SQL queries, system configuration, or data design.



• Regularly monitor and optimize system performance.



Data Security & Governance:



• Implement data security best practices.



• Ensure compliance with data governance policies and regulations.



Cloud Integration and Management:



• Implement and manage Teradata on AWS infrastructure, leveraging AWS services for scalability and performance.



• Manage data migration from on-premises Teradata environments to AWS cloud-based Teradata instances.



• Ensure seamless integration between Teradata and AWS services like S3, Redshift, RDS, and Lambda.



Cloud Infrastructure:



• Design and deploy cloud infrastructure (VPCs, EC2 instances, S3 buckets) to support Teradata environments.



• Implement networking and security best practices in AWS to ensure secure data access and storage.



Automation and DevOps:



• Automate infrastructure provisioning, scaling, and monitoring using AWS tools like CloudFormation, Terraform, or AWS CLI.



• Use AWS Lambda for automating ETL processes or handling specific workloads.



Data Warehousing and Analytics:



• Leverage AWS data services (e.g., Redshift, Athena, Glue) to complement Teradata workloads.



• Implement data pipelines using AWS services for ETL processes, integrating data from various sources into Teradata.



Security and Compliance:



• Implement AWS security best practices, such as IAM roles and policies, to secure Teradata environments in the cloud.



• Ensure compliance with relevant data governance policies, including encryption at rest and in transit.



Collaboration:



• Work closely with data architects, analysts, and other stakeholders to understand data needs and deliver solutions.