Data Engineer Tech Lead
Descrição da Empresa
A Olisipo é uma das principais e mais sólidas referências nacionais como talent recruiter, talent builder e talent care na área das tecnologias de informação. A nossa missão é encontrar o melhor projeto para cada pessoa e, para cada empresa, o melhor talento na área das tecnologias de informação. #ConnectingITPeople
Descrição da Função
Profile: - Bachelor's or Master's degree in Computer Engineering, Computer Science, or equivalent professional experience; - At least 4 years of experience in Data Engineering, with at least 2 years in Tech Lead or similar roles; - Solid experience in AWS, namely Lambda, Glue, Step Functions, CloudFormation, and CloudWatch; - Strong knowledge of cloud solution architecture; Proficiency in Python and SQL; - Experience with MySQL, PostgreSQL, Redshift databases and NoSQL concepts; - Experience in workflow orchestration (Apache Airflow, AWS Step Functions); - Knowledge of ETL and Big Data, with experience in Kafka being an advantage; - Experience or familiarity with Iceberg Tables; - Strong awareness of production, monitoring, and troubleshooting; - Good communication skills, critical thinking, and technical leadership orientation; - English level B2 or higher. Responsibilities: - Lead the design, implementation, and maintenance of scalable data solutions on AWS and/or Azure, aligned with the defined architecture; - Use AWS services such as Lambda, Glue, Step Functions, CloudFormation, and CloudWatch to build robust pipelines and solutions; - Develop and optimize data pipelines, ETL processes, and orchestration workflows using Apache Airflow and AWS Step Functions; - Define architectures considering scalability, performance, security, and cost optimization; - Design and optimize data models for high query performance; - Write and optimize SQL queries, identifying and resolving performance bottlenecks; - Manage data integration in Redshift, MySQL, and PostgreSQL; - Work with large volumes of data in Amazon S3, using Athena for interactive analysis; - Implement logging, tracing, and monitoring mechanisms (CloudTrail, AWS X-Ray); - Translate business requirements into technical tasks and distribute work across the team; - Provide technical leadership, mentoring, and support to the engineering team; - Create technical documentation and deliver internal training; - Challenge business requirements and propose innovative solutions to improve efficiency and performance. We offer: - Health insurance; - Free online training through the Udemy platform; - On-site and remote training at Olisipo's Learning Center; - Free certifications (after passing the exam); - Discounts at Olisipo Partners (in the areas of health and well-being, fitness, travel, among others); - Free psychological consultations; - Possibility of a salary advance, without commissions.
Localização
- Porto, Portugal