Cloud Data Engineer
Descrição da Empresa
A Olisipo é uma das principais e mais sólidas referências nacionais como talent recruiter, talent builder e talent care na área das tecnologias de informação. A nossa missão é encontrar o melhor projeto para cada pessoa e, para cada empresa, o melhor talento na área das tecnologias de informação. #ConnectingITPeople
Descrição da Função
Profile: - Beetween 3 and 5 years of experience building data pipelines on public cloud platforms (preferably AWS; Azure/GCP also relevant); - More than a year of experience in data solution architecture, design, development, and support - Strong experience in database architecture, ETL, BI, Big Data, ML, and Advanced Analytics; - Hands-on experience with Spark ecosystems (Hadoop is a plus); - Experience with containerized applications (Docker) and Linux-based infrastructure (preferred); - Experience in pipeline orchestration tools (Airflow, Step Functions, AWS workflow); - Expertise in Lakehouse architectures (Iceberg) and optimizations (Compaction, Z-Order, etc.); - Proficient in software development and automation using Python, SQL, SQL Server, SSIS, Docker, and CI/CD practices; - Practical experience with key AWS services: Lambda, Glue, Crawlers, Step Functions, DynamoDB, and AWS Batch; - Knowledge of DevOps and Agile principles, infrastructure as code (CloudFormation, Terraform), and artifact/vulnerability management; - Awareness of cloud foundational services (networking, security, storage, IAM); - Fluency in English and knowledge of French is a plus. Responsibilities: - Advise on data strategy, architecture, and implementation approaches; - Design and develop data pipelines on public cloud platforms (primarily AWS; Azure/GCP when relevant); - Implement and optimize Lakehouse architectures; - Build, automate, and maintain ETL processes, data workflows, and orchestration (Airflow, Step Functions, AWS workflow); - Ensure high standards of reliability, automation, maintainability, and security in all delivered solutions; - Support incident resolution, root cause analysis, and preventive measures when required; - Work closely with delivery managers, architects, project managers, developers, and business analysts in a collaborative setup; - Participate in relevant ceremonies, reviews, and knowledge exchange sessions; - Deliver functional and optimized data pipelines aligned with business and technical requirements; - Document designs, implementations, and operational guidelines; - Provide regular updates on progress, risks, and improvements, and offer training or guidance to internal teams as needed. We offer: - Health insurance; - Free online training through the Udemy platform; - On-site and remote training at Olisipo's Learning Center; - Free certifications (after passing the exam); - Discounts at Olisipo Partners (in the areas of health and well-being, fitness, travel, among others); - Free psychological consultations; - Possibility of a salary advance, without commissions.
Localização
- Porto, Portugal