Descrição da Empresa

A Olisipo é uma das principais e mais sólidas referências nacionais como talent recruiter, talent builder e talent care na área das tecnologias de informação. A nossa missão é encontrar o melhor projeto para cada pessoa e, para cada empresa, o melhor talento na área das tecnologias de informação.   #ConnectingITPeople

Descrição da Função

Profile: - Bachelor’s degree in Computer Science, Engineering, or a related field; - Minimum of 5 years of professional experience in Software Engineering or Data Engineering; - Strong programming skills in Python within large-scale, high-performance production environments; - Hands-on experience with big data frameworks such as Apache Spark and PySpark; - Solid expertise in data modeling and working with structured and unstructured data; - Experience with streaming platforms, particularly Apache Kafka; - Strong understanding of distributed systems and modern data architectures; - Experience with GCP (valued); - Proven experience in large-scale data platform migrations from Cloudera (CDH/HDP) to Snowflake or Databricks; - Strong expertise in building and orchestrating data platforms from scratch, including integrations such as CDC, Reverse ETL, and API ingestion; - Familiarity with data governance and data integration tools; - Experience with microservices development in Python; - Knowledge of CI/CD pipelines, data versioning, and experiment tracking; - Experience with tools such as Git and ETL frameworks like Airflow, dbt, or Talend (nice to have); - Experience with NoSQL databases such as Redis or Neo4j and data lake architectures (nice to have); - Knowledge of machine learning workflows and algorithms (nice to have); - Strong problem-solving skills and ability to work in complex, data-intensive environments; - Excellent communication skills and ability to collaborate with cross-functional teams; - Fluency in English (mandatory). Responsibilities: - Design, develop, and maintain scalable data processing pipelines using frameworks such as Apache Spark, PySpark, and Apache Beam; - Build and maintain Python-based microservices to support data-driven features in production; - Develop internal tools to support CI/CD processes, experiment tracking, and data versioning; - Collect, process, and integrate large datasets from multiple sources, including databases, APIs, and file systems; - Ensure data integrity, consistency, and quality through validation and monitoring processes; - Optimize data systems for performance, scalability, and high availability; - Implement best practices for data security, privacy, and access control; - Collaborate with data scientists, analysts, and engineers to support analytics and machine learning workflows; - Lead and execute data platform migration initiatives from on-premise Cloudera environments to cloud-native solutions such as Snowflake or Databricks; - Re-architect legacy data workloads, including Hive and Impala, and migrate HDFS-based data; - Build and scale greenfield data platforms, including infrastructure, ingestion layers, transformation frameworks, and integrations; - Implement advanced data integration patterns such as CDC, Reverse ETL, and real-time ingestion pipelines; - Ensure minimal downtime and zero data loss during migration and deployment processes; - Contribute to continuous improvement of data architecture, tools, and best practices. We offer: - Health insurance; - Free online training through the Udemy platform; - On-site and remote training at Olisipo's Learning Center; - Free certifications (after passing the exam); - Discounts at Olisipo Partners (in the areas of health and well-being, fitness, travel, among others); - Free psychological consultations; - Possibility of a salary advance, without commissions.

Localização

• Lisboa, Portugal