Data Architect
About the Knowledge Center:
The Knowledge Center, located in Gdańsk, is a hub for innovation and develop state-of-the-art business and technology solutions to help us navigate the future. And we want to do that together with you.
Team and Project overview:
We aim to build a big data analytics platform for a global shipping company - Hapag-Lloyd AG. In Gdańsk, we established a Knowledge Center. Within it - Solutions for Analytics (SFA). A group of top IT experts, designers, and executors, will apply AWS cloud technology in the maritime and shipping industry. We are quality-driven and have a passion for cloud (especially AWS), analytics, and the power of positive changes, technology solutions bring to business globally. Data Architect will work closely with data analysts, data engineers, data scientists, and other stakeholders to design and maintain data models.
Responsibilities:
- Develop a reference architecture for our new technology software solutions
- Design secure, scalable, and highly available cloud infrastructure on AWS
- Recommend and assist the cross-functional development team in the automation, improvement, and management of the ETL (Extract, Transform and Load) and ELT (Extract, Load and Transform) work
- Data structure reviews, achieving the highest possible system performance by conducting software optimizations
- Continually identifying and implementing improvements through technical debt reduction
- Meeting regulatory compliance standards
Requirements - must have:
- A minimum of 3 years of experience in a similar role
- Bachelor’s degree in computer science, computer engineering, or relevant field
- Designing diagrams (draw.io or similar, knowledge of The Theory of modern big data warehousing
- Experience with AWS services: S3, EC2, Lambda, SNS, SQS, Athena Glue, KMS/HMS (or similar)
- In-depth understanding of Snowflake, Data Bricks/Spark
- Proven work experience in the development, management, and monitoring of data pipelines
- Comfortable in one or more of Java, Scala, or Python with the ability to be ‘hands on when required
- Working with unstructured, semi-structured, and structured data
Desirable Skills:
- Experience and passion to work in the DevOps approach
Requirements – nice to have:
- Data Integration tool (IICS)
- BI tools (QlikView/Qlik Sense)
- Scheduling (currently Airflow)
- CI/CD (Terraform, Docker, GitLab)
- Replication layer (DMS, Qlik Replicate
- Distributed streaming (Kafka)
- SQL and NoSQL databases
We offer: