Big Data Engineer | HirePartners.lt

Your main task will be:

Contribute expertise to the overall data eco-system’s engineering best practices, standards, and architectural approaches.
Collaborate in authoring, reviewing, and approving data requirements and designs, including ETL, data movement, pipelines, business intelligence, and analytics.
Work alongside a team of engineers to build and maintain a data platform and Master Data platform.
Participate in creating architecture and solution blueprints to meet business requirements.
Collaborate with the project management team to develop the overall implementation solution plan and actively contribute to project life cycle phases within the solution area.
Continuously monitor and improve the performance of data systems, ensuring they are optimized for speed, reliability, and scalability.
Ensure data quality and integrity by establishing data governance best practices, including data validation, transformation, and security protocols.
Work closely with business and technology representatives to gather functional and technical requirements.
Stay informed about the latest technology trends, particularly in the areas of data platforms.

Contribute expertise to the overall data eco-system’s engineering best practices, standards, and architectural approaches.
Technologies such as PySpark, Impala, Hive, HDFS, ClickHouse, Zabbix, and Airflow.
Collaborate with the project management team to develop the overall implementation solution plan and actively contribute to project life cycle phases within the solution area.
Continuously monitor and improve the performance of data systems, ensuring they are optimized for speed, reliability, and scalability.
Ensure data quality and integrity by establishing data governance best practices, including data validation, transformation, and security protocols.
Work closely with business and technology representatives to gather functional and technical requirements.
Stay informed about the latest technology trends, particularly in the areas of data platforms.

Job Requirements

1+ years of experience in data engineering or a similar technical role.
Focus on technologies such as PySpark, Python, Linux, HDFS, Elasticsearch, ClickHouse, and various SQL databases.
Experience with version control systems, particularly Git, for code collaboration and tracking changes.
Proficiency in working with an on-premises Cloudera DataLake environment.
Understanding of AWS, NiFi is a plus.
Knowledge of SQL for working with databases.
Experience with ETL processes and tools, such as Apache Airflow.
Strong analytical skills, with the ability to solve complex technical issues and optimize data systems.
Experience with Docker is beneficial.
Understanding of data warehouse concepts and experience working with dbt.
Knowledge of data quality assurance and governance best practices.
Understanding of data privacy and security considerations.
Effective communication skills to collaborate with cross-functional teams and stakeholders.
Strong problem-solving skills and the ability to troubleshoot issues in data pipelines.
Eagerness to learn and adapt to new technologies and industry trends in data engineering.
Familiarity with Agile methodologies and ability to work in a collaborative team environment.
Proficiency in English.
Willingness to learn and share expertise with others.

As part of our client team You:

Have freedom to implement best ideas
Be a part of an inspiring working culture that values high standards
Be an owner of your flexible working hours and results
Have FUN at team building activities and workshops
Personal phone with unlimited calls, sms and internet, mobile portable internet, Go3 TV on your phone and TV
Additional health and accident insurances, accumulation of tier III pensions
Benefit from other motivational perks such as healthy snacks and fruits, coffee, team lunches and much more!

Gross salary 2000-3900 EUR/month