This position is no longer accepting applications.
It stays published for reference. Browse open web3 jobs or similar roles below.
Data Engineer
Aston
Remote
middle
14 days ago
full-time
closed
Responsibilities
- Implement high-load data processing pipelines to ensure reliable and uninterrupted data replication from IT systems.
- Solve complex data preparation tasks in target analytical storage (DataLake, SandBox, FeatureStore) for feature engineering needed for machine learning model development.
- Develop and maintain documentation for the developed functionality.
- Timely reflect task statuses in Jira.
- Check the quality of code written by data engineers.
Requirements
- 4+ years of experience as a Data Engineer.
- Excellent knowledge of Python: data structures and algorithms, OOP, and functional programming.
- Experience in writing unit and integration tests, knowledge and experience with NumPy, Pandas.
- Experience in developing and implementing services for loading and processing unstructured data (text, xml, json) from external sources.
- Ability to understand data provider APIs using available documentation.
- Excellent knowledge of SQL: complex queries using analytical window functions, profiling tools for performance optimization.
- Experience with Oracle, Postgres, Greenplum databases.
- Experience with workflow engines for batch data processing (Airflow).
- Experience in developing complex, high-load data processing applications based on PySpark.
- Experience with Apache Kafka: producers/consumers in Python, consumer groups.
- Experience in developing streaming data processing services using Spark Structured Streaming, Flink.
- Experience with Flink.
Conditions
- Full-time employment.
- Remote work.
- 5/2 work schedule.