Internship: Data engineering project - Design & setup data pipelines & data lakes AI/ML processing - Ieper, Belgium
Internship: Data engineering project: Design and setup data pipelines and data lakes for AI/ML processing
Your future job
More and more Artificial Intelligence / Machine Learning capabilities
are being developed to optimise the business processes and to get more insight
in the big amount of data that is produced.These AI/ML engines need data
provided in a way that is suited for their specific application.
Within this project, you will foresee in the provisioning of the data via data pipelines, in the data conversion, enrichment, consolidation,... and the storage of it in a cloud data lake and possible other needed big data stores (e.g. BigQuery).
Additionally, every step in the setup has to be monitored for proper functioning. Also data visualisation will be needed.
- bachelor or master in IT
Main technologies used:
- Java (Maven, sprint boot, apache camel, IntelliJ,…)
- Python, Django, GraphQL
- Google Cloud Dataflow, Google data lakes
Google Composer (Airflow in the cloud), Google BigQuery, Google Data Studio
- ETL (Talend)
- Test automation based on Cucumber and Selenium
- Microservices and event-based application architecture (using event brokers like ActiveMQ and Google Cloud Pub/Sub)
- Local and Google Cloud infrastructure (apps engine, BigQuery, Google Data Studio,…)
- Source code management, continuous delivery pipelines, service mgmt platforms (Git/gitlab CI, Docker, Kubernetes,….)
- a challenging job in a dynamic high-tech international environment
- the opportunity to take ownership of your professional passion in order to contribute to the success of the company
- an enjoyable, team-oriented and professional atmosphere in a flat-structured organization
- versatile development opportunities