Big Data Expert
You will have the opportunity to provide your expertise in the area of the Big Data technologies (e.g. ApacheStorm, Apache Kafka, etc.) expected to be used for orchestrating the EUMETSAT Polar System – Second Generation (EPS-SG) processing chains. These processing chains will be exercised through the EPS-SG Payload Data Acquisition and Processing (PDAP) sub-segment which will be part of the EPS-SG Overall Ground Segment (OGS).
You will also provide your engineering support to RHEA business related to BIG DATA and our internal projects.
The scope of work will include:
- Familiarize with the relevant components of the EPS-SG programme, the design of the EPS-SG Overall Ground Segment (OGS) and its PDAP sub-segment with a focus on the DPI sub-system.
- Provide expertise and advice on Big Data Technologies used by DPI.
- Contribute to the review of DPI related technical documentation.
- Support to follow the development and testing of the DPI.
- Investigate possible further optimizations of the DPI design.
- Support to assessment of EPS-SG Operational Processors.
- Provide training on Big Data Technologies.
- Provide support to the RHEA business related to BIG DATA and our internal projects.
The following skills and experience are mandatory:
- You have a proven in-depth understanding of Big Data Technologies, including as a minimum the following ones:
- Apache Kafka;
- Apache Storm;
- You have experience in programming using Java programming language;
- You have experience with data processing of large data volumes, including distributed resource management tools;
- You have proven experience in providing similar expertise;
- You have proven Project Management skills demonstrated through successful completion of similar projects;
- You have a proven ability to communicate in English, as all communication with EUMETSAT will be in English.
The following skills would be highly desirable:
- You have experience in design and development of operational processors for Earth Observation missions;
- You have a good understanding of other Big Data Technologies, such as but not limited to:
- Apache Spark;
- Apache Flink;
- Apache Heron;
- You have experience with programming using scripting languages, such as Python, Perl, Shell;
- You have software engineering experience for data processing, preferably for satellite data;
- Retrieving, handling and manipulating different data formats, e.g., NetCDF-4, HDF5, GRIB, BUFR and binary data;
- Database (relational/non-relational) implementation and maintenance preferably using PostgreSQL, Oracle or MongoDB.