Job reference: VN22-39
Location: Reading, UK or Bonn, Germany
Deadline for applications: 07/08/2022
Publication date: 13/06/2022
Salary and Grade: Grade A2: £63,782.28 or EUR 78,702.36 basic salary, net of tax + other benefits
Contract type: STF-PL /STF-PS
Department: Forecast
Contract Duration: 1.5 and 4 years, with possibility of extension
Job summary
Post-processing of raw forecast output data into refined products is critical for users of ECMWF’s forecasts. It is, however, a significant computational and I/O challenge with severe scalability implications as the number of customers, number of generated products and volume of processed data continue to rise. The upcoming scientific ambition of the centre will lead to order of magnitude increases in the volume of data that needs to be processed. We expect raw data volumes to reach 1 PiB per forecast cycle (four times daily) as we plan for Exascale HPC weather forecasting by 2030.
The Data Processing Services team, within the Development Section in the Forecast Department, is responsible for the design and development of all software and systems related to ECMWF’s post-processing services. This includes specialist software for on-the-fly post-processing and product generation in ECMWF’s critical meteorological pipelines, as well as data services for on-demand data extraction, particularly in support of ECMWF’s cloud activities.
The scalability, performance, and robustness of the full, time-critical operational weather forecasting pipeline in the face of future model upgrades is a significant focus of the team. As such, the team actively engages in research of novel, potentially disruptive, algorithms and methods to achieve these objectives, and actively works to evaluate upcoming hardware options. It also invests in the analysis and development of new data processing paradigms in support of novel data-centric workflows which will enable new insights into ECMWF’s massive data sets. As part of this research effort, the team often seeks and secures external funding (e.g., via EU Horizon programme) to supplement investments in the areas of data handling, I/O technologies and scalable workflows.
The team is currently in the process of rebuilding ECMWF’s post-processing suites with the aim of higher performance and better resource efficiency. We use the very latest technologies and apply modern software development practices to ensure the software is highly maintainable and ultra-reliable, ready for 24/7 time-critical operations.
This vacancy is for two positions, one supporting the EU Horizon programme (ACROSS/IO-SEA/EUPEX projects) and the other the Copernicus programme.
The successful candidates will be responsible, as part of a team, for modernising a range of processing stacks downstream of the forecast model. This includes, for example, those used internally by the Evaluation Section for statistical analysis and climatology. This work will initially focus on upgrading the existing post-processing tools into a harmonised Python eco-system, interfacing dedicated C++ libraries, and improving their scalability for future resolution upgrades of ECMWF’s forecasts. There will be very close collaboration with the analysts of the Evaluation Section and Production Section, as well as other members of the Development Section.
Responsibilities
Modernize and support the operational post-processing suite
Support the adoption of the new post-processing suite into operations
Develop and maintain ECMWF’s ecosystem of interoperable Python-based post-processing packages and their underlying C++ libraries
Develop and test novel I/O, parallelization and data processing strategies, and the exploration of new hardware options
Contribute to on-going EU projects and to securing external funding for novel technologies and algorithms for scalable post-processing
About ECMWF
ECMWF is the European Centre for Medium-Range Weather Forecasts. It is an intergovernmental organisation created in 1975 by a group of European nations and is today supported by 34 Member and Co-operating States, mostly in Europe. The Centre’s mission is to serve and support its Member and Co-operating States and the wider community by developing and providing world-leading global numerical weather prediction. ECMWF functions as a 24/7 research and operational centre with a focus on medium and long-range predictions and holds one of the largest meteorological archives in the world. The success of its activities relies primarily on the talent of its scientists, strong partnerships with its Member and Co-operating States and the international community, some of the most powerful supercomputers in the world, and the use of innovative technologies such as machine learning across its operations.
Over the years, ECMWF has also developed a strong partnership with the European Union, and for the past seven years has been an entrusted entity for the implementation and operation of the Climate Change and the Atmosphere Monitoring Services of the EU’s Copernicus component of its Space Programme, as well as a contributor to the Copernicus Emergency Management Service. The collaboration does not stop there and includes other areas of work, including High Performance Computing and the development of digital tools that enable ECMWF to extend its provision of data and products covering weather, climate, air quality, fire and flood prediction and monitoring.
ECMWF will be a major partner in the implementation of the Destination Earth (DestinE) initiative, together with ESA and EUMETSAT as partners. The objective of the European Commission DestinE initiative is to deploy several highly accurate thematic digital replicas of the Earth, called Digital Twins, to monitor and predict natural and human activities as well as their interactions, to develop and test scenarios that would enable more sustainable developments and support corresponding European policies for the Green Deal.
ECMWF has recently become a multi-site organisation, with its headquarters based since its creation in Reading, UK, its new data centre in Bologna, Italy, and new offices in Bonn, Germany.
It has also recently adopted a hybrid organisation model which allows its staff to mix both office working and teleworking. This generous and flexible model provides our staff with considerable flexibility to spend time outside or away from their duty station and decide how they wish to manage their professional working time at ECMWF. ECMWF is an organisation that values the whole being and both understands and values the need for flexibility in the way its staff work.
For additional details, see www.ecmwf.int.
Personal attributes
Excellent analytical and problem-solving skills with a proactive, continuous improvement approach
Initiative and ability to work collaboratively with other ECMWF teams but also able to work independently
Good interpersonal and communication skills
Dedication, passion, and enthusiasm to succeed both individually and across teams of developers
Highly organised with the capacity to work on a diverse range of tasks to tight deadlines
Education
Advanced university degree (EQ7 level or above) or equivalent professional experience in computer science or engineering, computational science, physics or natural sciences, mathematics, or a related discipline.
Experience
Demonstrated experience developing large scale, object-oriented software in Python
Demonstrated experience developing software to process large datasets, with strong understanding of I/O concerns and access to data storage
Experience in C++ development and interoperability between Python/C++ is a strong advantage
Experience in parallelization and scalable software design, particularly within an HPC environment, with a good understanding of performance-related issues, would be a distinct advantage
Experience developing or operating software in high-availability operational environments would be advantageous
Knowledge, skills, languages
Competence in Object-Oriented Analysis and Design
Familiarity with software lifecycle maintenance is highly desirable
Demonstrated ability programming in UNIX/Linux
Ability to write software in an HPC or scientific computing environment would be advantageous
Candidates must be able to work effectively in English and interviews will be conducted in English
Good knowledge of one of the Centre’s other working languages (French or German) would be an advantage
Other information
Grade remuneration: The successful candidates will be recruited at the A2 grade, according to the scales of the Co-ordinated Organisations and the annual basic salary will be EUR 78,035.40 or GBP 63,782.28 net of tax. ECMWF also offers a generous benefits package, including a flexible teleworking policy. This vacancy is for two positions with candidates to be as assigned to the employment categories STF-PL (4 year contract) or STF-PS (contract up to 31 March 2024) as defined in the ECMWF Staff Regulations. Full details of salary scales and allowances available on the ECMWF website at www.ecmwf.int/en/about/jobs, including the ECMWF Staff Regulations and the terms and conditions of employment.
Start date: As soon as possible
Length of contract: The contract duration for the two Research Software Engineer positions is either 4 years with the possibility of contract extension, depending on requirements and funding availability, or approximately 1.5 years until 31 March 2024, with the possibility of further contract extensions in the future depending on requirements and funding availability.
Location: The positions can be based at ECMWF’s duty stations in Reading, UK, or in Bonn, Germany. As a multi-site organisation, ECMWF has adopted a hybrid work model which allows flexibility to staff to mix office working and teleworking. Successful applicants and members of their family forming part of their households will be exempt from immigration restrictions.
Interviews will be conducted by videoconference (MS Teams).
Who can apply
Applicants are invited to complete the online application form by clicking on the apply button.
At ECMWF, we consider an inclusive environment as key for our success. We are dedicated to ensuring a workplace that embraces diversity and provides equal opportunities for all, without distinction as to race, gender, age, marital status, social status, disability, sexual orientation, religion, personality, ethnicity and culture. We value the benefits derived from a diverse workforce and are committed to having staff that reflect the diversity of the countries that are part of our community, in an environment that nurtures equality and inclusion.
Applications are invited from nationals from ECMWF Member States and Co-operating States, as well as from all EU Member States:
ECMWF Member and Co-operating States are: Austria, Belgium, Bulgaria, Croatia, Czech Republic, Denmark, Estonia, Finland, France, Georgia, Germany, Greece, Hungary, Iceland, Ireland, Israel, Italy, Latvia, Lithuania, Luxembourg, Montenegro, Morocco, the Netherlands, Norway, North Macedonia, Portugal, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey and the United Kingdom.
In these exceptional times, we also welcome applications from Ukrainian nationals for this vacancy.
Applications from nationals from other countries may be considered in exceptional cases.