Oceanographic data and information management in the Mediterranean and Black Seas: recent advances and future perspectives

E. Balopoulos(1) and C. Maillard(2)

1. National Centre for Marine Research, Institute of Oceanography, P.O. Box 712, 190 13 Anavyssos, Greece.
2. IFREMER/SISMER, Centre de Brest, BP 70, Plouzane, FRANCE.

Extended abstract

The Mediterranean and the Black Seas have been the subjects of several international initiatives related to their oceanographic data and information management. This is particularly true of the last decade during which large-scale projects, supported by international organizations [e.g., European Union (EU), Intergovernmental Oceanographic Commission of UNESCO (IOC/UNESCO)] were implemented in the above-mentioned regions. These projects were either specifically devoted to oceanographic data and information management, e.g., the projects MODB, MEDATLAS, MEDAR/MEDATLAS, EURONODIM, SEASEARCH, GODAR etc., or were research projects which included a strong data management component, such as the EU Mediterranean Targeted Project (MTP), the EU/MATER Projects, etc.

In the early 90’s the Mediterranean Ocean Data Base (MODB) initiative supported by the EU provided the marine community with a comprehensive data set of temperature and salinity profiles for the Mediterranean Sea (including 9.915 CTD casts, 25.568 bottle casts and also 82.201 XBT and MBT Stations). In addition the MODB developed temperature and salinity climatological fields for the above-mentioned region and also advanced software products for data analysis and visualization.

The MEDATLAS Project (1994-1997), combining the efforts of the National Oceanographic Data Centres (NODCs) of France, Greece and Spain and using the IOC/IODE channels, provided the user community with an updated and fully quality-controlled data base of temperature and salinity profiles of the Mediterranean Sea (including 15.533 CTD casts, 29.048 bottle data and also 75.507 XBT 81.464 MBT profiles) and compiled a revised climatological statistics for the region. This data product made available to the user community through a set of three (3) CD-ROMs is internationally accepted today as a powerful tool, especially for the marine science community. The CD-ROMs also provide software for visualization, selection and extraction of the data using multiple selection criteria.

The EU/MATER database (1996-2000), made available to the user community through a CD-ROM includes data collected in the Mediterranean Sea, during 102 sea cruises, carried out within the framework of the basin-scale research Project MATER (Mass transfer and ecosystem Response). The database consists of a great diversity of data types, (including 3.141 CTD profiles, 62 floating buoy time series, 1.727 bottle data, 13 thermistor time series, 359 ADCP vertical profiles, 110 sediment trap time series, 447 ADCP time series, 634 XBT profiles, 210 current time series, 70 sediment core data), and software for visualization and extraction and extraction of the data.

Major data management activities in the Black Sea include the development of the Black Sea regional historical interdisciplinary database. This was created within the framework of the NATO TU-Black Sea Project (1994-1997). It includes all main physical, chemical and biological variables for the entire Black Sea basin and covers the most crucial period in the history of the Black Sea ecosystem, starting from the “background” situation in 1960, up to the drastic changes which have occurred in recent years (26.035 stations, 8.364.731 data vales). All data have been quality checked by qualified groups of regional experts, well acquainted with the Black Sea data. Numerical values are quality-flagged using the MEDAR/MEDATLAS flagging system.

Recently (1998-2001), the MEDAR/MEDATLAS initiative coordinated by IFREMER/SISMER, extended the positive experience gained within the MEDATLAS Project to the entire Mediterranean and Black Sea communities. Furthermore, it enlarged the Mediterranean database, including bio-chemical parameters, and greatly improved the spatial and temporal resolution of the statistics, by adding other data not yet included in the past programmes. In this effort there was participation from data centres and specialized marine institutes from 20 countries and the international organizations EU, IOC/UNESCO and ICES. The MEDAR/MEDATLAS Project enabled the development of a multi-parameter database of high quality oceanographic and marine data sets for the Mediterranean and Black Seas. This includes the following parameters: water temperature (284.371 profiles), salinity (118.009 profiles), CTD (35.679 profiles), Dissolved Oxygen (44.928 profiles), XBT and MBT (161.848 profiles), pH (14.512 profiles), Alkalinity (2.548 profiles), Phosphate (20.761 profiles), Nitrate (10.572 profiles), Nitrite (10.508 profiles), Ammonium (5.239 profiles), Silicate (15.920 profiles), HSUL (1.843 profiles), NTOT (153 profiles), TPHS (2.381 profiles). This database is made available to the user community in the form of four (4) CD-ROMs. The CD-ROMs also include annual, seasonal and monthly climatology for temperature and salinity, annual and seasonal climatology for oxygen, silicate and phosphate, as well as annual climatology for nitrate, nitrite, PH, ammonium, alkalinity and chlorophyll. In addition the CD-ROMs include software for visualization, selection (using multiple selection criteria) and extraction (in three different formats) of the data.

Concerning oceanographic and marine metadata, several directories for the Mediterranean Sea have been developed within the framework of the European-scale Project EURONODIM (1998-2001). These directories are currently updated and further expanded within the framework of the Pan-European-scale Project SEASEARCH, in which 31 countries participate, including all the countries of the Black Sea and almost all the countries of the Mediterranean region.

All the above-mentioned initiatives contributed to the collection of a large amount of historical and recent data gathered in the Mediterranean and Black Seas. However, a major achievement of the afore-mentioned projects is the establishment of a network of data centres and specialised marine institutes, which is mainly based on the IOC/IODE network and during the last few years has been one of the most active data management networks in the world. The MEDATLAS and MEDAR/MEDATLAS projects in particular, established the contacts and created trust between data managers and scientists across the region. Their main products were only possible by establishing common standards for data handling across the region. The MEDATLAS data format, based on IOC/IODE standards, has become one of the most useful, flexible and commonly used oceanographic data formats anywhere on the Globe. Indeed it has earned itself a top place in the hierarchy of the most commonly accepted data formats. MEDATLAS also created evolving data management quality control standards, compatible with existing international systems and has captured these standards in a software system called SCOOP, which was developed by IFREMER/SISMER and is freely available to the oceanographic community.

However, in all the above-mentioned initiatives the approach followed for data and metadata management is a centralized one. Data or metadata or both coming from various sources are accumulated for two or three years as a centralized data or metadata holding and then at the end of the Project are made accessible to the wide user community through a set of CD-ROMs or Web sites. In recent years and due to technology development there are more and more supporters of the view that such an approach today is slow and inflexible and by no means meets the needs for fast access to data and metadata. Back in 1994 it was envisaged that the ultimate outcome of the MEDATLAS initiatives would become the establishment of a distributed oceanographic data network for the Mediterranean and Black Sea. It was also believed that such a network would become a model for the development of other regional networks and ultimately a distributed global system of data centres. The Mediterranean and Black Sea data management communities now feel ready to start with the final steps towards a truly distributed oceanographic data system for the region. Back in 1994 there was no real understanding as to how such a goal could be achieved, but technology, in particular web technologies, wrapped around rapidly evolving meta-language systems, including the Extensible Markup Language XML, now provide solutions.

Building on advanced technology developments, a new regional data management system [MEDBLACK-ODN (Mediterranean and Black Seas Oceanographic Data Network)] has been designed by data centres of the Mediterranean and Black Seas. This is based on a further expansion of the regional data network established by the MEDAR/MEDATLAS project, so as to include partners from all the Mediterranean and Black Sea countries. The data management approach of the system is based on a decentralized (“virtual”) model. According to this, data remain at the data sources (national or regional archiving systems) and the query software is accepted by the local security systems. Any user request addressed to the system via an Internet-accessible “virtual” data centre, is automatically forwarded by the web interface to each data repository and to the real time data structures of the operational oceanography. The results from each centre are aggregated in order to provide a fully integrated product, including the selected data, their documentation and the location maps. In this case data are not duplicated, and their immediate availability is guaranteed. However, since data sources are independent, their integration needs sophisticated methods and navigation tools to facilitate access to data from decentralized data sources. This requires the development and use of techniques and tools, many of which have a strong innovative character. The proposed system will provide for the first time an integrated access to both historical data and near-real time data of the operational oceanography. Furthermore, it will contribute to the development of new technological systems and standards in order to get timely and secure access to data.

Therefore, the new data management system for data management of the Mediterranean and Black Sea will be an Internet-accessible and searchable system for making available a large volume of a wide range of fully quality-controlled multi-parameter oceanographic and marine data for the Mediterranean and Black Seas. Its friendly visual user interface will ensure multipurpose use of the system allowing a large variety of users from different sectors to perform individual searches without being experts on data base use. In the same way users will be able to download the data sets or data products they need and use them for their own purpose. This system based mainly on national oceanographic data archiving systems of the Mediterranean and Black Seas countries will ensure more effective use not only of the European infrastructure but also of the infrastructures of other countries bordering the Mediterranean and Black Seas. Furthermore, the establishment of common standards and data management practices (based on international standards) used by the national archiving systems of the Mediterranean and Black Sea countries ensures availability of high quality data and enhances international co-operation, necessary for the sustainable development of the regions. Moreover co-operation of the MEDBLACK-ODN with international organizations (e.g., IOC, ICES, EU, etc.), as well as with leading data management Centres from the NIS States (World Data Centre B) and the USA (World Data Centre A) will confirm the international role of the European marine data management activities.

Undertaking this task in the Mediterranean and Black Seas is particularly challenging and, if successful, the rewards will be immense. The Mediterranean and Black Seas are surrounded by more than 30 different countries with almost as many different languages. There is also a huge variation in scientific capacity across the region, both in terms of human resources and technical ability. This new regional data management model will attempt to tackle both the technical solutions required for a truly distributed system, but will also take a further step to allow all countries/institutes in the region to be involved at a similar level. Thus a prime aim of this initiative is to increase national capacity so that all partners have equal capability. Ultimately, it is envisaged that each country or institute will retain “physical ownership” of its own data, taking care of updates to its data (e.g., after quality control cycles), be responsible for its continuous archival and documentation, and have control over access to the data. Achieving this objective will imply the end of ‘conventional’ oceanographic data centres as we know them today. Instead the distributed system will offer opportunities for much faster access as required by operational needs, and the elimination of data duplication (the problem of the same data being available in different forms in different data sets).