IPY Discussion Forums 2012-02-08T09:57:32Z Copyright (c) 2009 ExpressionEngine tag:classic.ipy.org,2009:05:06 ISO to DIF crosswalk tag:classic.ipy.org,2009:start/index.php/forums/viewthread/.109 2009-05-06T21:09:16Z 2012-02-08T09:57:32Z Mark A. Parsons Andrew Wolf writes:

We recently took a look at developing an ISO 19115 profile for GCMD DIF
metadata, as part of the UK Natural Environment Research Council’s
metadata activities. A first report is available:
http://proj.badc.rl.ac.uk/ndg/export/5244/TI01-discovery/trunk/doc/NDG%20ISO%20metadata%20v1.0.pdf
The crosswalk was mostly straightforward, but a number of issues were
identified (and outlined in the ‘Issues’ section, p24-25).

Any feedback welcome.

Regards,
Andrew

-----------------------------------------------------
Dr Andrew Woolf ()
Environmental e-Science and Spatial Informatics
STFC e-Science Centre
Rutherford Appleton Lab
Chilton, Didcot, Oxon., OX11 0QX, UK
Ph: +44 (0)1235 778027
http://www.e-science.stfc.ac.uk/organisation/staff/andrew_woolf

]]> Some issues concerning DIF and GCMD as IPY metadata hub tag:classic.ipy.org,2008:start/index.php/forums/viewthread/.86 2008-12-11T21:01:50Z 2012-02-08T00:40:17Z Øystein Godøy The national IPY data management in Norway is organised in a project (DOKIPY) where databases at Institute of Marine Research (IMR), Norwegian Polar Institute (NPI) and Norwegian Meteorological Institute (METNO) are synchronised at the metadata level using OAI-PMH serving DIF. We do also plan to use OAI-PMH to synchronise IPY metadata with GCMD. None of us, except for the NPI, has previous experience in using DIF. Thus we have had some interesting discussions, especially concerning how to map between various metadata standards. We have also had a dialogue with Melanie at GCMD on these issues. She has approved that we continue this discussion within this forum. In the present situation we have 2 main issues for discussion:

Incomplete DIF records
In at least one of the databases we have several metadata elements lacking proper DIF information as these were received prior to the IPY specification. We will of course edit these as soon as possible, but have decided to share this information at the national level (sometimes only minor information is lacking) although these records cannot be handled by GCMD before they are made complete. We will add the dummy text - Not Available to the elements not properly covered.

Do anyone else experience this problem and if so, how is it handled? This is however, not a big issue and will be solved by time anyway.

Changes in keywords
A bit more problematic in the current situation is the GCMD handling of the controlled vocabularies being used. In our discussion with GCMD we received the following information:

2) “How often do you see keywords change?”

GCMD: We have made additions to the keywords (note every time we add a new platform or instruments, that is a change to keywords). We still are actually quite dynamic (with additions). We expect to make significant Science Keyword changes to the list in 2009. Changes may include (1) renaming keywords, (2) deleted keywords, (3) adding new keywords, (4) changing where the keyword(s) lies in the hierarchy, (5) combining keywords.

What really worries us here is the problem of mapping GCMD keywords to other controlled vocabularies (e.g. Science Keywords to CF standard names) when GCMD plans to rename, delete or change location of keywords. In an operational context this creates a huge task maintaining the mapping between DIF and other standards. None of our databases use DIF internally, we all map to (and from) DIF.

We believe that will pose a problem for other IPY data management services as well. How do you plan to handle it? Could we make a coordinated effort towards GCMD concerning the procedures for changing keywords? XML-representations of the changes? This kind of changes to a controlled vocabulary may cause a lot of extra work.

We are however happy with the GCMD plans to assign identifiers to the controlled vocabularies. That will certainly make life easier in the future.

4) “To manage updates to keywords (both simple changes to science keywords, data center names, project names, and other) I think we would need each name to refer to an ID. The get_valid service (for example: http://gcmd.gsfc.nasa.gov/OpenAPI/get_valids.py?type=datacentervalid) could have an extra column that would give us the ID of the data center. At regular intervals we could then download the complete list and process it. We would use the ID to replace the names with already have in our database with the new ones. As the service is today we do not know if any of the names should replace names that we already have. I guess more complex changes (one data center splits into two, or similar) could be handled by email. ”

GCMD: We will be glad to inform you of changes by email. We plan to assign a specific identifier to each keyword in the future.

We would like some feedback on how other IPY data centres are handling these issues and not least how it should be handled within IPYDIS.

]]> GCMD Science Keywords updates tag:classic.ipy.org,2008:start/index.php/forums/viewthread/.85 2008-12-09T20:56:01Z 2012-02-07T22:04:15Z Mark A. Parsons The parameter fields in the profile require the use of the controlled vocabulary of the GCMD Science Keywords

The GCMD expect to make significant changes to the list in 2009. Given that semantics are one of the more challenging aspects of interoperability and interdisciplinary work, it would be wise to monitor and contribute to the GCMD’s development. I have always found them to be very responsive to the community.

The best way to keep abreast of GCMD activity is through the CEOS Interoperability Forum

-m.

]]> NetCDF/CF tag:classic.ipy.org,2008:start/index.php/forums/viewthread/.82 2008-11-16T09:48:04Z 2012-02-07T18:19:47Z Øystein Godøy Do anybody know whether anyone have used NetCDF/CF for biological/chemical data? I am trying to wrap some ocean biological and chemical observations (e.g. pigments) into NetCDF/CF. The main challenge is the standard names. I may tag them with GCMD science keywords, but would like to use CF descriptions as well if these are available somewhere.

]]> GCMD DIF tag:classic.ipy.org,2008:start/index.php/forums/viewthread/.81 2008-11-06T18:34:25Z 2012-02-08T03:42:43Z Øystein Godøy We use DIF to exchange metadata and have run into a problem concerning existing datasets and databases that in some circumstances contain incomplete records for them to fulfil the DIF requirements. While attempting to complete the records we would still like to export/show them. Do anybody know whether there is a standard way of describing missing values/elements within DIF?

]]>