IPY Discussion Forums

Status of Swedish IPY Data Repository

2010-03-18T11:46:59Z

The number of datasets in the Swedish IPY Data Repository (http://ipy.smhi.se) has increased slowly. Theres is now metadata for 58 datasets in our repository. Most of those datasetsonly only include metadata but no data are available. But many of the publishers have marked the data as “Data Set Progress = Complete” and give a link to their web-site (or to our OpenDAP-service). Then a user might think that on this web-site there is data available. But for most of the datasets this is not true. This might be frustrating for the user, who search for data. The definitiion of “Data Set Progress = Complete” is “Complete refers to data sets in which no updates or further data collection will be made”. This definition does not says that data shall be made availble for external usage. Maybe a better definition would be “Completed refers to data sets in which no updates or further data collection will be made and the data are made available for external usage”???

Of course one possibility is not insert the link (Related URL) until data are available. But reserachers often have interesting web-sites and it might be of general interest to present the link to this web-site even before data is available.

Any comments or ideas?

Review of GEO portals

2009-03-12T16:41:20Z

Some IPY projects have been trying to make use of developing components in GEOSS. The portals they are developing could be particularly helpful. So one way that each of you may become involved is through reviewing the candidate portal prototypes for GEOSS. The portals continually evolve so if you have already submitted a review and some time has passed, you are encouraged to see the progress that has been made. To view the portals, visit http://earthobservations.org/gci_gp.shtml. All feedback can be submitted online using a form on the portals page.

-m.

Opportunities to promote IPY data

2008-09-11T19:08:19Z

I will be giving two presentations in the coming months on the status of IPY data. One will be part of a “key session” at CODATA, the other as part of a Union session at the very large American Geophysical Union conference.

I would like to highlight the successes of how IPY data are being managed well. This is a great opportunity to promote your tools, interfaces, data sets, etc. that support IPY.

Please send me things you’d like to promote--images, slides, web sites, etc. ()

I will be sure to provide full attribution and credit, of course.

-m.

Data Archiving

2008-06-11T21:38:34Z

As I have been contacting and engaging with the various partners in the IPYDIS, much of our attention has been on sharing metadata with some discussion of interoperability. We have scarcely begun to address the big problem of long term preservation of the data. Yet this will be central to the IPY legacy.

A few DIS partners have asked about data archiving, and a number of World Data Centers are contributing. Yet as I said to one IPY colleague, this is a problem for all of humanity. It is much bigger than IPY. It is incumbent upon us all to learn more about data archiving for the long term and sahre thos experiences across IPY, which may serve as an example for the rest of Earth system science.

A colleague asked for some general information on data archiving and preservation, and I provided the following references. Perhaps they are of use. What other resources are out there? What are individual projects doing to ensure the long term preservation of and access to their data?

Here’s some background material on data archiving.

The benchmark is:

CCSDS (Consultative Committee for Space Data Systems). 2002. Reference Model for an Open Archival Information System (OAIS) CCSDS 650.0-B-1 Issue 1. Washington, DC: CCSDS Secretariat. Available at http://public.ccsds.org/publications/archive/650x0b1.pdf.

Stemming from this is the PREMIS (PREservation Metadata: Implementation Strategies) http://www.oclc.org/research/projects/pmwg/
This contains more details on implementation. It is largely oriented to the library community, but NSIDC has been exploring it for developing more complete preservation metadata describing fixity, provenance, and context.

Cornell University developed a nice online tutorial on digital preservation. See http://www.icpsr.umich.edu/dpm/.
This is a good place to start.

Finally, my colleague, Ruth Duerr, recently gave a lecture on “preparing for ingest” which was really about all the things you need to think about when acquiring a data set for archiving. See ftp://sidads.colorado.edu/pub/ppp/conf_ppp/Duerr/Preparing_Data_for_Ingest.ppt

-m.

IPYDIS Town Hall Meeting

2007-08-30T17:38:42Z

Last week was the IPY GeoNorth Conference (http://ess.nrcan.gc.ca/ipygeonorth/). I used this opportunity to host an informal 1.5 hour discussion (a town hall meeting) on the evolution of the IPYDIS. Here are the results of that discussion.

Attendees
Mark Parsons, National Snow and Ice Data Center (Co-Chair IPY Data Policy and Management Subcommittee)
Ellsworth LeDrew, Univ. Waterloo (Co-Chair, Canadian IPY Data Committee)
Peter Pulsifer, Carlton Univ.
Jagtar Bhath, Canadian Forest Service
Bea Csatho, Univ. of Buffalo
Tony Schenk, Ohio State Univ.
Peter Ng, Government of the North West Territories
Karen Edwards, Canadian IPY Secretariat
Jean Brodeur, Natural Resources Canada (involved with North American Profile)
Yves Crevier, Canadian Space Agency
Fraser Taylor, Carlton Univ.
Doug Nebert, US Geological Survey (involved with GEOSS interoperability efforts)

Background
I presented a general overview on what the IPYDIS is and what it is trying to accomplish. I also provided background in the following topics, all of which elicited discussion. Particular topics discussed included
- the IPY Union Catalog
- the IPY metadata profile (http://ipydis.org/metadata.html)
- Communication mechanisms
- a possible IPYDIS directory
- the role of the Data Committee and data coordinators
- upcoming events

Union Catalog
Everybody generally agreed with the general approach to sharing metadata through a union catalog. There was some discussion on the protocol for sharing (e.g. OAI-PMH), but it was agreed that simpler is better. Note: I currently have a project that shares metadata between four data centers by simply transmitting FGDC formatted XML. This will link with the IPY portal at the GCMD.

If you are interested in harvesting or serving metadata, please let me know.

Metadata Profile
Again this was generally accepted. The need to validate and provide immediate feedback after entering a record was emphasized

Most discussion centered around controlled vocabulary, especially for parameters and location. We are currently using GCMD vocabulary for these fields, but I suspect that will evolve. On suggestion was to take a “folksonomy” approach and mine free test keywords.

There was also some discussion about registering the vocabulary, linking to the North American Profile, and working with “organically grown commons-based” approaches such as WordNet (http://wordnet.princeton.edu/). Unfortunately, my notes are rather sketchy here. Can anyone provide more details?

The Marine Metadata Interoperability (MMI) project (http://mmi.org) might also be useful here.

Communication
I presented options of email, this forum, a wiki, and social networks. None were received with great acclaim. This forum was agreed as a simple method that maintained a record.

IPYDIS Directory
I presented the possibility of a directory of IPYDIS participants and this was well received if designed as a “directory of experts”. Stay tuned for developments in this area.

Data Committee
To date the Data Committee has been emphasizing the data policy of open access and working to identify what data are to be collected. Town hall participants felt that there is now a need to focus on the preservation of the data collected as part of IPY. (This was brought up in the general session at GeoNorth too). The Data Committee should develop a “digital archiving strategy” and work with the JC and our sponsors to encourage broad adoption and support. The general idea is that data archiving should be mandatory for IPY participants. We need to identify what projects are currently doing for data archive and what the should be doing. The concept of a consistent publishing framework was emphasized. This will be a topic at the next Data Committee meeting.

A related topic was to develop a small brochure or flyer that demonstrates what projects need to do to ensure their data are preserved and accessible. One participant described this as a brief cookbook or recipe projects could follow. It should not only describe what projects should do, but how they benefit.

Priorities
It was agreed that the top priorities for the near term are to identify what data are being collected and to enable basic discovery of these data using the IPY profile BUT to gain buy in we need to emphasize effective data access mechanisms and capture best practices.

I think the new IPYDIS web site (coming soon) will be an excellent means to highlight some of the cool data tools and access mechanisms. For example, I will highlight NSIDC’s Atlas of the Cryosphere (http://nsidc.org/data/atlas/index.html). PolarView (http://www.polarview.org/) is another good example. What else is out there?