Report on Data Format Session at 2001 IAU WG Meeting

The preliminary version of documentation of the format for the interchange of calibrated optical/IR data being developed by Tom Pauls of NRL and John Young of COAST was presented and a number of issues discussed.

The proposed format is in the general form of a relational database with different components stored in FITS binary tables. The different components are related by means of indices into other tables. For example the target being observed is given as an index in the target table.

The discussion points were the following:

1. Should there be a single data table type that includes all data types, or multiple table types, one per data type?
2. What data types are needed? Currently considered are Complex visibilities, Squared visibilities and triple products.
3. Should Virtual columns be used? If all entries in a column are the same, they can be replaced by a keyword. This trades storage aginst complexity. If this is to be implemented, how will it be done?
4. Array geometry (OI_ARRAY) table. Should this be a single table with all possible, nominal, locations or a separate table per configuration giving the actual locations. The difference is mostly of use for astrometry.
5. Should the allowable visibility units include physical units such as Janskys as well as normalized units. For circumstellar disks and nonstellar objects normalized visibilities are hard to define.

Discussion

Items 1 and 2

These were discussed jointly. It was generally concluded that there should be a separate table type for each data type. There was no general sentiment in favor of additional types at present but it will be straightforward to implement new data types as new table types. It was also concluded that the names of the tables needed to be explicitly for Optical/IR interferometry data. The new names are the following:

Item 3

It was concluded that there was no strong argument for using virtual keywords and the simplicity of not using them more than made up for possible increases in storage space.

Item 4

The discussion of this issue was inconclusive.

Item 5

There was a general consensus that physical units as well as normalized units should be allowed by the format definition.

Other issues

Preben Grosbol (ESO) expressed concern that the relational database aspect of the format might be disrupted if constituent tables were moved to separate files. A possible solution is to include a unique identifier in each data set to establish which sets of tables go together.

Action Items

• Tom Pauls (NRL) is to be the administrative secretary for the data interchange format discussion.
• The current document will be modified following the discussion at the meeting and recirculated. The time scale for the recirculation and comments is the end of 2001.
• A library of reading and writing software will be circulated.
• Test data will be circulated amoung Alain Chelli (JMMC), Pascal Ballester (ESO), John Young (COAST) and Tom Pauls/Christian Hummel (NRL/NPOI).
• A set of imagable data derived from undisclosed models will be generated by COAST and NPOI and circulated as the basis of a "beauty contest" of imaging algorithms to be held at the next meeting of the working group. Bill Cotton has agreed to judge.
• Use of the format for astrometric purposes will be investigated. Francoise Delplanck (ESO) has agreed to help.

Bill Cotton