A Simple Example: The HDF SDS to ArcGIS Raster Tool

Many oceanography products are published in HDF format but ArcGIS. Arc 9.1 cannot read HDF. Arc 9.2 is supposed to support HDF but my experience has not been very good. For example, I have been unable to get it to read PO.DAAC QuickSCAT ocean winds HDFs, and it is extremely slow at reading NOAA NODC 4 km AVHRR Pathfinder SST HDFs. It also appears that Arc cannot detect the projection used by HDF files, which is not surprising due to the lack of a widely-adopted standard for HDF metadata attributes. As a result, you must copy the HDF to some other format, shift and rescale the data, and define the projection yourself.

HDF conversion utilities exist but as far as I've seen, none are available as geoprocessing tools. The MGET HDF SDS to ArcGIS Raster tool will convert a Scientific Data Set in an HDF file to ArcGIS raster format. It automatically detects the data type and dimensions from the HDF metadata but you must provide the coordinates of the lower-left corner, the cell size, and the NODATA value. These values may also be present in the metadata as attributes but there is no reliable naming convention for these attributes so the tool does not attempt to guess what they might be.

Invoking the tool from ArcGIS

Example output

Here is 199001.s04m1pfv50-sst-16b.hdf from the NOAA NODC 4 km AVHRR Pathfinder Project in ArcMap, zoomed to the southeastern United States.

Invoking the tool from Python

Each tool is exposed as a Python method. To invoke the tool, you simply import the appropriate module and call the method.

from GeoEco.DataManagement.HDFs import HDF

HDF.SDSToArcGISRaster(u'c:\\temp16\\199001.s04m1pfv50-sst-16b.hdf', u'c:\\temp16\\sst199001',
                      u'sst', -180.0, -90.0, 0.0439453125, buildPyramids=True, overwriteExisting=True)

For brevity, this and the following examples omit the defineProjection parameter (the long projection string does not show up well in the Trac Wiki system).

Invoking the tool through Windows COM Automation

On Windows, each tool is exposed as a COM Automation class. This allows easy invocation from many programming environments, for example:

VBScript

Set hdf = WScript.CreateObject("GeoEco.HDF")

hdf.SDSToArcGISRaster "c:\\temp16\\199001.s04m1pfv50-sst-16b.hdf", "c:\\temp16\\sst199001", "sst", -180.0, -90.0, 0.0439453125)

JScript

var hdf = WScript.CreateObject("GeoEco.HDF");

hdf.SDSToArcGISRaster("c:\\temp16\\199001.s04m1pfv50-sst-16b.hdf", "c:\\temp16\\sst199001", "sst", -180.0, -90.0, 0.0439453125);

R

First install the rcom package so you can call COM Automation components from R. Then:

> library(rcom)
> hdf <- comCreateObject("GeoEco.HDF")
> comInvoke(hdf, "SDSToArcGISRaster", "c:\\temp16\\199001.s04m1pfv50-sst-16b.hdf", "c:\\temp16\\sst199001", "sst", -180.0, -90.0, 0.0439453125)
NULL
> 

MATLAB

>> hdf = actxserver('GeoEco.HDF');
>> hdf.SDSToArcGISRaster('c:\\temp16\\199001.s04m1pfv50-sst-16b.hdf', 'c:\\temp16\\sst199001', 'sst', -180.0, -90.0, 0.0439453125, 0.0);
>>

Invoking the tool through Windows COM

Each tool is also exposed on Windows as a normal COM class using a dual interface. For example, the interface to the HDF class is defined in IDL as follows:

[
    dual,
    uuid(478E9F8D-0D41-4630-9D7B-271B2D47CB9E),
    helpstring("IHDF Interface"),
    pointer_default(unique)
]
interface IHDF : IDispatch
{
    // Methods
    [id(1), helpstring("ExtractHeader Method")] HRESULT ExtractHeader([in] BSTR inputFile, [in] BSTR outputFile, [in, defaultvalue(0)] VARIANT_BOOL overwriteExisting);
    [id(2), helpstring("GetSDSDataType Method")] HRESULT GetSDSDataType([in] BSTR inputFile, [in] BSTR sdsName, [out, retval] BSTR *dataType);
    [id(3), helpstring("GetSDSDimensions Method")] HRESULT GetSDSDimensions([in] BSTR inputFile, [in] BSTR sdsName, [out, retval] VARIANT *dimensions);
    [id(4), helpstring("GetSDSNames Method")] HRESULT GetSDSNames([in] BSTR inputFile, [out, retval] VARIANT *sdsNames);
    [id(5), helpstring("SDSToArcGISRaster Method")] HRESULT SDSToArcGISRaster([in] BSTR inputFile, [in] BSTR outputRaster, [in] BSTR sdsName, [in] double xLowerLeftCorner, [in] double yLowerLeftCorner, [in] double cellSize, [in, optional] VARIANT nodataValue, [in, optional] VARIANT coordinateSystem, [in, optional] VARIANT buildPyramids, [in, optional] VARIANT overwriteExisting);
    [id(6), helpstring("SDSToArcInfoASCIIGrid Method")] HRESULT SDSToArcInfoASCIIGrid([in] BSTR inputFile, [in] BSTR outputFile, [in] BSTR sdsName, [in] double xLowerLeftCorner, [in] double yLowerLeftCorner, [in] double cellSize, [in, optional] VARIANT nodataValue, [in, optional] VARIANT overwriteExisting);
    [id(7), helpstring("SDSToBinaryRaster Method")] HRESULT SDSToBinaryRaster([in] BSTR inputFile, [in] BSTR outputFile, [in] BSTR sdsName, [in, defaultvalue(0)] VARIANT_BOOL overwriteExisting);
};

The IDL is automatically generated during the Windows installation package build procedure, and is compiled into a COM type library which is registered during installation. This allows early-bound languages such as C# to instantiate the classes and invoke the methods through normal COM (a "vtable call"):

using GeoEco;

namespace Example
{
    class Program
    {
        static void Main(string[] args)
        {
            HDF hdf = new HDF();

            hdf.SDSToArcGISRaster("c:\\temp16\\199001.s04m1pfv50-sst-16b.hdf",
                                  "c:\\temp16\\sst199001",
                                  "sst",
                                  -180.0,
                                  -90.0,
                                  0.0439453125,
                                  null,
                                  null,
                                  null,
                                  null,
                                  null,
                                  null,
                                  null,
                                  true,
                                  true);
        }
    }
}

Before compiling this code in Microsoft Visual Studio, you must add a reference to the COM component named "GeoEco X.Y Type Library", where X and Y are the MGET version numbers