On ArcGIS 9.2, Step 1 of HabModExample2 fails with ArcGISError: ... Invocation of CopyRaster_Management(...) raised RuntimeError: Error in executing function

[jjr8]

Here is an example log:

Executing (Find HDFs and Convert SDS To ArcGIS Rasters (3)): HDFFindAndConvertToArcGISRasters C:\HabModExample2\OceanographyFiles\SST\8Day C:\HabModExample2\OceanographyRasters\SST\8Day\qual qual -180 -90 0.0439453125 # *-qual* false # # # # # # false false false false GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]] # # # # # # # true os.path.join(outputWorkspace, os.path.dirname(inputFile[len(directoryToSearch)+1:]), 'qual' + os.path.basename(inputFile).split(u'.')[0][:7]) os.path true C:\HabModExample2\OceanographyRasters\SST\8Day\qual
Start Time: Tue Nov 11 12:58:08 2008
Running script HDFFindAndConvertToArcGISRasters...
Finding files and inserting rows into table "work": directory="C:\HabModExample2\OceanographyFiles\SST\8Day", wildcard="*-qual*", searchTree=False, minSize=None, maxSize=None, minDateCreated=None, maxDateCreated=None, minDateModified=None, maxDateModified=None
Finished inserting rows: 0:00:00 elapsed, 7 rows inserted, 0:00:00.001142 per row.
Updating field "outputRaster" in table "work"...
Update complete: 0:00:00 elapsed, 7 rows updated, 0 deleted, 0 unchanged, 0:00:00.000571 per row.
Querying the table "work" to build lists of inputs and outputs...
Query complete: 0:00:00 elapsed, 7 rows retrieved, 0:00:00.000285 per row.
Checking for existing outputs for each input...
Finished checking for existing outputs: 0:00:13 elapsed, 7 inputs checked, 0:00:01.862571 per input.
Processing 7 inputs...
Converting SDS "qual" in HDF file C:\HabModExample2\OceanographyFiles\SST\8Day\1999217-1999224.m0483pfv50-qual.hdf to ArcGIS raster C:\HabModExample2\OceanographyRasters\SST\8Day\qual\qual1999217...
Error in executing function
Failed to execute (CopyRaster).
ArcGISError: An ArcGIS geoprocessing function failed. This usually results from a problem with your inputs. Please check them and try again. Also review any preceding error messages and the detailed error information that appears at the end of this message. If you suspect the failure is due to a programming mistake in this tool or ArcGIS, please contact the author of this tool for assistance. Detailed error information: ArcGIS Geoprocessor object 0x015DE1D0: Invocation of CopyRaster_Management(*args=('C:\\Temp\\GeoEcoTemp_Jason\\tmp0cojo0\\output', 'C:\\HabModExample2\\OceanographyRasters\\SST\\8Day\\qual\\qual1999217')) raised RuntimeError: 
Error in executing function
Failed to execute (CopyRaster).
The following consequences resulted from the original error:
Could not copy ArcGIS raster C:\Temp\GeoEcoTemp_Jason\tmp0cojo0\output to C:\HabModExample2\OceanographyRasters\SST\8Day\qual\qual1999217
Could not convert C:\Temp\GeoEcoTemp_Jason\tmpnh7mjn\binary to raster C:\HabModExample2\OceanographyRasters\SST\8Day\qual\qual1999217
Could not convert SDS "qual" in HDF file C:\HabModExample2\OceanographyFiles\SST\8Day\1999217-1999224.m0483pfv50-qual.hdf to ArcGIS raster C:\HabModExample2\OceanographyRasters\SST\8Day\qual\qual1999217
Error in script HDFFindAndConvertToArcGISRasters.
Error in executing: cmd.exe /C C:\Python24\Lib\SITE-P~1\GeoEco\ARCGIS~1\Scripts\HDFFIN~1.PY  "C:\HabModExample2\OceanographyFiles\SST\8Day" "C:\HabModExample2\OceanographyRasters\SST\8Day\qual" "qual" "-180" "-90" "0.0439453125" "#" "*-qual*" "false" "#" "#" "" "" "" "" "false" "false" "false" "false" "GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]]" "#" "#" "#" "#" "#" "#" "#" "true" "os.path.join(outputWorkspace, os.path.dirname(inputFile[len(directoryToSearch)+1:]), 'qual' + os.path.basename(inputFile).split(u'.')[0][:7])" "os.path" "true" "C:\HabModExample2\OceanographyRasters\SST\8Day\qual"

Failed to execute (Find HDFs and Convert SDS To ArcGIS Rasters (3)).
End Time: Tue Nov 11 12:59:54 2008 (Elapsed Time: 1 minutes 46 seconds)

Note that this problem seems to always occur when trying to create the raster C:\HabModExample?2\OceanographyRasters?\SST\8Day\qual\qual1999217.

I was able to develop the following reliable repro scenario:

1. Download the HabModExample?2.zip that was originally released for MGET 0.6.
2. Unzip it to C:\HabModExample?2, as instructed.
3. Download and unzip the OBIS-SEAMAP data, as instructed.
4. Start ArcCatalog.
5. Open the toolbox in C:\HabModExample?2 for Arc 9.2 and Python 2.4.
6. Edit the Step 1 model.
7. Open the Model menu and select Run Entire Model.

This reliably reproduced the problem on two separate machines. It caused a single file to be created: C:\HabModExample?2\OceanographyRasters?\SST\8Day\qual\qual1999217 and then the error happened. Interestingly, this file contained the following:

IMAGINE Grid Creation Placeholder:0

This suggests a strange bug in ArcGIS: rather than trying to create a binary ArcInfo grid (which should be done since the destination raster does not have a file extension), it started to create an IMAGINE file (.img file). Then it failed (I don't know why).

I was not able to come up with a simpler repro scenario. The problem appears to be a bug in ArcGIS 9.2; I will open a separate ticket on this. After I made some additional changes to the model and saved it again, I was not able to reproduce the problem.

I was able to work around the problem by changing the tool to call Copy_management instead of CopyRaster?_management. This is what MGET will now do for ArcGIS 9.2 and prior. Note that, due to #310, CopyRaster?_management will still be called on 9.3.

UPDATE: Due to the problem I described below on 12/08/08, I am reverting this fix for ArcGIS 9.2. Now, ArcGIS 9.1 will call Copy_management, but 9.2 and later will call CopyRaster?_management.

[jjr8]

Fixed in [323], released in MGET 0.7a3.

[jjr8]

It appears that this does not work properly on ArcGIS 9.2. On 9.2, Copy_Management sometimes fails with "System Error (0): Unexpected Error" like this:

Copying ArcGIS raster C:\Temp\GeoEcoTemp_L.dallarosa\tmpirndvn\output to C:\fronts\fronts2\test2...
ArcGIS Geoprocessor object 0x00B5E3F0: Invoking Copy_Management(*args=('C:\\Temp\\GeoEcoTemp_L.dallarosa\\tmpirndvn\\output', 'C:\\fronts\\fronts2\\test2'))...
System Error (0): Unexpected Error.

I observed this sporadically on one of my 9.2 machines but I thought it was a Windows Vista compatibility problem. I also got a report from Luciano Dalla Rosa saying he could reproduce it on demand with a specific scenario on Arc 9.2 SP6. I sent him a private fix that changed Copy_Management back to CopyRaster?_Management and it worked.

Since Luciano and I have encountered this problem in at least two scenarios, and the HabModExample?2 failure only occurred with a specific snapshot of the HabModExample?2 directory structure, I am going to change the code back to use CopyRaster?_Management.

[jjr8]

Luciano's log file showing the failure.

[jjr8]

The updated fix (changing ArcGIS 9.2 back to using CopyRaster?_management) was checked in as [337] and released in MGET 0.7a5.