Ticket #386 (new Defect)

Opened 1 month ago

Last modified 1 month ago

On ArcGIS 9.3 and 9.3.1, Sample Rasters and several other tools sometimes fail immediately with <type 'exceptions.SystemExit'>: 1

Reported by: jjr8 Assigned to: jjr8
Priority: Medium Milestone: 0.8
Component: Core - ArcGIS Interop Version:
Keywords: Cc:

Description

The output in the ArcGIS geoprocessing window is very short:

Executing (Calculate Field Using a Python Expression): FieldCalculateArcGISField H:\Sea_Ice\Geodatabase.mdb\SightingsPoints_NSIDC_Stereo UBIcePolygon row.GPS_Zeit.strftime(r'H:\Sea_Ice\UBremen_Ice_Edge_Shapefiles\ice_edge_%Y%m%d_Edited.shp') "[GPS_Zeit] >= #01-13-2007 00:00:00# AND [GPS_Zeit] <= #01-13-2007 23:59:59#" # # # H:\Sea_Ice\Geodatabase.mdb\SightingsPoints_NSIDC_Stereo
Start Time: Fri Oct 09 16:59:29 2009
Running script FieldCalculateArcGISField...
<type 'exceptions.SystemExit'>: 1
Failed to execute (Calculate Field Using a Python Expression).
End Time: Fri Oct 09 16:59:31 2009 (Elapsed Time: 2.00 seconds)

The tools that are affected are those that instantiate the geoprocessor with COM Automation rather than the arcgisscripting Python module to work around a long-standing bug in ArcGIS in which the arcgisscripting geoprocessor cannot write NULL to a database cursor (see this thread for more).

The affected tools are those that need to write to cursors. In MGET 0.7, they are, alphabetically:

  • Calculate Field Using a Python Expression
  • Calculate Fields Using Python Expressions
  • Classify Oceanic Nino Index (ONI) Episodes in Table
  • Create Fishnet for Points
  • Create Table from ESRL Climate Index Time Series at URL
  • Create Table from ESRL Climate Index Time Series at URLs
  • Create Table from ESRL Climate Index Time Series in Text File
  • Create Table from ESRL Climate Index Time Series in Text Files
  • Get DiGIR Resources as Table
  • Sample Polygons
  • Sample Rasters
  • Sample Rasters Listed in Fields
  • Sample Time Series Table
  • Search DiGIR Records and Create Points

In addition, all of MGET's Statistics tools are affected because the rpy Python module they use to communicate with R also relies on pywin32. Those tools fail with an error message more similar to the one in the traceback below. Because of this, I will open a separate ticket for them.

The Problem

The actual failure occurs in the Python for Windows Extensions (pywin32) Python package, build 213 or later. The underlying traceback looks similar to this:

  ...
  File "C:\Python25\Lib\site-packages\win32com\__init__.py", line 6, in <module>
    import pythoncom
  File "C:\Python25\lib\site-packages\pythoncom.py", line 3, in <module>
    pywintypes.__import_pywin32_system_module__("pythoncom", globals())
  File "C:\Python25\lib\site-packages\win32\lib\pywintypes.py", line 20, in __import_pywin32_system_module__
    if not sys.platform.startswith("win32"):
AttributeError: 'NoneType' object has no attribute 'platform'

The __import_pywin32_system_module__ function in pywin32's pywintypes.h fails when it tries to access the platform attribute of the Python sys module. This should never happen.

pywin32 build 213 introduced a lot of changes, including some relating to Python 3.0 compatibility. My guess is that these changes are not compatible with the unusual way that ArcGIS 9.3.x hosts the Python interpreter (using a "sub-interpreter", presumably for speed).

Determining the root cause will be very difficult. I will attempt to raise this with the pywin32 team, but the only way I know how to reproduce it is by running Python code from ArcGIS. I doubt that the pywin32 developers will be interested in that kind of repro scenario. I am not sure it will be worth it for me to debug this anytime soon, as it will probably require installing a debug build and source of both Python and pywin32.

Workarounds

The problem does not appear to occur with pywin32 build 212 or earlier. If you have a later version, downgrade to 212 and see if the problem goes away.

The failure does not happen all the time. On some machines it is very rare; on others, quite frequent. It seems to occur more frequently when you first run a tool that does NOT use COM Automation (i.e. one that is not listed above) and then try to run one that DOES use COM Automation. If it happens, try closing your ArcGIS application (ArcMap or ArcCatalog) and trying the failing tool again, first, before running any other tools.

Finally, it does not seem to happen if the tool is invoked outside of the ArcGIS GUI applications (ArcMap and ArcCatalog). If you write and run scripts from the Windows command prompt (a.k.a. DOS), you can avoid the failure.

Change History

10/13/09 17:17:58 changed by jjr8

Starting with release 0.8 alpha 2, MGET will attempt to detect this problem and report a more helpful error message:

RuntimeError: Failed to instantiate the ArcGIS geoprocessor using COM Automation. (Due ArcGIS bug NIM010734, this tool is required to instantiate the geoprocessor with COM Automation rather than the arcgisscripting Python module.) The pythoncom module from the "Python for Windows Extensions" package (also known as "pywin32") could not be imported due to an incompatibility between recent releases of pywin32 and ArcGIS. You may be able to work around this problem temporarily by restarting ArcGIS and running this tool again (do not run any other tools before running this one). For a more permanent solution, uninstall your existing copy of pywin32 and install build 212 instead. Finally, please see http://code.env.duke.edu/projects/mget/ticket/386 for the current status of this problem.

I am leaving this ticket open for now.