….some further experiments with ipythonand ArcGIS python geoprocessing. Here is a simple script to convert a bunch of coverages to shapefile. There are 9 coverages occupying 3mb, it takes about 6 minutes, consumes 2 processors to 80-90% capacity and chews through 300mb of ram:
cd w:/Env-dat.003/2007-March/workspace/envy_ed2import arcgisscripting gp = arcgisscripting.create() gp.Workspace = ‘./’ todo = !dir /b fwtc* for cov in todo: try: print cov gp.FeatureClassToShapefile( cov + ‘/arc’, ‘./shp’) except: print gp.GetMessages()
Contrast that to using fwtools ogr2ogr, which takes 10 seconds:
for %a in (fwtc*) do ogr2ogr -f “esri shapefile” ogr\%a %a
The problem is, ogr2ogr doesn’t currently translate everything arcpy does. It leaves behind the metadata and fills voids (though these can be filtered out later as the attributes are empty, but that is an extra step). There’s also no way (that I can see) to select only arcs or only points.
(June 6th) another episode in the ogr vs arcgis battle:
Goal: Project 70 shapefiles in various subdirectories occupying 300mb.
ArcGIS BatchProject interactively:
- search catalog for k:\incoming\NHN*.shp
- fire up Batch Project toolbox
- drag’n'drop search results (wait 2+ minutes for verification step to complete)
- oops, error in selected projection parameters. failed.
- try again, this time using Command Line Window to see what’s going on. (2+ minutes for verification)
- figure out problem, fix parameters (2+ min for verification, which repeats every time you attempt to Insert Variable or other command change)
- execute the command. (Finished in 21 minutes, 43 seconds.)
total arc time: >45 minutes.
- research command syntax, examine old notes and scripts (5 minutes)
- build and test command line (2 minutes)
- execute (7 minutes, 12 seconds)
total ogr time: \~15 minutes.So even if we subtract the debugging time for the problematic projection parameters (which have never caused me trouble before, but that is another story), ogr is still twice as fast. Which could be 3 times as fast if the command syntax were internalized and I didn’t have to look it up. Guess where I’m going to be spending some optimisation energy!
And now for the initial impetus for posting this comment, saving my command line!
for /r %a in (nhn*.shp) do @ogr2ogr ogr %a -t_srs z:\Arcutils\prj\YkAlbers.proj4