matt wilkie a hewer of maps,2011-06-11:/atom/ 2015-05-11T07:00:00Z acrylamid Extract Invisible Attachment,2015-05-11:/Fixes/extract-invisible-attachment 2015-05-11T07:00:00Z matt wilkie <p>I periodically get email from ios users with invisible attachments. </p> <blockquote> <p><strong>TLDR:</strong> view raw, save base-64 to text, uudecode using <strong><code>certutil --decode</code></strong> (a native tool in Win7+)</p> </blockquote> <p>Last night I finally figured out what is going on: the attachment is marked as <em>"inline"</em> in the header, but the client (gmail in this case, but can happen with others too) doesn't render the attachment correctly. However since the media is inline it's not visible/extractable like regular attachments are.</p> <h5>Symptoms</h5> <ul> <li>Email says there is in attachment (e.g. has a paperclip), but there is no place to click and save or download the attachment</li> <li>broken image icon in message body</li> </ul> <p><img alt="" src=""/> </p> <h5>The fix</h5> <ul> <li>view original/raw message</li> <li>Find <code>Content-Disposition: attachment; filename="some_filename.pdf"</code> and save everything between <code>Content-Id: &lt;AAA888...&gt;</code> and <code>--Apple-Mail-AAA888...--</code> lines (don't include those lines) to a text file, e.g. <code>some_filename.uue</code></li> <li>open command shell and run <code>certutil -decode some_filename.uue some_filename.pdf</code></li> </ul> <h6>Sources</h6> <ul> <li>Fowarded emails with attachments not showing up with paperclip icon,!topic/gmail/Mdujqj1zUic</li> <li>Message with pdf attachment doesn't show as having an attachment,!topic/gmail/LbUIrEqzRXE</li> <li>Super User: <a href="">How can I extract an attachment from the email body (encoded as base64)</a></li> <li>Dick on for the <em>certutil</em> tip</li> </ul> The 1st modern Cahill-Keyes Map, maybe,2013-12-18:/GIS/the-1st-modern-cahill-keyes-map-maybe 2013-12-19T00:26:29Z matt wilkie <p>All maps distort things -- one or more of area, shape and distance. They [splatten] a roundish sphere into a flat piece of paper. Some distortions are much worse than others. The <a href="\GIS\the-1st-modern-cahill-keyes-map-maybe\Projects\cahill-keyes-butterfly-map">Cahill-Keyes</a> projection is the best invention to date we humans have for minimizing the effects. Unfortunately very few know about it or have seen it. This map by Duncan Webb of Australia, possibly the first modern full scale version of the CK map projection could go a long way to rectify our common (mis)perception of the relative shapes and sizes of where we live. <em>If</em> it sees production -- you can help make that happen by pledging to buy one before January 3rd:</p> <p><a href=""><img alt="Duncan Webb's hardcopy Cahill-Keyes Map project" src=""/></a></p> <p>What's all the fuss about? There are many resources out there explaining it more completely and eloquently than I can, but here's an info-graphic to get you started. Each of the pinkish blobs on the maps below <strong>occupy the same size and area of ground</strong>, but they sure don't look like it do they?</p> <p><a href=""><img alt="Various map projections with tissot error ellipses" src=""/></a></p> <p><a href=""><img alt="Various map projections" src=""/></a></p> <p>[0]: <em>well, 'splatten' should be word darn it!</em></p> Oh Assign, Assign, wherefore art thou Assign?,2013-05-17:/Itches/oh-assign-assign-wherefore-art-thou-assign 2013-05-17T07:54:00Z matt wilkie <p>Most of my computer life of the last few years has lived within Dropbox. It's proven to be a very hands off largely no-thought means of ensuring the 3 Windows computers I use regularly at work and home all have almost identical tool chains and environments. It's always bugged me though, that my "home" is buried under <code>C:\Users\matt\Dropbox</code> (though at least with Win7 it's not under that horrible "Documents and Settings" folder!). Way too freakin much typing just to get to the point of starting work, or fun.</p> <p>I've tried several methods for making this more livable, and each succeeds to a certain extent -- though <em>none</em> come close to matching what we had 20 years ago with the <a href="" title="scan for assign">Amiga</a>.</p> <h4>Move Dropbox to root of C:</h4> <p>yech. user data on same partition as operating system. never been comfortable with this. I want OS and Programs over <em>[there]</em>, my docs and stuff <em>[here]</em>, and easily wipe or restore one without touching the other.</p> <h4>Mapped "Network" Drive</h4> <p>Open a command shell, fire off <code>net use B: \\localhost\c$\users\%username%\Dropbox /persistent:yes</code>.</p> <p>(So nice that A: and B: almost never exist anymore, and are thus available, even when joined to a domain.)</p> <p>Worked swimmingly well for several months, until learning that the mysterious Mecurial and Bazaar lock errors I would encounter at seemingly random occasions and took hours to get out of were due to timing issues in the underlying network transport protocols, even though everything was happening on the local machine and darn fast, to human perception.</p> <p>Also can't use any commands requiring elevated privileges because "B:" isn't available to Administrator. (why o why doesn't Windows have <code>sudo</code>??)</p> <h4>Subst-tituted Drive</h4> <p><code>subst b: c:\users\%username%\dropbox</code></p> <p>An effective cure for the timing issues noted previously. Suffers from lack of sudo, which has been real problem while trying to work on improving Leo and Osgeo4w install packages.</p> <h4>Separate Partition</h4> <p>The Dropbox app steadfastly refuses to use a bare drive as it's home, and insists on creating a folder called <code>Dropbox</code> where you point it. If you run a client older than 1.6 or so, before the database was encrypted, it is possible to open the <code>config.db</code> file in an sqlite3 aware db program and change the folder name manually, including to a bare drive letter. </p> <p>There are some features lost with this, plus you'll also be syncing the <em>Recycle Bin</em> and <em>System Volume Information</em>, which could cause other issues.</p> <h4>Mounted NTFS Folder</h4> <p><strong>Finally</strong>, yesterday, I hit open the best possible solution short of the Amiga's Second Coming: create a separate partition and volume and instead of assigning it a drive letter mount it to an empty NTFS folder, and then <code>subst</code> that. </p> <div class="highlight"><pre>C:\Users\<span class="o">%</span>username<span class="o">%</span>\Dropbox <span class="o">&lt;--</span> mount volume here subst b: C:\Users\<span class="o">%</span>username<span class="o">%</span>\Dropbox </pre></div> <p>For extra flourish, size the volume to your Dropbox limit. Doing so you always know at a glance how close you are to your quota. (Good idea to make it 15% than actual quota for breathing room. It's enough space to defrag, and Windows will get markedly slower on volumes with less free board.)</p> <p><img alt="dropbox volume in My Computer" src=""/></p> Python fixit snippets,2013-05-16:/Python/python-fixit-snippets 2013-05-16T17:54:00Z matt wilkie <p>Very short problems about python, found and fixed.</p> <div class="highlight"><pre><span class="o">&gt;</span><span class="n">pip</span> <span class="n">failed</span> <span class="n">to</span> <span class="n">create</span> <span class="n">process</span><span class="p">.</span> </pre></div> <p>whups, this worked yesterday. Whatup? --&gt; drive letters/paths have been changed. The path to python.exe specified in pip.exe has moved. Solution: <a href="">reinstall pip</a> (and any other py programs similarly impacted).</p> Portable Leo from scratch on Windows,2013-04-28:/Projects/portable-leo-from-scratch-on-windows 2013-04-28T07:00:00Z matt wilkie <p>Here is a recipe for installing <a href="">Leo</a> from scratch on Windows (Win7 tested). The only prerequisites are command line <code>wget</code> in PATH and an internet connection.</p> <p>About 50mb of files are downloaded and 200mb consumed when done.</p> <p>Open a command prompt and run:</p> <div class="highlight"><pre><span class="nb">mkdir</span> <span class="n">X:</span><span class="o">\</span><span class="n">testing</span> <span class="n">pushd</span> <span class="n">X:</span><span class="o">\</span><span class="n">testing</span> <span class="n">wget</span> <span class="o">-</span><span class="n">O</span> <span class="n">apt</span><span class="o">.</span><span class="n">exe</span> <span class="o">--</span><span class="nb">no</span><span class="o">-</span><span class="n">clobber</span> <span class="n">http:</span><span class="sr">//</span><span class="n">download</span><span class="o">.</span><span class="n">osgeo</span><span class="o">.</span><span class="n">org</span><span class="sr">/osgeo4w/x</span><span class="mi">86</span><span class="sr">/release/</span><span class="n">apt</span><span class="o">/</span><span class="n">apt</span><span class="o">-</span><span class="n">r1193M</span><span class="o">.</span><span class="n">exe</span> <span class="n">SET</span> <span class="n">OSGEO4W_ROOT</span><span class="o">=</span><span class="nv">%</span><span class="err">~</span><span class="nv">dp0</span><span class="o">\</span><span class="n">root</span> <span class="n">apt</span> <span class="n">setup</span> <span class="n">apt</span> <span class="n">update</span> <span class="n">apt</span> <span class="n">install</span> <span class="n">pyqt4</span> <span class="n">sip</span> <span class="n">call</span> <span class="n">root</span><span class="o">\</span><span class="n">osgeo4w</span><span class="o">.</span><span class="n">bat</span> <span class="n">wget</span> <span class="o">--</span><span class="nb">no</span><span class="o">-</span><span class="n">check</span><span class="o">-</span><span class="n">certificate</span> <span class="n">http:</span><span class="sr">//gis</span><span class="n">t</span><span class="o">.</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="sr">/maphew/</span><span class="mi">5393935</span><span class="sr">/raw/i</span><span class="n">nstall</span><span class="o">-</span><span class="n">pip</span><span class="o">.</span><span class="n">py</span> <span class="n">python</span> <span class="n">install</span><span class="o">-</span><span class="n">pip</span><span class="o">.</span><span class="n">py</span> <span class="n">GO</span> <span class="n">python</span> <span class="n">install</span><span class="o">-</span><span class="n">pip</span><span class="o">.</span><span class="n">py</span> <span class="n">GO</span> <span class="n">pip</span> <span class="n">install</span> <span class="o">-</span><span class="n">i</span> <span class="n">https:</span><span class="sr">//</span><span class="n">testpypi</span><span class="o">.</span><span class="n">python</span><span class="o">.</span><span class="n">org</span><span class="o">/</span><span class="n">pypi</span> <span class="n">leo</span><span class="o">-</span><span class="n">editor</span> <span class="n">python</span> <span class="n">apps</span><span class="o">\</span><span class="n">Python27</span><span class="o">\</span><span class="n">Scripts</span><span class="o">\</span><span class="n">leo</span> </pre></div> <p>After this the directory <code>X:\testing\root</code> can be renamed and/or moved anywhere.</p> <p>To run leo in future sessions, simply call <code>x:\path\to\root\osgeo4w.bat python apps\Python27\Scripts\leo</code>, either as a batch file or windows shortcut.</p> <p>IMPORTANT NOTE - this is just a proof of concept. It pulls together a couple of my different projects, and is very satisfying in that regard (yay! they work!), but is not ready for prime time. Feedback welcome.</p> <p>(((this is a post in progress, see <a href=";fromgroups=#!searchin/leo-editor/portable|sort:date/leo-editor/pH4mHaSRbfc/tQcxjn37xKUJ">this thread</a> for more)))</p> Plus 1 for static site generators!,2013-04-20:/web-dev/plus-1-for-static-site-generators 2013-04-20T10:28:00Z matt wilkie <p>There is no time like the present to really delve into and master static web generation:</p> <p><em><a href="">Huge attack on WordPress sites could spawn never-before-seen super botnet. Ongoing attack from &gt;90,000 computers is creating a strain on Web hosts, too</a></em></p> <p><em><a href="">Ongoing malware attack targeting Apache hijacks 20,000 sites. Mysterious "Darkleech" exposes visitors to potent malware exploits</a></em></p> <p>I received the ArsTechnica newsletter with the above stories in my mailbox in the same 5 minute window that Google Webmaster Tools informed me that one of our websites for my work has likely been compromised. </p> <p>No, it's not running Wordpress, but it is running a blog-style php platform that is several versions behind current because I didn't want to break the site while trying to upgrade a system I don't understand. Now this particular bit of procrastination has turned into a bite!</p> <p>Guess I know what I'm doing for the next week or more. :-/</p> Install Pip script,2013-04-16:/Python/install-pip-script 2013-04-16T21:22:00Z matt wilkie <p><em>Introducing a pure python means to install pip and it's requirements <a href="">all in one go</a>.</em></p> <p>Installing <a href="">pip</a> on Windows is relatively straightforward, now, but still a tad bumpy. Basically it consists of </p> <ol> <li>Download distribute or setuptools</li> <li>Install it with <code>python</code></li> <li>Download the get-pip script</li> <li>Install with <code>python</code></li> </ol> <p><em>(Aside: curiously the <a href="">distribute docs</a> say you can use pip to install distribute!)</em></p> <p>On Windows this means installing <code>wget</code> or <code>curl</code> in order to follow the install instructions as given, or pointing and clicking your through <em>click-this-url &gt;&gt; save as &gt;&gt; choose a spot &gt;&gt; open a python shell &gt;&gt; ...etc.</em>. Easy, but boring and easy to make small mistakes which means repeating all or part of the recipe. </p> <p>So as part of my continuing quest to learn enough to be called a true pythonista, I wrote a small script to do it all in one go, <a href=""></a>. It is adapted from <a href="">Fernando Macedo</a>'s answer in <a href="">How to install pip on windows?</a>. The seminal idea is it use <code>urllib</code> to do the download, which is included in Python's standard library -- isn't it just great to have batteries included? (though it would begs the question of why there isn't a python package installer battery included...)</p> <p>The script does a bit more, but here's the essential idea (the blog renderer is messing up the indentation, sorry):</p> <div class="highlight"><pre><span class="n">sources</span> <span class="p">=</span> <span class="p">[</span> <span class="s">&#39;;</span><span class="p">,</span> <span class="s">&#39;;</span><span class="p">,</span> <span class="p">]</span> <span class="k">for</span> <span class="n">src</span> <span class="n">in</span> <span class="n">sources</span><span class="p">:</span> <span class="n">f</span> <span class="p">=</span> <span class="n">urlopen</span><span class="p">(</span><span class="n">src</span><span class="p">).</span><span class="n">read</span><span class="p">()</span> <span class="k">try</span><span class="p">:</span> <span class="n">exec</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="n">except</span><span class="p">:</span> <span class="n">pass</span> </pre></div> <p>Passing on an exception instead of dealing with it isn't a great idea, maybe even a stupid one, but without it only the first file is executed. Comments, fixes, and enhancements are all welcome. :) </p> <p>See <a href=""></a></p> Deploy website with rsync,2013-03-17:/web-dev/deploy-website-with-rsync 2013-03-17T07:00:00Z matt wilkie <p><em>Recipe for copying my Acrylamid-generated static html website to my webhost (<a href=""></a>).</em></p> <p>In a nutshell, I use <a href=";sub=0&amp;lang=en">QtdSync</a> to generate the basic rsync update script, and then add rsync advanced options as make sense to me. From QtdSync's progress screen the rsync command line can be copied and pasted to create a batch file (strip the <code>-e ... sshpass...</code> line for interactive password prompting).</p> <p>QtdSync is a bit twitchy around the <em>"backup/don't backup into subfolder"</em> options. Sometimes they appear to have the opposite effect of what they say. I still found it much easier than trying to construct the appropriate rsync script from scratch myself, just test on a small data set until the settings are ironed out. The backup set <code>.qtd</code> files are straightforward XML so it's pretty easy to check the results of your settings.</p> <h5>QtdSync hard coded rsync options</h5> <p>These are always used.</p> <p><code>-rltv</code> which means:<br /> <code>--recursive</code><br /> <code>--links</code> - <em>copy symlinks as symlinks</em><br /> <code>--times</code> - <em>preserve modification times</em><br /> <code>--verbose</code></p> <p><code>--progress</code> - <em>show progress</em></p> <h5>QtdSync editable advanced rsync options</h5> <p><code>--delete</code> - <em>delete extraneous files from destination dirs</em><br /> <code>--force</code> - <em>force deletion of directories even if not empty</em> <br /> <code>--ignore-errors</code> - <em>delete even if there are I/O errors</em><br /> <code>--hard-links</code> - <em>preserve hard links</em></p> <h5>Extended rsync options</h5> <p>Add these in the <em>Rsync Expert Settings</em> pane. When added as Task they override the global settings.</p> <p><code>--compress</code> - <em>compress file data during the transfer</em>, saves bandwidth, speeds up transfers, 'nuff said</p> <p><code>--skip-compress=%_ignore_ext%</code> - <em>skip compressing files with a suffix in LIST</em>, not yet using, but seems like a good idea</p> <p><code>--checksum</code> - <em>skip based on checksum, not mod-time &amp; size</em>, because preserving timestamps between Windows and 'nix doesn't always work. I suspect it's faster to use <code>--modify-window=NUM</code> with a window like 2secs (that's what <a href=""><code>xxcopy /ff</code></a> uses) but I'm not that concerned with performance and checksum should be more reliable.</p> <p>For these next two, if something goes wrong the website will stay in previous state. Not quite the same as atomic updates in a db but close enough for me.<br /> <code>--delay-updates</code> - <em>put all updated files into place at transfer's end</em><br /> <code>--delete-delay</code> - <em>find deletions during, delete after</em></p> <p><code>--human-readable</code> - <em>output numbers in a human-readable format</em></p> Going Acrylic,2013-03-16:/web-dev/going-acrylic 2013-03-16T07:00:00Z matt wilkie <p><em>Recipe used for converting httrack snapshot of Mindtouch wiki to markdown for acrylamid</em></p> <h4>Generate the processing list</h4> <div class="highlight"><pre><span class="n">dir</span> <span class="o">/</span><span class="n">s</span><span class="o">/</span><span class="n">b</span> <span class="o">\</span><span class="n">www</span><span class="o">\</span><span class="n">maphew</span><span class="p">.</span><span class="n">com</span><span class="o">\*</span><span class="p">.</span><span class="n">html</span> <span class="o">&gt;</span> <span class="n">process</span><span class="o">-</span><span class="n">list</span><span class="p">.</span><span class="n">xt</span> </pre></div> <p>Edit process-list and remove junk, fix bad filenames (resultant from double quotes in name). </p> <p>Copy to excel and:</p> <ul> <li>convert text to table using <code>\</code> for delim, </li> <li>apply conditional formatting highlighting <code>.html</code></li> <li>sort by html column</li> <li>remove duplicates</li> <li>save tab delimited</li> <li>search and replace tabs with <code>\</code>, removing dupes</li> </ul> <h4>Scripted Html to Markdown to Acrylamid</h4> <div class="highlight"><pre><span class="k">for</span> <span class="o">/</span><span class="n">f</span> <span class="c">%a in (process-list-cleaned.txt) do @mkdir .%~pa</span> <span class="k">for</span> <span class="o">/</span><span class="n">f</span> <span class="c">%a in (process-list-cleaned.txt) do ^</span> <span class="p">@</span><span class="n">pandoc</span> <span class="o">--</span><span class="n">to</span> <span class="n">markdown</span> <span class="o">--</span><span class="n">standalone</span> <span class="o">--</span><span class="n">template</span> <span class="n">acrylamid</span><span class="o">-</span><span class="n">pandoc</span><span class="o">-</span><span class="n">template</span><span class="p">.</span><span class="n">txt</span> &quot;<span class="c">%a&quot; -o &quot;.\%~pa\;</span> <span class="n">acrylamid</span> <span class="n">init</span> <span class="n">converted</span> <span class="n">rd</span> <span class="o">/</span><span class="n">s</span><span class="o">/</span><span class="n">q</span> <span class="n">converted</span><span class="o">\</span><span class="n">content</span> <span class="n">move</span> <span class="n">www</span><span class="o">\</span><span class="n">maphew</span><span class="p">.</span><span class="n">com</span> <span class="n">converted</span> <span class="n">pushd</span> <span class="n">converted</span> <span class="n">rename</span> <span class="n">maphew</span><span class="p">.</span><span class="n">com</span> <span class="n">content</span> <span class="n">rd</span> <span class="o">/</span><span class="n">s</span><span class="o">/</span><span class="n">q</span> <span class="p">.</span><span class="o">.\</span><span class="n">www</span> <span class="n">copy</span> <span class="o">\</span><span class="n">www</span><span class="o">\</span><span class="n">acr</span><span class="o">\</span><span class="n">confy</span><span class="p">.</span><span class="n">py</span> <span class="o">.\</span><span class="n">conf</span><span class="p">.</span><span class="n">py</span> <span class="n">xcopy</span> <span class="o">\</span><span class="n">www</span><span class="o">\</span><span class="n">acr</span><span class="o">\</span><span class="n">theme</span><span class="o">\*</span> <span class="n">theme</span><span class="o">\*</span> <span class="n">acrylamid</span> <span class="n">compile</span> <span class="o">--</span><span class="n">search</span> <span class="p">::</span> <span class="nb">fix</span> <span class="n">title</span> <span class="n">collisions</span> <span class="n">as</span> <span class="n">needed</span> </pre></div> <h4>Clean up header &amp; footer crud</h4> <p>Remove Mindtouch leftovers such as <em>"javascript must be enabled"</em> and <em>"Powered by..."</em>, etc. etc. by using search and replace across all open files.</p> <p>Vim regexes:</p> <div class="highlight"><pre><span class="p">:</span><span class="n">bufdo</span><span class="p">:</span><span class="c">%s/^This application requires Javascript to be enabled\.\_.* Table of contents$//</span> <span class="p">:</span><span class="n">bufdo</span><span class="p">:</span><span class="c">%s/\*No headers\*//</span> <span class="p">:</span><span class="n">bufdo</span> <span class="c">%s/Powered by \[MindTouch Core\_.*maphew)//e</span> <span class="p">:</span><span class="n">bufdo</span> <span class="c">%s/---\n\n\n*/---\r\r/e</span> <span class="p">:</span><span class="n">bufdo</span> <span class="c">%s/$title:\(.*\) - maphew$/title:\1/e</span> </pre></div> <p>Sources:</p> <ul> <li></li> <li></li> </ul> <h5>Fix dates</h5> <p><strong>This marks the end of the automated repeatable process.</strong></p> <p>Read through <em>Special_RecentChanges</em> and manually edit each .md to reflect last time touched. We don't have dates for content which was migrated to Mindtouch (prior to 17.10.2009).</p> <h4>Add tags</h4> <p>The folders are the primary tag, so search *.md and sort by location, then drag'n'drop each group into a handy text editor and paste the appropriate tag(s) in each. </p> <p>I'm sure there's a way to automate this, but I decided it would be faster to brute force my way through (esp. since we've already broken with repeatable process anyway).</p> Spring, Cycles,2013-03-01:/category/spring-cycles 2013-03-01T07:00:00Z matt wilkie <p>Well, here I am once again looking at the detritus of my website and wondering what to do with it. Each page represents, at least, an hour or three of endeavour. Many have no relevance any more and or of little if any interest to anyone, including me. Others though... well lets just say I'm not quite ready to pull the plug and let them drain away.</p> <p>While not a regular experience, this is a recurring one. We've been here before. Each time I've devoted a few days or months to rewriting or transforming old content into a new container, add some new content along the way. From static html to wiki, to a cms, to another wiki, back to static html, whatever this current cycle results in.</p> <p>One of the historical features of this cycle I'm noting today is: after the phoenix dies and is resurrected, the life soon fades. The form remains, because in the digital world there is no decomposition, but the energy is gone. The words remain, unchanging. No or very few pages are re-introduced for a long time, months to years.</p> <p>I find this very curious. Most of my waking life is spent interacting with computers. And most of that interacting is on the internet, consuming and commenting and feeding email, mailing lists, blogs, and myriad fora. How strange then, that my own piece of web real estate is so poorly maintained.</p> <p>I don't know that this current renewal will, in fact, be a renewal, resulting in growth. I don't know that it won't either. I just know that this time around I'm somewhat more interested in watching the play of the impulses and energy behind the effort than the particulars of what system will be used this time around.</p> <p><em>((which, if you're curious, will be <a href="">acrylamid</a>, though after reading about <a href="">acrylamide</a> I'm wondering about the choice in project name... ;-))</em></p> <p>Okay, enough of the intro, on with the game!</p> Record internet radio with VLC, updated,2013-01-16:/HowTo/record-internet-radio-with-vlc-updated 2013-01-16T07:00:00Z matt wilkie <p>I needed to record a CBC radio piece today. For most cases, it appears the easiest route is to use <em>streamtuner</em> and <em>streamripper</em>, however there is something about my particular computer or CBC’s streaming style that made it difficult for me. Since it took a couple of hours to figure it all out, I figured I’d better save it for future reference. So here is a vlc only method which works, but with more clicks and it’s is easier to select a combination of options that don’t work than do. This is the combination which finally worked for me:</p> <ul> <li>start VLC</li> <li>File > Open Network Stream</li> <li>for HTTP/HTTPS/FTP/MMS: <a href="" title=""></a></li> <li>check ’steam/save’, select ‘Settings’, then ‘Outputs’:</li> <li>check ‘Play Locally’</li> <li>check ‘File’, set filename as you wish</li> <li>Encapsulation method: ‘Raw’</li> <li>check ‘Audio codec’, and select ‘mpga’ from the list, bitrate as you like (96kb is okay for this particular stream)</li> </ul> <p>It’s the audio codec part which tripped me up the most, even though I was saving mp3 selecting mp3 as a code yields a 0 byte file.</p> <p>For command line use, copy the source and destination urls from the dialogs created above, however it is necessary to change the syntax somewhat. Namely the <code>:sout</code> option needs to be single quoted.</p> <blockquote> <p><code>vlc ‘:sout=#transcode{acodec=mpga,ab=96,channels=2}:duplicate{dst=display,dst=std{access=file,mux=raw,dst="/home/matt/Radio/cbc-north.mp3"}’</code></p> </blockquote> <p>Here is the same in a bash script, you should only need to edit the INSTREAM and OUTFILE lines to use elsewhere</p> <blockquote> <p><code>#!/bin/sh NOW=$(date +%F_%k-%M) INSTREAM= OUTFILE=/home/shared/Audio/Radio/cbc-north_$NOW.mp3 echo … echo …Recording CBC North internet radio to echo    $OUTFILE echo …expect a 5 second delay before hearing anything echo …and a few “access_mms” errors prior to that. echo … /usr/bin/vlc $INSTREAM ‘:sout=#transcode{acodec=mpga,ab=96,channels=2}:duplicate{dst=display,dst=std{access=file,mux=raw,dst="'$OUTFILE'"}}’</code></p> </blockquote> <p><strong>2013-Jan-16</strong>\</p> <p>Thanks for writing up the howto on the use of headless vlc to capture streaming audio.\ \ It took a little experimentation to make it work on my hardware (an arm-based pogoplug v4).  I don't know if the changes are due to newer versions of vlc or pertain more to how it was built for the pacman database that arch linux uses (the os that's running on my pp4).  Either way, I thought I'd send what I did your way in case you were interested in updating/adding to the howto.\</p> <blockquote> <p><code>#!/bin/sh NOW=$(date +%F_%k-%M) INSTREAM= OUTFILE=/home/user1/wwoz_$NOW. mp3 echo echo recording WWOZ echo cvlc $INSTREAM --sout "#transcode{acodec=mpga,ab=96,channels=2}:std{access=file,mux=raw,dst="$OUTFILE"}"</code>\</p> </blockquote> <p>I wound up using cvlc as opposed to vlc so that it wouldn't complain about dbus access (lack of X11 really).  There's something I have yet to sort on the output file side as libmp3lame complains "[libmp3lame @ 0xb3e80] channel_layout not specified".  It does record, but what the exact parameters it writes the file with I haven't yet looked into.\ \ Regards,\ John\ \ <em>\&lt;John D. Doyle III></em></p> <hr /> <p>Sources:</p> <ul> <li><a href="" title="">HOWTO: StreamTuner + StreamRipper, another way to get music</a></li> <li><a href="" title="">Building an Internet radio recorder with VLC Player</a><a href="" title="">\ </a></li> <li><a href=";t=40822" title=";t=40822">Command line HTTP/File stream: nothing to play</a></li> </ul> What does your Map projection say about you?,2011-11-14:/Other/what-does-your-map-projection-say-about-you 2011-11-14T07:00:00Z matt wilkie <p><a href="">Xkcd</a> has a wonderful <a href="">comic on map projections</a> in common and not so common use, and what each of them have to say about the personalities of those who like them. A blog post and accompanying conversation at <a href="">Explain XKCD</a> fills out some of the jokes.</p> <p>I liked so much I've reworked it into a 10"x12" mini-poster for printing:</p> <p><a href=""><img alt="xkcd-map-projections_10x12in.png" src=""/></a></p> Wanted: dead simple argparse examples,2011-09-15:/Itches/wanted-dead-simple-argparse-examples 2011-09-15T07:00:00Z matt wilkie <p>The standard <a href="">documentation</a> for the <a href="">argparse</a> python module, while excellent I'm sure, is too much for my tiny brain to grasp. I don't need to do math on the command line or meddle with formatting lines on the screen or change option characters. All I want to do is "<em>if arg is A, do this, if B do that, if none of the above show help</em> <em>and quit</em>"</p> appy,2011-09-01:/Projects/appy 2011-09-01T07:00:00Z matt wilkie <p>I’ve been sporadically working on a little command line installer for OSGeo4W, called <a href="" title="">apt</a>. It is based on <a href="" title="">cyg-apt</a>, a command line apt-get-like installer for cygwin byJan Nieuwenhuizen. I currently have two forks. One for o4w alone, which I’ve renamed apt, and the second for cygwin alone, which I’ve left as cyg-apt. Forking is not a good idea when there is so much in common, so I’m going to try and bring them together. One app, two installers. Neither name fits this new job so I’m going to use appy for the time being. It will have to change eventually as there’s another, much bigger, python project with the <a href="" title="">appy name</a>, but I don’t want to spend hours agonizing over a name. Again. <img alt=":)" src="\\blog\wp-includes\images\smilies\icon_smile.gif"/></p> <p>The appy series of posts will chronicle my misadventures as I try and teach myself programming by making this small tool a little bit more useful. To me anyway!</p> <p><a href="" title="">Matt’s cyg-apt</a>: my only claim to (in)famy here is converting system calls to md5sum, tar, etc. to pure python. So now the only pre-requisite is python. All the real work and intelligence in this program is a result of Jan’s work.</p> <p>I created a <a href="" title="">standalone exe</a> of the o4w version which can be used to <a href="" title="">install o4w from scratch</a>, but at 2.5mb it’s a little big for what it actually does.</p> <hr /> <p>Things are going really well. I'm nothing short of astounded with what me, a nonprogrammer, has been able to accomplish in a few dozen hours (albeit spread over many evenings over a year). Why just last night in cyg-apt I added a function to <a href=";r=94" title=";r=94">use the last used mirror</a> instead of hardcoding it. Concieved, thought about, and implemented all in one session. Now <strong>that</strong> is cool!</p> <p><em>(many moons later)</em>     I love python, it makes me feel like I know what I'm doing. ;-)   Around two hours ago I discovered apt was broken because the server <a href="" title="">changed it's ini format</a>. One hour later I had it fixed, tested and committed with a routine I've never used before.</p> <h5>2011-09-01 a major milestone reached, <a href="" title="">Apt</a> eats Apt!</h5> <p>Last night I completed building <strong>a package of apt.exe can upgrade itself</strong>. This means, I think, that now apt should be able to replace any and all packages in Osgeo4W in one go including python. There is much left to do, it's only been tested on two computers so far, and it consumes way more diskspace and bandwidth than I think it should, but it works!</p> <p>To try it out, point your existing apt or o4w setup program (see <em>Testing Repository</em>at end of <a href="" title="">this page</a>) at <a href="" title=""></a> and upgrade away.</p> <div class="highlight"><pre><span class="n">apt</span> <span class="o">--</span><span class="n">mirror</span> <span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">download</span><span class="p">.</span><span class="n">osgeo</span><span class="p">.</span><span class="n">org</span><span class="o">/</span><span class="n">osgeo4w</span><span class="o">/</span><span class="n">testing</span><span class="o">/</span> <span class="n">update</span> <span class="n">apt</span> <span class="n">new</span> <span class="n">apt</span> <span class="n">upgrade</span> </pre></div> <p>After this running apt with no parameters should yield the usual help page with a version number at the top that resembles:</p> <div class="highlight"><pre><span class="o">-</span><span class="p">={</span> <span class="n">Id</span><span class="p">:</span> <span class="n">apt</span><span class="p">.</span><span class="n">py</span> 1179 2011<span class="o">-</span>09<span class="o">-</span>01 12<span class="p">:</span>42<span class="p">:</span>24<span class="n">Z</span> <span class="n">maphew</span> <span class="p">}=</span><span class="o">-</span> <span class="n">apt</span> <span class="p">[</span><span class="n">OPTION</span><span class="p">]...</span> <span class="n">COMMAND</span> <span class="p">[</span><span class="n">PACKAGE</span><span class="p">]...</span> </pre></div> <p>If the first line is just [option] etc. the new apt.exe hasn't been activated. Running %osgeo4w_root%\bin\apt.bat should fix that. If not please let me know.</p> Mindtouch,2011-08-03:/web-dev/mindtouch 2011-08-03T07:00:00Z matt wilkie <h5>v10 license file for Core</h5> <p><a href="" title="">Mindtouch</a> now consolidates all of it's products into a single set of packages, which is good, however this necessitates contacting HQ for a license file even for the open source Core version. Some are bothered by this and don't want to give Mindtouch Corp their email address. If you are in this group feel free to use the <a href="" title="/@api/deki/files/64/=mindtouch-v10-core-license.xml">attached license file</a> (rename to 'license.xml'). It is "licensed" to <a href=";x=0&amp;y=0" title=";x=0&amp;y=0"></a>.</p> <h5>Downgrade from Enterprise to Core</h5> <p>As a result of <a href="" title="">this thread</a> I periodically get private messages asking how to downgrade to the open source. It's really very simple (though I don't know if this is still true with v10):</p> <blockquote> <p>hi maphew, we are not allowed to talk about that in the forum... don't know why! so, just delete the license.xml, restart dekihost and you're back on core =)</p> <p>best regards,<br /> ---snip---</p> </blockquote> Police make it hard to report,2011-07-06:/Musings/police-make-it-hard-to-report 2011-07-06T07:00:00Z matt wilkie <p>Police make it hard to report things to them.</p> <p>Last week while on a canoe trip we spotted a minivan crashed in the river, at the bottom of a 30m+ cliff. From the open driver's side door It looked like it had been pushed over the edge, and, judging from the multi-level water stains on the side, had been there for a week or two at least. Still, being good citizens when we got to somewhere we could make a call we phoned the police just in case nobody had reported it yet. They received the report easily enough but then demanded name, phone number, street address, a few other things I don't recall, and birthdate. Needing to know name and number makes sense, address is a bit more intrusive, but okay, who knows, maybe I'm a disgruntled lover trying to get someone in trouble. But birthdate? Take a hike! That's irrelevant to the issue at hand and none of their business.</p> <p>This morning while walking the dog along a bush trail I found a bike that looked stolen or abandoned. Late this evening it was still there. Now I know from past experience that a bike left on it's own in the trees will very likely soon end up a kilometer or two down the trail smashed and bent, and with some of the better bits missing altogether. An unknown faceless someone would likely be very grateful to have it moved somewhere for safekeeping and reported to the authorities, meaning the police, and eventually returned safely home to them. And I'd feel pretty good for having played the good samaritan. Trouble is I don't want the police treating me like a suspect when all I'm trying to do is hand someone else a good turn.</p> <p>I'll call them, eventually, an unknown faceless someone deserves it, but I have to overcome resistance to do so. Maybe next time I see something by the trail or in the water I'll just keep going. Come to think of it, maybe that's why I have the past experience in the first place of seeing unattended things transformed into junk and garbage. Others before me had decided it's just not worth the hassle; "nothing to see here, move along."    That's sad.</p> Pale Blue Dot,2011-05-17:/Musings/pale-blue-dot 2011-05-17T07:00:00Z matt wilkie <div style="float:right;"> <img src="" alt="Pale Blue Dot - small.jpg"/> </div> <p><a href="" title="">This article</a> came across my desk and inspired me to make a poster. Well, it inspired me to find a poster, but I didn't locate one that resonated just the right way with me, so I made my own.</p> <p><em>A portion of the first ever 'portrait' of the solar system taken by Voyager 1 in 1990 from a distance of more than 4 billion miles from Earth. Our world is a mere point of light, a crescent only 0.12 pixels in size, coincidentally right in the center of a scattered light ray, resulting from taking the image so close to the sun.</p> <p>"We succeeded in taking that picture , and, if you look at it, you see a dot. That's here. That's home. That's us. On it, everyone you ever heard of, every human being who ever lived, lived out their lives. The aggregate of all our joys and sufferings, thousands of confident religions, ideologies and economic doctrines, every hunter and forager, every hero and coward, every creator and destroyer of civilizations, every king and peasant, every young couple in love, every hopeful child, every mother and father, every inventor and explorer, every teacher of morals, every corrupt politician, every superstar, every supreme leader, every saint and sinner in the history of our species, lived there on a mote of dust, suspended in a sunbeam.</p> <p>The earth is a very small stage in a vast cosmic arena. Think of the rivers of blood spilled by all those generals and emperors so that in glory and in triumph they could become the momentary masters of a fraction of a dot. Think of the endless cruelties visited by the inhabitants of one corner of the dot on scarcely distinguishable inhabitants of some other corner of the dot. How frequent their misunderstandings, how eager they are to kill one another, how fervent their hatreds. Our posturings, our imagined self-importance, the delusion that we have some privileged position in the universe, are challenged by this point of pale light.</p> <p>Our planet is a lonely speck in the great enveloping cosmic dark. In our obscurity -- in all this vastness -- there is no hint that help will come from elsewhere to save us from ourselves. It is up to us. It's been said that astronomy is a humbling, and I might add, a character-building experience. To my mind, there is perhaps no better demonstration of the folly of human conceits than this distant image of our tiny world. To me, it underscores our responsibility to deal more kindly and compassionately with one another and to preserve and cherish that pale blue dot, the only home we've ever known.</p> <p>-- Carl Sagan, "Reflections on a Mote of Dust" </em></p> <p><a href=""><strong>Listen</strong> to Carl read</a> this passage<br /> <a href=""><strong>Watch</strong> the speech</a> set to music and imagery*<br /> <a href=""><strong>Purchase</strong> a hardcopy</a> in a varity of sizes, paper &amp; framing options<br /> <a href=""><strong>Download</strong></a> in editable svg format (350k) </p> <p><a href=""><img alt="Flattr this" src="" title="Flattr this"/></a></p> <h5>Sources &amp; Inspiration:</h5> <ul> <li>iJourney - <a href=""></a></li> <li>*Carl Sagan "Pale Blue Dot" (New Version) Spread This! - <a href=""></a></li> <li>Sky Image Lab - <a href=""></a></li> <li>Matthew Beckler - <a href=""></a></li> <li>Howard Abrams - <a href=""></a> (Howard has an audio only version; took quite awhile to find.)</li> </ul> <p><a href="" title="Creative Commons License"><img alt="Creative Commons License" src=""/></a></p> <p>Pale Blue Dot by <a href="\Musings\Pale_Blue_Dot.html">Matt Wilkie</a> is licensed under a <a href="">Creative Commons Attribution-Share Alike 2.5 Canada License</a> Based on a work at <a href=""></a>. Permissions beyond the scope of this license may be available at <a href="\Musings\pale-blue-dot\"></a>.</p> Time Traveller Drop Points,2011-05-17:/Other/time-traveller-drop-points 2011-05-17T07:00:00Z matt wilkie <p>I was cleaning out old email and ran across this important message which fell through the cracks. I don't have the time to fulfill this urgent request, but thought perhaps I could help by getting the word out to someone who can. Thus was born the <a href=";ie=UTF8&amp;t=p&amp;msa=0&amp;msid=110062113008218362869.00047dc44105664577020&amp;ll=42.494251,-71.142998&amp;spn=0.011075,0.017252&amp;z=14&amp;source=embed" title=";ie=UTF8&amp;t=p&amp;msa=0&amp;msid=110062113008218362869.00047dc44105664577020&amp;ll=42.494251,-71.142998&amp;spn=0.011075,0.017252&amp;z=14&amp;source=embed">Time Traveller drop points</a>:</p> <blockquote> <p>A web map service for Time Travellers who need to arrange for pick up and or delivery of specialty goods. On the go and need a supply dump provisioned? Put your drop point and request here.\ \ <strong>NOTE</strong>: payment and delivery term arrangements are <strong>*strictly*</strong> between traveller and deliverer. We don't want to know about it and will not refund a deal gone bad.</p> </blockquote> <p><a href="" title=""><img alt="Flattr this" src="\\button\flattr-badge-large.png" title="Flattr this"/></a></p> <p>—–Forwarded Message—–\ Subject: Dimensional warp Generator Needed ubamz\ Date: 09 Aug 2003 16:33:10 +0000</p> <p>Hello,</p> <p>I’m a time traveler stuck here in 2003. Upon arriving here my dimensional warp generator stopped working. I trusted a company here by the name of LLC Lasers to repair my Generation 3 52 4350A watch unit, and they fled on me. I am going to need a new DWG unit, prefereably the rechargeable AMD wrist watch model with the GRC79 induction motor, four I80200 warp stabilizers, 512GB of SRAM and the menu driven GUI with front panel XID display.</p> <p>I will take whatever model you have in stock, as long as its received certification for being safe on carbon based life forms.</p> <p>In terms of payment:</p> <p>Payment can be made in Galactic Credits, Platinum gold, or 2003 currency upon safe delivery of unit.</p> <p>INSTRUCTIONS MUST BE FOLLOWED EXACTLY:</p> <p>Please transport unit in either a brown paper bag or box to below coordinates on Saturday August 9th at (exactly 3:00pm) Eastern Standard Time on the dot. A few minutes prior will be ok, but it cannot be after. If you miss this timeframe please email me. I will not be there prior to 2:45pm EST, (so do not transport before then).</p> <p>\ View <a href=";ie=UTF8&amp;t=p&amp;msa=0&amp;msid=110062113008218362869.00047dc44105664577020&amp;ll=42.494251,-71.142998&amp;spn=0.011075,0.017252&amp;z=14&amp;source=embed" title=";ie=UTF8&amp;t=p&amp;msa=0&amp;msid=110062113008218362869.00047dc44105664577020&amp;ll=42.494251,-71.142998&amp;spn=0.011075,0.017252&amp;z=14&amp;source=embed">Time Traveller drop points</a> in a larger map</p> <p>Item is to be delivered at Majority Lane in Woburn, Massachusetts located at: Latitude N 42.49430 &amp; Longitude W 071.14275 and the Elevation is 140 feet.</p> <p>WARNING: DO NOT ATTEMPT TO TRANSPORT ITEM BY REGULAR MEANS OF TELEPORTATION. THEY ARE MONITORING AND WILL REDIRECT THE SIGNAL!!</p> <p>I DO NOT CARE HOW YOU HAVE TO GET IT HERE, JUST DO IT IN A WAY THAT NO SPYING EYES WILL POSSIBLY BE ABLE TO REDIRECT or BLOCK THE TRANSFERENCE. IT IS VERY IMPORTANT THAT YOU BE ABLE TO MONITOR THE TRANSFER.</p> <p>HOW ARE YOU GOING TO SEND IT SO THAT THEY CANNOT REDIRECT OR BLOCK IT??? If in doubt do not transport actual unit until your method of transfer can be confirmed as a success. You just might need to send a intergalactic courier to deliver item safely to me. If so be VERY careful at how they approach me.</p> <p>It is best if you send a intergalactic courier to deliver, this way you can be certain the unit arrives ok, However If you are certain that you have the means to teleport unit in a safe manner please send a (separate) email to me at: <a href="\Other\time-traveller-drop-points\" title=""></a> only after unit has been safely delivered with payment instructions.</p> <p>Do not reply directly back to this email as it will only be bounced back to you.</p> Decollaring with nearblack,2011-04-28:/Itches/decollaring-with-nearblack 2011-04-28T07:00:00Z matt wilkie <p><em><a href="" title="to-be-decollared.jpg"><img alt="to-be-decollared.jpg" src=""/></a>The <a href="" title="/@api/deki/files/63/=to-be-decollared.tif">attached geotiff</a> image has two collars to be removed. The outermost consists of pure black (0,0,0) while the innermost is off-white (ranges from 240 thru 255). I used <a href="" title="">gdalsetnull</a> to change 0,0,0 to nodata, then ran `<a href="" title="">nearblack</a> -white` on the result. I detect no difference in the output image even with a very large fuzz factor (-near 50) .</em></p> <p><em>What am I doing wrong?</em></p> <h5>Answers</h5> <p>From Even Rouault I learned that <a href="" title="">nearblack doesn't grok nodata</a>, it just looks at the pixel values. Consequently it never sees the white collar, except a small portion at the very top of the image.</p> <p>Thankfully Luke Pinner, who's come to my aid <a href="\Itches\Projects\appy.html" title="Projects/appy">before</a>, has an elegant solution using a VRT intermediate file <a href="" title="">(ref)</a>:</p> <div class="highlight"><pre><span class="c">#Change black to white</span> <span class="n">gdalbuildvrt</span> <span class="o">-</span><span class="n">srcnodata</span> 0 <span class="o">-</span><span class="n">vrtnodata</span> 255 <span class="n">to</span><span class="o">-</span><span class="n">be</span><span class="o">-</span><span class="n">decollared</span><span class="p">.</span><span class="n">vrt</span> <span class="n">to</span><span class="o">-</span><span class="n">be</span><span class="o">-</span><span class="n">decollared</span><span class="p">.</span><span class="n">tif</span> <span class="c">#Run nearblack -white</span> <span class="n">nearblack</span> <span class="o">-</span><span class="nb">white</span> <span class="o">-</span><span class="n">o</span> <span class="n">decollared</span><span class="p">.</span><span class="n">tif</span> <span class="n">to</span><span class="o">-</span><span class="n">be</span><span class="o">-</span><span class="n">decollared</span><span class="p">.</span><span class="n">vrt</span> </pre></div> ipy and arcgisscripting,2011-04-21:/Projects/ipy-and-arcgisscripting 2011-04-21T07:00:00Z matt wilkie <p>….some further experiments with <a href="" title="">ipython</a>and <a href=";pid=706&amp;topicname=Writing_Python_scripts" title=";pid=706&amp;topicname=Writing_Python_scripts">ArcGIS python</a> geoprocessing. Here is a simple script to convert a bunch of coverages to shapefile. There are 9 coverages occupying 3mb, it takes about 6 minutes, consumes 2 processors to 80-90% capacity and chews through 300mb of ram:</p> <blockquote> <div class="highlight"><pre><span class="n">cd</span> <span class="n">w</span><span class="p">:</span><span class="o">/</span><span class="n">Env</span><span class="o">-</span><span class="n">dat</span><span class="p">.</span>003<span class="o">/</span>2007<span class="o">-</span><span class="n">March</span><span class="o">/</span><span class="n">workspace</span><span class="o">/</span><span class="n">envy_ed2import</span> <span class="n">arcgisscripting</span> <span class="n">gp</span> <span class="p">=</span> <span class="n">arcgisscripting</span><span class="p">.</span><span class="n">create</span><span class="p">()</span> <span class="n">gp</span><span class="p">.</span><span class="n">Workspace</span> <span class="p">=</span> ‘<span class="o">./</span>’ <span class="n">todo</span> <span class="p">=</span> !<span class="n">dir</span> <span class="o">/</span><span class="n">b</span> <span class="n">fwtc</span><span class="o">*</span> <span class="k">for</span> <span class="n">cov</span> <span class="n">in</span> <span class="n">todo</span><span class="p">:</span> <span class="k">try</span><span class="p">:</span> <span class="n">print</span> <span class="n">cov</span> <span class="n">gp</span><span class="p">.</span><span class="n">FeatureClassToShapefile</span><span class="p">(</span> <span class="n">cov</span> <span class="o">+</span> ‘<span class="o">/</span><span class="n">arc</span>’<span class="p">,</span> ‘<span class="o">./</span><span class="n">shp</span>’<span class="p">)</span> <span class="n">except</span><span class="p">:</span> <span class="n">print</span> <span class="n">gp</span><span class="p">.</span><span class="n">GetMessages</span><span class="p">()</span> </pre></div> </blockquote> <p>Contrast that to using <a href="" title="">fwtools</a> <a href="" title="">ogr2ogr</a>, which takes 10 seconds:</p> <blockquote> <div class="highlight"><pre><span class="k">for</span> <span class="c">%a in (fwtc*) do ogr2ogr -f “esri shapefile” ogr\%a %a</span> </pre></div> </blockquote> <p>The problem is, ogr2ogr doesn’t currently translate everything arcpy does. It leaves behind the metadata and fills voids (though these can be filtered out later as the attributes are empty, but that is an extra step). There’s also no way (that I can see) to select only arcs or only points.</p> <h5>(June 6th) another episode in the ogr vs arcgis battle:</h5> <p>Goal: <strong><em>Project 70 shapefiles in various subdirectories occupying 300mb</em></strong>.</p> <h6><strong>ArcGIS BatchProject interactively:</strong></h6> <ul> <li>search catalog for k:\incoming\NHN*.shp</li> <li>fire up Batch Project toolbox</li> <li>drag’n'drop search results (wait 2+ minutes for verification step to complete)</li> <li>run</li> <li>oops, error in selected projection parameters. failed.</li> <li>try again, this time using Command Line Window to see what’s going on. (2+ minutes for verification)</li> <li>figure out problem, fix parameters (2+ min for verification, which repeats every time you attempt to Insert Variable or other command change)</li> <li>execute the command. (Finished in 21 minutes, 43 seconds.)</li> </ul> <p><strong>total arc time: >45 minutes.</strong></p> <h6><strong>ogr2ogr:</strong></h6> <ul> <li>research command syntax, examine old notes and scripts (5 minutes)</li> <li>build and test command line (2 minutes)</li> <li>execute (7 minutes, 12 seconds)</li> </ul> <p><strong>total ogr time: \~15 minutes.</strong>So even if we subtract the debugging time for the problematic projection parameters (which have never caused me trouble before, but that is another story), ogr is still twice as fast. Which could be 3 times as fast if the command syntax were internalized and I didn’t have to look it up. Guess where I’m going to be spending some optimisation energy!</p> <p>And now for the initial impetus for posting this comment, saving my command line!</p> <blockquote> <div class="highlight"><pre><span class="k">for</span> <span class="o">/</span><span class="n">r</span> <span class="c">%a in (nhn*.shp) do @ogr2ogr ogr %a -t_srs z:\Arcutils\prj\YkAlbers.proj4</span> </pre></div> </blockquote> Don't show internal boundaries, or: Arcinfo Regions, we miss you!,2011-04-18:/Itches/don-t-show-internal-boundaries-or-arcinfo-regions-we-miss-you 2011-04-18T07:00:00Z matt wilkie <p>this page is to support the <a href=""> question</a> of the same name, namely how does one most efficiently use arcgis and feature classes to achieve some the same things which used to be possible with arcinfo workstation and region sub-coverages.</p> <p><img alt="" src="\\8NpAT.gif"/></p> <p>Please see attached archives for a v10 <a href="" title="/@api/deki/files/60/">map package</a> which demonstrates the problem and (simulated) desired results, and the <a href="">source arcinfo coverage</a> from which the simulation is built.</p> COMfounded python,2011-01-25:/Projects/Canvec/comfounded-python 2011-01-25T07:00:00Z matt wilkie <p>I think my python-COM environment is messed up.</p> <p><strong>Got</strong></p> <div class="highlight"><pre><span class="n">D</span><span class="p">:</span><span class="o">\&gt;</span> <span class="n">python</span> <span class="n">list</span><span class="o">-</span><span class="n">fc</span><span class="p">.</span><span class="n">py</span> <span class="p">.</span><span class="o">./</span><span class="n">Scratch</span><span class="o">/</span><span class="n">blank_canvec</span><span class="p">.</span><span class="n">gdb</span> <span class="o">&lt;</span><span class="n">COMObject</span> <span class="o">&lt;</span><span class="n">unknown</span><span class="o">&gt;&gt;</span> </pre></div> <p>and <strong>expected</strong>something <a href="" title="">like this</a>:</p> <p>../Scratch/blank_canvec.gdb\ [u'counties.shp', u'roads.shp']\ counties.shp\ roads.shp</p> <h6>Possible culprits:</h6> <p><img alt="" src=""/></p> <ul> <li>using Mark Cederholm's <a href="" title="">python+arcobjects</a> recipe requires registering com stuff, which conflicts wth PythonWin's com stuff.</li> <li>I'm using <a href="" title="">Arcgis with python 2.6</a> instead of the shipped 2.5. This hasn't proved problematic before, but maybe I just haven't wondered into this section of the forest before (which means I need to <a href="\Projects\Canvec\Search_PATH_for_duplicates.html" title="Projects/Search PATH for duplicates">eat crow for Jason Scheirer</a>).</li> <li>I have concurrent python 2.5 and python 2.6 installs, 32 and 64bit, and they are walking over each other</li> <li>one of the many other programs which distribute static python dll's is in path and colliding</li> <li>I'm just very poor snake charmer. :)</li> </ul> <h5>Troubleshooting</h5> <p>Win32 bug#3076741, <a href=";aid=3076741&amp;group_id=78018&amp;atid=551954" title=";aid=3076741&amp;group_id=78018&amp;atid=551954">"This object does not support enumeration"</a>. Using win32com.client.gencache registers com objects on the fly, fixes one problem and creates others.</p> <h6>update, 2011 Jan 25</h6> <p>I think I've narrowed it to the win32 extension, as presented by the <a href="" title="">arcgisscripting</a>wrapper.</p> <p>I uninstalled all versions of python and cleared all directories from PATH that had any kind of python??.dll, then installed python 2.5.4, and numpy from the arcgis dvd (as per <a href="" title="">instructions here</a>). If <em>PYTHONPATH</em> contains ...\ArcGIS\bin, then we get the expected results, if not, and we add to Lib\site-packages: win32com not found. If we then install the win32com package we get the \&lt;COMObject \&lt;unknown>> error.</p> <p>The readme for PythonWin on the install dvd says <em>"ArcGIS no longer installs PythonWin because the recommended methodology for creating the geoprocessor has changed to using a new Python module that ESRI has developed ...".</em></p> <p>Sooooo, and win32com work for most things, but not for gp.ListFields() or gp.ListFeatureClasses() functions. I bet this also applies to other list functions too, see <a href=";d=34873" title=";d=34873">changes in list methods</a> knowledgebase article. This means that python 2.6 and 2.7 can be used with ArcGIS 9.x, but only if you stay away from the list functions.</p> Retrieve members of AD group,2011-01-18:/HowTo/retrieve-members-of-ad-group 2011-01-18T07:00:00Z matt wilkie <p><em>How to retrieve members and email of an Active Directory group in a re-usable format.</em></p> <p>Locate <a href="" title="">dsget</a> command line utility and put it in path.</p> <p>Open an administrative command shell and:</p> <div class="highlight"><pre><span class="n">dsget</span> <span class="n">group</span> &quot;<span class="n">CN</span><span class="p">=</span><span class="n">mygroup</span><span class="p">,</span><span class="n">OU</span><span class="p">=</span><span class="n">mydepartment</span><span class="p">,</span><span class="n">OU</span><span class="p">=</span><span class="n">Users</span> <span class="n">and</span> <span class="n">Groups</span><span class="p">,</span><span class="n">DC</span><span class="p">=</span><span class="n">mydomain</span><span class="p">,</span><span class="n">DC</span><span class="p">=</span><span class="n">ca</span>&quot; <span class="o">-</span><span class="n">members</span> <span class="o">|</span><span class="n">dsget</span> <span class="n">user</span> <span class="o">-</span><span class="n">samid</span> <span class="o">-</span><span class="n">display</span> <span class="o">-</span><span class="n">email</span> </pre></div> <p><img alt="dsget-list-name-emails.png" src="\HowTo\retrieve-members-of-ad-group\"\files\59\=dsget-list-name-emails.png"/></p> Searching Active Directory,2011-01-18:/HowTo/searching-active-directory 2011-01-18T07:00:00Z matt wilkie <p>This is from the <em>I can't believe I didn't know that and <strong>why</strong>haven't I been using it for the last ten years!??</em> department.</p> <p>Many times in the last decade I've<strong>needed to<strong><em>*search Active Directory</em></strong>*for a person</strong>'s name, login name, what members belong to what group and so on. Each time the need has been pressing enough I've gone out and searched the 'net for solutions, and each time after a time come up with a solution. Said time might have been 10 mintues or 2 hours depending on what in particular I was trying to do.  The solutions I've found and used ranged from the easy to use <a href="" title="">AD-Explorer</a> to roll your own python or batchfile scripts (that have taken further hours to develop to suit local circumstances).</p> <p>Encountering the need again today, and realizing my local stash of recipes was much <a href="\HowTo\searching-active-directory\Retrieve_members_of_AD_group.html" title="How To/Retrieve members of AD group">too scary or technical</a> to hand off to our regular staff who would actually be using it, I embarked on the quest again. Lo and behold, every version of Windows from (at least) XP on up includes the built in <em>Find Users, Contacts, and Groups</em>. It's buried under 'My Network Neighbourhood > Common Tasks' which I'd given up on long ago as being a useless "my computer" place to visit (when I want to see the network it's far more common to want to see the workstations and servers near me and turned on than the last 30 or more places I've been to). Who knew? (obviously not me!)</p> <p>Anyway, here's the meat of it and you can't put it anywhere you want, so it's actually convenient: Create a shortcut to <strong>rundll32.exe dsquery, OpenQueryWindow</strong>. And that's it, you're done.</p> <p>Sheesh.</p> <p>Thank you Information Systems &amp; Technology at University of Waterloo for <a href="" title="">illuminating me</a>.</p> Long Live the Aurora Text Editor,2010-10-20:/Musings/long-live-the-aurora-text-editor 2010-10-20T07:00:00Z matt wilkie <p>Thanks to the persistence of Kent Nassen and the generosity of Jeff Wunderlich, the <a href="" title="">Aurora Text Editor</a> can be downloaded and registered, “…the .bat file command I use to register is: ac -reg “Jeff Wunderlich” “xujcwpkgzngqmqn” 1 Feel free to share it with anyone on the net.”</p> <p>So what’s the big deal? Well I covered most of it a few years ago in <a href="" title="">Musing on the Favourite Text Editor</a> (slow link, involves time travel), but at the moment my biggest cause for joy is that I can, once again, select a column and inserting a character or range of characters (useful for turning fixed width text records into delimited text), and filling a columnar selection with an increasing pattern (1,2,3… a,b,c…) similar to Excel’s extend series command.</p> <p>Since I wrote the musing’s essay I’ve been a happy <a href="" title="">Cream</a> user,with occasional forays into <a href="" title="">Notepad++</a> when speed and portability (happy on a usb key) is desired. Now that Aurora is around and permanently free I’m positively cheery. Yes I know <a href="" title="">TextPad</a> can do column fills, and TP is a very fine editor indeed. I bought it for work after all, but it’s not cross platform and I don’t want to pay for it on for every computer I use. Besides, Aurora is only a 650kb download. Where else can you get so much editor bang for your buck? Seriously, where else?</p>