Combining ConfigParser and argparse with parse_known_args()

Edited to add: The best place to discuss this topic is on my original answer on StackOverflow.

I'm a fan of both the ConfigParser and argparse modules in my Python scripts and I've always thought it would be great to have a way to combine them. In other words allow the user of a script to provide a command line option that specified a configuration file that specified defaults for the command line options.

Recently I discovered the parse_known_args() method, which allows one to do just that.

Here's the script p.py that demonstrates this:


Here's a configuration file and a demonstration of how it works:


So this is what I was looking for, the caller can specify a configuration file with defaults, but override those defaults with more command line options.

There's only one problem, the help option only shows the configuration file option:


That's because the '-h' is processed by the parse_known_args() instead of the final parse_args(). The way to fix this is to create two ArgumentParsers and use the add_help parameter when creating first to suppress it from parsing -h:


Now help works like you would expect:

Using 'git stash' as a Todo list

I noticed I've been using 'git stash' as a todo list for simple development tasks. The scenario is that I'm doing some coding and run across a minor bug that I don't want to fix right at that moment, I write myself a todo as a comment right at the relevant piece of code, e.g.:

# The following function should check the argument and make sure
# it is an integer greater than zero otherwise it throws a really weird error:
#  ...some example of error...

And then save it to the stash with with a descriptive message (use 'git stash save --patch' if I have other stuff in my working tree):

git stash save "foo.c TODO: check argument is greater than zero"

Then later I use 'git stash pop' to bring my comments back into my working tree, make the fix and commit it. (You can name the TODO stash if you've save other things to the stash after the Todo.)

This is a reverse of the "interrupted workflow" example in the git-stash man page. Instead of interrupting my work for the bug, I'm pushing the bug into the stash so it doesn't interrupt me.

The nice thing is the comment is right in the code where it is relevant. Plus by using the string "TODO" in the message, I can quickly view any stashed todos:

% git stash list | grep -i todo
stash@{0}: On develop: foo.c TODO: check argument is greater than zero

Obviously this doesn't replace a real bug tracking system, but it works well for fixes I want to get to "soon" without interrupting my current flow. If it turns out I can't get to it soon, it's easily committed so I don't lose it.

Update 11/29: Someone on HN suggested branches were a better way to do this. I played with it briefly and worked out the following script to use branches (this is a quick hack and missing a lot of error checking, don't even thing about trying to use it): (Update: doesn't quite work as written, you need to use "git symbolic-ref HEAD" as described here.)

#!/bin/sh
msg=$1;shift
# Kudos: http://stackoverflow.com/questions/1593051/how-to-programmatically-determine-the-current-checked-out-git-branch
branch=`git name-rev --name-only HEAD` 
git checkout -b $msg
git add -p
git commit -m "$msg"
git checkout $branch

So this approach would work, and I kinda like the idea, but frankly by the time you polish this script enough to make it robust, you may as well just use a real issue tracker.

My first experience with backscatter security...

On my recent flight from Denver, I had my first experience with a new backscatter security scanner. And I have to say, I found it very disturbing.

First, it caught me by surprise, I was returning from a workshop, having a good conversation with a colleague, going through security on autopilot. I put my stuff on the x-ray belt and then the first unusual thing happened: the TSA guard asked if I anything in my pockets. "No metal" I said. "No anything" they clarified. Odd, I thought, and double checked. "OK, nothing" I affirmed, just my boarding pass in my hand. Then I was ushered towards a device next to the normal metal detector which looks like a clear vertical tube about the size of a broom closet (the photo in this LATimes article seems close).

Now unlike walking through a metal detector, where you basically walk through like you would a doorway, the backscatter process is much more controlled and time consuming. First they took my boarding pass. Then I had to step in and assume a fixed position with my hands in a triangle over my head (thumbs and index fingers touching - the photos I find online don't really show how high your hands have to be). I had to hold that position for several seconds while the device pivoted around me, closing the entrance I walked in and opening an exit towards the other side. While I was standing there all the images I'd seen online were going through my head and the thought that somewhere someone was now basically looking at me naked while I had to hold this pose. (It probably takes a moment for the image to develop so in fact they weren't looking at me yet.) 

Then I was told to wait in the device for a few more seconds, presumably until someone verified they got a good image. The guard at the exit had a walkie-talkie they were using through the process to communicate with someone looking at the image - I don't recall if they got a clear signal to let me exit or if they just waited a fixed period of time. After I exited I figured I was done, but I was then directed to another spot, outside the scanner, with foot outlines on the floor, and had to wait for another 5-10 seconds until the guard got a "male subject clear" message from their walkie-talkie (actually, the first time the message was garbled and the guard had to ask for a repeat, hence my knowledge it's a two-way communication channel).

Then I was given back my boarding pass and was free to go collect my stuff from the x-ray belt, where it had been sitting for a while at this point. (I wonder what would keep someone from walking off with it? I certainly was in no position to keep an eye on it. It wasn't clear TSA was.)

So why did I find it disturbing? First, I have to say the TSA guards were courtesy through out, they did nothing to contribute to my discomfort besides being part of the process. I can think of three reasons why it bothered me: (1) from start to finish, it's much more controlled. Unlike the metal detector, where you walk through on your own volition, it is a series of steps where you stand in specific places and poses - you feel very tightly controlled. (2) It's very asymmetric: somewhere, someone you can't see is scrutinizing you and you don't even know where or who they are. (3) It's very revealing. Having seen pictures online, I know just how laid bare I was. I've never been strip searched, so I won't say it's comparable, but with the combination of being tightly controlled and knowing I'm being seen basically naked, it's the only analogy that comes to mind.

Why was I picked to go through the device? Best I could tell, they just picked the next person in line as soon as the scanner opened up.

Did it make me feel safer? No. From everything I read (google for "odds of dying from terrorism"), I know I'm much less likely to die from a car crash, cancer and many other things. My odds of being killed by terrorism on that flight from Denver I'm sure were much lower than my odds from dying from human error on the flight.

Would I do it again? You can decline to undergo the process, but based on my experience, don't expect this option to be offered, you'll probably have to assert it. There is some debate as to how you'll be treated if you do so. I did find it disturbing enough that if I'm not in a hurry, I will decline it and take the alternative.

Added 9/20/2010: Some experiences with declining to undergo the process collected by Lauren Winstein.

 

New professional blog: blog.vonwelch.com

I've started a new professional blog at blog.vonwelch.com. This blog will contain professional updates as well as my thoughts on computing and computer security. Having this blog will give me a place to editorialize and keep Security Unwrapped focused on explanations of cybersecurity. blog.vwelch.com (this blog you are reading now) will continue to be the repository of my personal thoughts unrelated to things computational.

Additionally I have "How did I do that?" - my collection of miscellaneous how tos - and Von's Ubuntu Experiences, my ongoing diary of Ubuntu/Mythbuntu system administration.

Look out Mickey!

DisneyLand uses, and cares for, feral cats to control rodents (via Lauren):

Years ago — no one seems to know when — feral cats began to sneak into the park, living among the park's trees and shrubs during the day. At night, they venture out, and an estimated 200 cats now prowl through Disneyland and neighboring California Adventure Park.

But instead of evicting the cats, Disneyland's animal wranglers work to control the feline population by spaying and neutering the adult cats and finding homes for all kittens born in the resort. The cats eat at five permanent feeding stations installed throughout the two parks.

New mortiser table

At right is my stock mortiser and table. It was OK but had a few short-comings:

• The table is rather short (~9"), so it didn't support long stock like table legs well. I was constantly having to come up with ways of supporting work pieces.
• The fence was short, so it was hard to clamp stops for the start and stops of the mortises when I was doing multiple identical workpieces.
• The short fence also made it hard to clamp the workpiece in place to keep it from drifting either right-left or away from the fence.

So I decided to replace the table with one of my own making, though I took a lot of guidance from Rockler's version.

Here's my version at right. It has a few improvements over the stock version:

• The table is a full 18" wide to better support stock.
• The fence has t-tracks in both the face and the top to allow for a variety of stops. I use a flip stop in the top track to set the start of my mortise and a fixed stop in the face to set the end of the mortise.
• The table also has two tracks perpendicular to the fence to allow for clamping the workpiece and keeping it tight to the fence. I use one rotary guide on the left to keep the workpiece against the fence (I ordered a pair of guides from Rockler, though they don't seem to sell them separately from their table at this time). On the right I use a fence clamp to hold the workpiece not only against the fence but in place horizontally during the down stroke of the mortiser.

For more photos showing greater detail, see my photo album.

A great post on standards by Adam Bosworth

Having been involved in a few standards myself, I couldn't agree more with Adam Bosworth's post on lessons learned about standards (via Joel Spolsky).