This has been a pet peeve of mine for years: pro­gram­ming shell scripts suck. They are ug­ly and er­ror prone. The on­ly rea­son why we still do it? There is no re­al re­place­men­t.

Or at least that was the case, un­til to­day I met iter­pipes at python.red­dit.­com

Iter­pipes is "A li­brary for run­ning shell pipe­lines us­ing shel­l-­like syn­tax" and guess what? It's bril­liant.

Here's an ex­am­ple from its PYPI page:

# Total lines in *.py files under /path/to/dir,
# use safe shell parameters formatting:

>>> total = cmd(
...     'find {} -name {} -print0 | xargs -0 wc -l | tail -1 | awk {}',
...     '/path/to/dir', '\*.py', '{print $1}')
>>> run(total | strip() | join | int)
315

Here's how that would look in shel­l:

find /path/to/dir -name '*.py' -print0 | xargs -0 wc -l | tail -1 | awk '{print $1}'

You may say the shell ver­sion looks bet­ter. That's an il­lu­sion caused by the evil that is shell script­ing: the shell ver­sion is bug­gy.

Why is it buggy? Because if I control what's inside /path/to/dir I can make that neat little shell command fail [1], but at least in python I can handle errors!

Al­so, in most ver­sions you could at­tempt to write, this com­mand would be un­safe be­cause quot­ing and es­cap­ing in shell is in­sane!

The iter­pipes ver­sion us­es the equiv­a­lent of SQL pre­pared state­ments which are much safer.

It's near­ly im­pos­si­ble to do such a com­mand in pure shell and be sure it's safe.

Al­so, the shell ver­sion pro­duces a string in­stead of an in­te­ger, which sucks if you in­tend to do any­thing with it.

And the most im­por­tant ben­e­fit is, of course, not when you try to make python act like a shel­l, but when you can stop pre­tend­ing shell is a re­al pro­gram­ming lan­guage.

Consider this gem from Arch Linux's /etc/rc.shutdown script. Here, DAEMONS is a list of things that started on boot, and this script is trying to shut them down in reverse order, unless the daemon name starts with "!":

# Shutdown daemons in reverse order
let i=${#DAEMONS[@]}-1
while [ $i -ge 0 ]; do
        if [ "${DAEMONS[$i]:0:1}" != '!' ]; then
                ck_daemon ${DAEMONS[$i]#@} || stop_daemon ${DAEMONS[$i]#@}
        fi
        let i=i-1
done

Nice uh?

Now, how would that look in python (I may have in­vert­ed the mean­ing of ck­_­dae­mon)?

# Shutdown daemons in reverse order
for daemon in reversed(DAEMONS):
    if daemon[0]=='!':
        continue
    if ck_daemon(daemon):
        stop_daemon(daemon)

Where stop_­dae­mon used to be this:

stop_daemon() {
    /etc/rc.d/$1 stop
}

And will now be this:

def stop_daemon(daemon):
    run(cmd('/etc/rc.d/{} stop',daemon))

So, come on, peo­ple, we are in the 21st cen­tu­ry, and shell script­ing sucked in the 20th al­ready.