The aha! moment

2010-02-19 22:07

I had a small task today in Marave. The goal was:

Fade in a widget
Set a variable
Fade in another one

It's important that things are done in that order and it's also important that the app remains interactive.

And here's the code to do that (simplified):

def fadein(thing, target=1., thendo=None):
    """
    * thing is a QWidget
    * thing.proxy is a QGraphicsWidget
    * thendo is callable
    * target is the desired opacity
    """

    thing.anim=QtCore.QPropertyAnimation(thing.proxy, "opacity")
    thing.anim.setDuration(200)
    thing.anim.setStartValue(thing.proxy.opacity())
    thing.anim.setEndValue(target)
    thing.anim.start()
    thing.anim.finished.connect(thing.anim.deleteLater)
    if thendo:
        thing.anim.finished.connect(thendo)

And this is how you use it:

def later():
    avar=avalue
    fadein(widget2)

fadein(widget1, thendo=later)

Isn't that lovely? Having functions as first class objects means I can just take later as a closure, along with widget2 and avar, which need only be defined in the local scope, and the call chain will work just as I wanted!

Yes, many other languages can do this, and in Javascript it's one of the most common tricks, but consider that PyQt is a wrapper for a C++ library!

I think this kind of usage shows the real added value PyQt brings to the table, it's not just that python avoids the boring compiles, or that you have the awesome standard library to use, but that the language itself enables you to do things that are not practical in C++.

In C++ the only way I can think of is creating a slot that's the equivalent of later, then chaining the signals... which means that this throwaway later becomes part of the interface of a class!

I would have to define later somewhere else on the file, separate from its only usage (maybe even inlined in the header file).

Even then, that's not equivalent: avalue may be something that was only avalable before the first call to fadein, (for example, the time of the first fadein): I would have to create a place to store it, make it reachable by later... and wht happens if you try to do this again while the first fadein is in progress?... it gets hairy.

Programming is like a slap in the face sometimes... you realize that things you use without even noticing are far from trivial.

So, remember young padawan: you can choose you tools. Choose wisely.

Extending Marave

2010-02-18 18:50

Marave is a text editor. If there's one thing that's true of most text editors, it's this: they lack the exact features you need.

So, the solution, in the ancient tradition of Emacs and Vim is... make it extensible.

I am a big fan of programs that can be extended by users.

So... here's the anatomy of a Marave plugin as it stands right now on SVN trunk, which of course can change any minute.

Creating a plugin

You just need to create a .py file in the plugins folder.

Here's the most basic plugin, which does nothing:

# -*- coding: utf-8 -*-

from plugins import Plugin
class Smarty(Plugin):
    name='smarty'
    shortcut='Ctrl+.'
    description='Smart quote and dash replacement'
    mode="qBde"

Default values for anything configurable (in this case, "mode") is just added to the class.

The mandatory fields:

shortcut: a keyboard shortcut that triggers this plugin
name: a short name
description: a one-line description of what it does

What does it do? It adds the plugin to the plugin list in the prefs dialog, and you can open its configuration dialog, where you can change the shortcut:

If you enable this plugin, whenever the shortcut is used the "run" method of the plugin is called.

Making the Plugin Configurable

This plugin supports different modes of operation. To make this reachable to the user, you need to implement a few extra methods.

The addConfigWidgets method takes a dialog argument and adds whatever you want there:

@classmethod
def addConfigWidgets(self, dialog):
    print 'Adding widgets to smarty config'
    l=dialog.ui.layout
    self.q=QtGui.QCheckBox(dialog.tr('Replace normal quotes'))
    if 'q' in self.mode:
        self.q.setChecked(True)
    self.b=QtGui.QCheckBox(dialog.tr('Replace backtick-style quotes (` and ``)'))
    if 'B' in self.mode:
        self.b.setChecked(True)
    self.d=QtGui.QCheckBox(dialog.tr('Replace -- by en-dash, --- by em-dash'))
    if 'd' in self.mode:
        self.d.setChecked(True)
    self.e=QtGui.QCheckBox(dialog.tr('Replace ellipses'))
    if 'e' in self.mode:
        self.e.setChecked(True)
    l.addWidget(self.q)
    l.addWidget(self.b)
    l.addWidget(self.d)
    l.addWidget(self.e)

And then the config dialog will look like this:

But then you need to save those options somewhere, which you do reimplementing saveConfig:

@classmethod
def saveConfig(self, dialog):

    self.shortcut=unicode(dialog.ui.shortcut.text())
    self.settings.setValue('plugin-'+self.name+'-shortcut', self.shortcut)

    newmode=""
    if self.q.isChecked():
        newmode+='q'
    if self.b.isChecked():
        newmode+='B'
    if self.d.isChecked():
        newmode+='d'
    if self.e.isChecked():
        newmode+='e'
    self.mode=newmode

    self.settings.setValue('plugin-smarty-mode',self.mode)
    self.settings.sync()

And you need to load those settings and put them in your class, too:

@classmethod
def loadConfig(self):
    print 'SMARTY loadconfig', self.settings
    if self.settings:
        sc=self.settings.value('plugin-'+self.name+'-shortcut')
        if sc.isValid():
            self.shortcut=unicode(sc.toString())
        mode=self.settings.value('plugin-smarty-mode')
        if mode.isValid():
            self.mode=unicode(mode.toString())

Making it Work

And yes, you need to make it do something useful. The plugin has access to a "client" which is Marave's main window. Everything is available there, somewhere ;-)

def run(self):
    print 'running smarty plugin'
    text=unicode(self.client.editor.toPlainText()).splitlines()
    prog=QtGui.QProgressDialog(self.client.tr("Applying smarty"),
                               self.client.tr("Cancel"),
                               0,len(text),
                               self.client)
    prog.show()
    output=[]
    for i,l in enumerate(text):
        output.append(unescape(smartyPants(l,self.mode)))
        prog.setValue(i)
        QtGui.QApplication.instance().processEvents()
    prog.hide()
    self.client.editor.setPlainText('\n'.join(output))

And there it is, if you enable the smarty plugin, you can "fix" your quotes, dashes and ellipsis with a key combination :-)

Full source code here: http://code.google.com/p/marave/source/browse/trunk/marave/plugins/smarty.py

Still to be done: other ways to integrate plugins into the UI, buttons, panels, etc.

Yak Shavings for February 16, 2010

2010-02-16 19:20

yak shaving: (idiomatic) Any apparently useless activity which, by allowing you to overcome intermediate difficulties, allows you to solve a larger problem.

A while ago, I wrote how I implemented a generic syntax highlighter for PyQt using Pygments.

I got a request for such a feature in Marave, so I digged that code and... it's freaking useless. It's just too slow for reasonable use.

So, that yak's hair is all grown up again, and I just got this new pair of scissors!

The goal is a way to highlight syntax in a QPlainTextEdit that:

Doesn't require programming to add a new highlighter
Doesn't require programming to add a new color scheme
Doesn't require me to spend a year writing highlighters for existing languages
Is fast enough

A quick google shows that for C++ you can use Source highlight qt which is based on GNU source highlight.

Alas, no python binding that I could find. So, let's write one!

Here it is: http://marave.googlecode.com/svn/trunk/marave/highlight/

And here's a screenshot of the demo program running, showing itself in its entirety:

You can create a color scheme using CSS, a language definition is a text file, there are a bazillion already written, it seems to be fast enough.

So, another yak shaved, another feature (not finished!) for Marave

Marave 0.5 is out!

2010-02-12 22:25

Just uploaded Marave 0.5 to the usual place. Marave is a relaxed, fullscreen text editor that tries not to distract you.

It even includes a simple music player just so you don't have to think about switching to another application!

This release has several bugs fixed, and looks a bit nicer.

The main new feature is ... internationalizacion. It now includes a spanish translation only, but if you want to help translating it to any other language, please step forward!

There are only 60 phrases to translate, so it shouldn't be more than one hour of work.

Here's a screenshot of this release:

Marave is free software under the GPLv2, and should work on any platform where PyQt works, which means Windows, Mac, and Unix-like operating systems, at least.

Packaging and shipping is HARD

2010-02-11 14:00

I have worked really hard on Marave, a full screen editor in the style of ommwriter, DarkRoom, WriteRoom, pyRoom, etc. I have worked very hard and I want users to use it.

Or not even that, I want them to have a chance of using it.

That means I want it to work on Windows (and maybe OSX some day, too, if someone helps me). Which means, I have to pakage it for windows.

Let's do a quick comparison here from the points of view of the user and the developer.

The User, In Linux

This is in Arch Linux, which is what I use, in other Linux variants it will be pretty much the same once Marave is a bit more well known.

yaourt -S marave-svn --noconfirm

That gets the code from SVN (right now it's the best way, later I will package releases, too), all required dependencies, builds and installs. It takes all of 15 seconds in my notebook.

After that, you have a fully working Marave.

In case it's not packaged for your distro, just install PyQt (which surely is) and run this command:

easy_install marave

The User, in Windows

You go to http://marave.googlecode.com, click on "Marave-0.5.win32.exe" (Not linked yet, it's not finished), then download a 10MB program. That is a 10MB program because windows doesn't believe in packages and dependencies. On Linux, a Marave package could be under 1MB (most of it images), and not be executable, just data.

Of course nowadays web browsers don't actually run programs on download, so... let's see it as a gallery!

Yes, save it.

Double click to open it

Yes, I agree

Sure, whatever

Nice...

Good to hear!

Now, this Marave that just got installed may or may not currently work because of a missing MSVCR90.DLL but that's for the next section...

The Developer, in Linux

First, here's the biggest problem a Linux packager can have:

Since Marave is a new app, and I develop it in the rather cutting-edge Arch Linux, it uses some newish features only available in recent versions of Qt. In fact, it doesn't work with PyQt < 4.6, which is not available in some slow distros, like Debian, or even in a not-latest Ubuntu.

Solution? Well, I could just ignore it, but what the heck, let's fix it instead!

Thanks to PyInstaller it's not even hard to do, here's the spec file:

a = Analysis([os.path.join(HOMEPATH,'support/_mountzlib.py'), os.path.join(HOMEPATH,'support/useUnicode.py'), 'marave/main.py'],
            pathex=['/home/ralsina/trunk/trunk'])

pyz = PYZ(a.pure)
exe = EXE(pyz,
        a.scripts,
        exclude_binaries=1,
        name=os.path.join('build/pyi.linux2/main', 'marave.exe'),
        debug=False,
        strip=False,
        upx=True,
        console=0 )

coll = COLLECT( exe,
            a.binaries,
            [('radios.txt','marave/radios.txt','DATA')],
            Tree('marave/icons','icons'),
            Tree('marave/backgrounds','backgrounds'),
            Tree('marave/clicks','clicks'),
            Tree('marave/stylesheets','stylesheets'),
            Tree('marave/themes','themes'),
            a.zipfiles,
            a.datas,
            strip=False,
            upx=True,
            name=os.path.join('dist', 'marave'))

Use this, and PyInstaller will produce a nice folder full of everything Marave needs to run on any Linux.

OTOH, if you can rely on a recent PyQt being available, it's also simple. Here's a packaging configuration for a similar package in Arch Linux (I must confess not having done one for Marave yet). For other distributions it should be about as simple, if more verbose, and someone else probably does it for you:

# Contributor: Roberto Alsina <ralsina@kde.org>
pkgname=python-rst2pdf
pkgver=0.12.1
pkgrel=4
pkgdesc="Create PDFs from simple text markup, no LaTeX required."
arch=('i686' 'x86_64')
url="http://rst2pdf.googlecode.com"
license=('custom')
depends=('python' 'setuptools' 'docutils' 'pygments' 'python-reportlab' 'python-simplejson' 'pil')
source=(http://rst2pdf.googlecode.com/files/rst2pdf-$pkgver.tar.gz LICENSE.txt)
optdepends=('uniconvertor: vector images support'
            'python-svglib: SVG support'
            'python-wordaxe: hyphenation'
            'pythonmagick: PDF images support')
build() {
cd $startdir/src/rst2pdf-$pkgver
python setup.py install --root=$startdir/pkg || return 1
install -D ../LICENSE.txt $startdir/pkg/usr/share/licenses/python-rst2pdf/COPYING
install -D doc/rst2pdf.1 $startdir/pkg/usr/share/man/man1/rst2pdf.1
}
md5sums=('ea6beda9a46f34ba42c4c94d48cc607a'
        '416f8046c66b9476cdbacda69a673afe')

And that's all you need to know about the process of packaging your app for Linux. It's easy to do, and most of the time, easy to do right!

Now, let's go to our final section...

Windows for the developer

First, remember that of relying on the system's version of Qt? Forget it, there is no system version available. And no python either. And noone is going to install it or your app, so it's "ship everything yourself" mode, or nothing.

But anyway, PyInstaller works for Windows too! So, using the same spec file, it works. Right?

Well, no beause of two problems.

Problem 1: You need an installer

Users are not going to open a zip somewhere, then do a shortcut to the binary on Windows, so you need to do some operations, and that means an installer.

Here's what I came up with to use NSIS, a free installer creator for Windows:

;--------------------------------
;Include Modern UI

!include "MUI2.nsh"

;--------------------------------
;General

;Name and file
Name "Marave"
OutFile "Marave-0.5.win32.exe"

;Default installation folder
InstallDir "$LOCALAPPDATA\Marave"

;Get installation folder from registry if available
InstallDirRegKey HKCU "Software\Marave" ""

;Request application privileges for Windows Vista
RequestExecutionLevel user

;--------------------------------
;Interface Settings

!define MUI_ABORTWARNING

;--------------------------------
;Pages

!insertmacro MUI_PAGE_LICENSE "LICENSE"
!insertmacro MUI_PAGE_DIRECTORY
!insertmacro MUI_PAGE_INSTFILES

!insertmacro MUI_UNPAGE_CONFIRM
!insertmacro MUI_UNPAGE_INSTFILES

;--------------------------------
;Languages

!insertmacro MUI_LANGUAGE "English"

;--------------------------------
;Installer Sections

Section "Install"

SetOutPath "$INSTDIR"
File /r "dist\marave"


;Store installation folder
WriteRegStr HKCU "Software\Marave" "" $INSTDIR

;Create uninstaller
WriteUninstaller "$INSTDIR\Uninstall.exe"

;Create shortcuts
CreateDirectory $SMPROGRAMS\Marave
CreateShortCut "$SMPROGRAMS\Marave\Marave.lnk" "$INSTDIR\marave\marave.exe" ; use defaults for parameters, icon, etc.
CreateShortCut "$SMPROGRAMS\Marave\Uninstall Marave.lnk" "$INSTDIR\Uninstall.exe" ; use defaults for parameters, icon, etc.

SectionEnd


;--------------------------------
;Uninstaller Section

Section "Uninstall"

Delete "$INSTDIR\Uninstall.exe"
RMDir /r "$INSTDIR"

DeleteRegKey /ifempty HKCU "Software\Marave"

SectionEnd

It's comparable to the effort of building a packaging file, really, except every time you want to test it... you install it. There is no way (AFAICS) to see what's inside the installer except running it!

When things fail, you get no error messages, at least not the kind that is useful for a developer, the guy that needs to know what went wrong.

And after it's finished, you may end with a non-working program because of...

Problem 2: system libraries don't exist

Python 2.6 binaries are built using Visual Studio. That means they require the Visual Studio Runtime, specifically MSVCR90.DLL. That contains what on Linux would be considered part of libc. (linux guy: imagine apps that depend on a specific libc... hard to do!)

On Linux that's part of the system. Further, if you wanted, you can redistribute it. On Windows... well, it's a bit different.

It's part of the "Visual C++ redistributables"
Installing that doesn't guarantee it will work (yes, I have tried)
The license for those 'redistributables' says you can't make them available for download.

I have been told that including that in your installer is fine and dandy, but how is that not making them available for download?

So what can you do when you need a library and can't ship it and the user won't install it?

Well, that's why there is no Windows binary of Marave yet. Of course if anyone can help, I'd be really, really happy!

Ralsina.Me — Roberto Alsina's website

Posts about open source (old posts, page 6)