A few days ago I saw a men­tion in twit­ter about a lan­guage called Nim

Se ve in­tere­sante ~90% Python-­like, com­pi­la a Bi­na­rio o JS, no de­pen­den­cias ni run­time.http­s://t.­co/JF3vIS­cWqG

— Juan_­Car­los (@juan­car­lospa­co) 27 de abril de 2018

And... why not. I am a bit stale in my pro­gram­ming lan­guage va­ri­ety. I used to be flu­ent in a dozen, now I do 80% python 10% go, some JS and al­most noth­ing else. Be­cause I learn by do­ing, I de­cid­ed to do some­thing. Be­cause I did not want a prob­lem I did not know how to solve to get in the way of the lan­guage, I de­cid­ed to reim­ple­ment the ex­am­ple for the python book I am writ­ing: a text lay­out en­gine that out­puts SVG, based on harf­buz­z, freetype2 and oth­er things.

This is a good learn­ing project for me, be­cause a lot of my cod­ing is glue­ing things to­geth­er, I hard­ly ev­er do things from scratch.

So, I de­cid­ed to start in some­what ran­dom or­der.

Preparation

I read the Nim Tu­to­ri­al quick­ly. I end­ed re­fer­ring to it and to Nim by ex­am­ple a lot. While try­ing out a new lan­guage one is bound to for­get syn­tax. It hap­pen­s.

Wrote a few "hel­lo world" 5 line pro­grams to see that the ecosys­tem was in­stalled cor­rect­ly. Im­pres­sion: builds are fast-ish. They can get ac­tu­al­ly fast if you start us­ing tcc in­stead of gc­c.

SVG Output

I looked for li­braries that were the equiv­a­lent of svg­write, which I am us­ing on the python side. Sad­ly, such a thing does­n't seem to ex­ist for nim. So, I wrote my own. It's very rudi­men­ta­ry, and sure­ly the nim code is garbage for ex­pe­ri­enced nim coder­s, but I end­ed us­ing the xml­tree mod­ule of nim's stan­dard li­brary and ev­ery­thing!

import xmltree
import strtabs
import strformat

type
    Drawing* = tuple[fname: string, document: XmlNode]

proc NewDrawing*(fname: string, height:string="100", width:string="100"): Drawing =
    result = (
        fname: fname,
        document: <>svg()
    )
    var attrs = newStringTable()
    attrs["baseProfile"] = "full"
    attrs["version"] = "1.1"
    attrs["xmlns"] = "http://www.w3.org/2000/svg"
    attrs["xmlns:ev"] = "http://www.w3.org/2001/xml-events"
    attrs["xmlns:xlink"] = "http://www.w3.org/1999/xlink"
    attrs["height"] = height
    attrs["width"] = width
    result.document.attrs = attrs

proc Add*(d: Drawing, node: XmlNode): void =
    d.document.add(node)

proc Rect*(x: string, y: string, width: string, height: string, fill: string="blue"): XmlNode =
    result = <>rect(
        x=x,
        y=y,
        width=width,
        height=height,
        fill=fill
    )

proc Text*(text: string, x: string, y: string, font_size: string, font_family: string="Arial"): XmlNode =
    result = <>text(newText(text))
    var attrs = newStringTable()
    attrs["x"] = x
    attrs["y"] = y
    attrs["font-size"] = font_size
    attrs["font-family"] = font_family
    result.attrs = attrs

proc Save*(d:Drawing): void =
   writeFile(d.fname,xmlHeader & $(d.document))

when isMainModule:
    var d = NewDrawing("foo.svg", width=fmt"{width}cm", height=fmt"{height}cm")
    d.Add(Rect("10cm","10cm","15cm","15cm","white"))
    d.Add(Text("HOLA","12cm","12cm","2cm"))
    d.Save()

While writ­ing this I ran in­to a few is­sues and saw a few nice things:

To build a svg{.nimrod} tag, you can use <>svg(attr=value){.nimrod} which is delightful syntax. But what happens if the attr is "xmlns:ev"{.nimrod}? That is not a valid identifier, so it doesn't work. So I worked around it by creating a StringTable{.nimrod} filling it and setting all attributes at once.

A good thing is the when{.nimrod} keyword. using it as when isMainModule{.nimrod} means that code is built and executed when svgwrite.nim{.nimrod} is built standalone, and not when used as a module.

An­oth­er good thing is the syn­tax sug­ar for what in python we would call "ob­jec­t's meth­od­s".

Because Add{.nimrod} takes a Drawing{.nimrod} as first argument, you can just call d.Add(){.nimrod} if d{.nimrod} is a Drawing{.nimrod}. It's simple, it's clear and it's useful and I like it.

One bad thing is that some­times im­port­ing a mod­ule will cause weird er­rors that are hard to guess. For ex­am­ple, this sim­pli­fied ver­sion fails to build:

import xmltree

type
    Drawing* = tuple[fname: string, document: XmlNode]

proc NewDrawing*(fname: string, height:string="100", width:string="100"): Drawing =
    result = (
        fname: fname,
        document: <>svg(width=width, height=height)
    )

when isMainModule:
    var d = NewDrawing("foo.svg")

$ nim c  svg1.nim
Hint: used config file '/etc/nim.cfg' [Conf]
Hint: system [Processing]
Hint: svg1 [Processing]
Hint: xmltree [Processing]
Hint: macros [Processing]
Hint: strtabs [Processing]
Hint: hashes [Processing]
Hint: strutils [Processing]
Hint: parseutils [Processing]
Hint: math [Processing]
Hint: algorithm [Processing]
Hint: os [Processing]
Hint: times [Processing]
Hint: posix [Processing]
Hint: ospaths [Processing]
svg1.nim(9, 19) template/generic instantiation from here
lib/nim/core/macros.nim(556, 26) Error: undeclared identifier: 'newStringTable'

WAT? I am not using newStringTable anywhere! The solution is to add import strtabs{.nimrod} which defines it, but there is really no way to guess which imports will trigger this sort of issue. If it's possible that importing a random module will trigger some weird failure with something that is not part of the stdlib and I need to figure it out... it can hurt.

In any case: it worked! My first work­ing, use­ful nim code!

Doing a script with options / parameters

In my python ver­sion I was us­ing do­copt and this was smooth: there is a nim ver­sion of do­copt and us­ing it was as easy as:

1. nimble install docopt
2. import docopt{.nimrod} in the script

The us­age is re­mark­ably sim­i­lar to python:

import docopt
when isMainModule:
    let doc = """Usage:
    boxes <input> <output> [--page-size=<WxH>] [--separation=<sep>]
    boxes --version"""

    let arguments = docopt(doc, version="Boxes 0.13")
    var (w,h) = (30f, 50f)
    if arguments["--page-size"]:
        let sizes = ($arguments["--page-size"]).split("x")
        w = parse_float(sizes[0])
        h = parse_float(sizes[1])

    var separation = 0.05
    if arguments["--separation"]:
        separation = parse_float($arguments["--separation"])
    var input = $arguments["<input>"]
    var output = $arguments["<output>"]

Not much to say, other that the code for parsing --page-size is slightly less graceful than I would like because I can't figure out how to split the string and convert to float at once.

So, at that point I sort of have the skele­ton of the pro­gram done. The miss­ing pieces are call­ing harf­buzz and freetype2 to fig­ure out text sizes and so on.

Interfacing with C libs

One of the main sell­ing points of Nim is that it in­ter­faces with C and C++ in a straight­for­ward man­ner. So, since no­body had wrapped harf­buzz un­til now, I could try to do it my­self!

First I tried to get c2n­im work­ing, since it's the rec­om­mend­ed way to do it. Sad­ly, the ver­sion of nim that ships in Arch is not able to build c2n­im via nim­ble, and I end­ed hav­ing to man­u­al­ly build nim-git and c2n­im-git ... which took quite a while to get right.

And then c2n­im just failed.

So then I tried to do it man­u­al­ly. It start­ed well!

• To link li­braries you just use prag­mas: {.link: "/us­r/lib/lib­harf­buz­z.­so".}{.n­im­rod}

• To de­clare types which are equiv­a­lent to void *{.n­im­rod} just use dis­tinct point­er{.n­im­rod}

• To de­­clare a func­­tion just do some gym­­nas­tic­s:

  proc cre­ate*(): Buf­fer {.­head­er: "harf­buz­z/h­b.h", im­portc: "h­b_buffer­_$1" .}{.n­im­rod}

• Cre­ates a nim func­tion called cre­ate{.n­im­rod} (the * means it's "ex­port­ed")

• It is a wrap­per around hb_buffer­_cre­ate{.n­im­rod} (see the syn­tax there? That is nice!)

• Says it's de­­clared in C in "har­f­buz­z/h­b.h"

• It re­turns a Buf­fer{.n­im­rod} which is de­clared thus:

type
    Buffer* = distinct pointer

Here is all I could do try­ing to wrap what I need­ed:

{.link: "/usr/lib/libharfbuzz.so".}
{.pragma: ftimport, cdecl, importc, dynlib: "/usr/lib/libfreetype.so.6".}

type
    Buffer* = distinct pointer
    Face* = distinct pointer
    Font* = distinct pointer

    FT_Library*   = distinct pointer
    FT_Face*   = distinct pointer
    FT_Error* = cint

proc create*(): Buffer {.header: "harfbuzz/hb.h", importc: "hb_buffer_$1" .}
proc add_utf8*(buffer: Buffer, text: cstring, textLength:int, item_offset:int, itemLength:int) {.importc: "hb_buffer_$1", nodecl.}
proc guess_segment_properties*( buffer: Buffer): void {.header: "harfbuzz/hb.h", importc: "hb_buffer_$1" .}
proc create_referenced(face: FT_Face): Font {.header: "harfbuzz/hb.h", importc: "hb_ft_font_$1" .}
proc shape(font: Font, buf: Buffer, features: pointer, num_features: int): void {.header: "harfbuzz/hb.h", importc: "hb_$1" .}

proc FT_Init_FreeType*(library: var FT_Library): FT_Error {.ft_import.}
proc FT_Done_FreeType*(library: FT_Library): FT_Error {.ft_import.}
proc FT_New_Face*(library: FT_Library, path: cstring, face_index: clong, face: var FT_Face): FT_Error {.ft_import.}
proc FT_Set_Char_Size(face: FT_Face, width: float, height: float, h_res: int, v_res: int): FT_Error {.ft_import.}

var buf: Buffer = create()
buf.add_utf8("Hello", -1, 0, -1)
buf.guess_segment_properties()

var library: FT_Library
assert(0 == FT_Init_FreeType(library))
var face: FT_Face
assert(0 == FT_New_Face(library,"/usr/share/fonts/ttf-linux-libertine/LinLibertine_R.otf", 0, face))
assert(0 == face.FT_Set_Char_Size(1, 1, 64, 64))
var font = face.create_referenced()
font.shape(buf, nil, 0)

Sad­ly, this seg­faults and I have no idea how to de­bug it. It's prob­a­bly close to right? Maybe some nim coder can fig­ure it out and help me?

In any case, con­clu­sion time!

Conclusions

• I like the lan­guage
• I like the syn­tax
• nim­ble, the pack­age man­ag­er is cool
• Is there an equiv­a­lent of vir­tualen­vs? Is it nec­es­sary?
• The C wrap­ping is, in­deed, easy. When it work­s.
• The avail­abil­i­ty of 3rd par­ty code is of course not as large as with oth­er lan­guages
• The com­pil­ing / build­ing is cool
• There are some strange bugs, which is to be ex­pect­ed
• Tool­ing is ok. VS­Code has a work­ing ex­ten­sion for it. I miss an opin­ion­at­ed for­mat­ter.
• It pro­duces fast code.
• It builds fast.

I will keep it in mind if I need to write fast code with lim­it­ed de­pen­den­cies on ex­ter­nal li­braries.

As part of a book project aimed at al­most-be­gin­ning pro­gram­mers I have writ­ten what may as well pass as the first part of a Git tu­to­ri­al. It's to­tal­ly stan­dalone, so it may be in­ter­est­ing out­side the con­text of the book.

It's aimed at peo­ple who, of course, don't know Git and could use it as a lo­cal ver­sion con­trol sys­tem. In the next chap­ter (be­ing writ­ten) I cov­er things like re­motes and push/pul­l.

So, if you want to read it: Git tu­to­ri­al for peo­ple who don't know git (part I)

PS: If the di­a­grams are all black and white, reload the page. Yes, it's a JS is­sue. Yes, I know how to fix it.

So, 9 years ago I wrote a post about how I would love a tool that took a JSON da­ta file, a Mako tem­plate, and gen­er­at­ed a re­port us­ing re­Struc­tured Tex­t.

If you don't like that, pre­tend it says YAM­L, Jin­ja2 and Mark­down. Any­way, same idea. Re­ports are not some crazy dif­fi­cult thing, un­less you have very de­mand­ing lay­out or need to add a ton of log­ic.

And hey, if you do need to add a ton of log­ic, you do know python, so how hard can it be to add the miss­ing bit­s?

Well, not very hard. So here it is, 9 years lat­er be­cause I am sit­ting at an au­di­to­ri­um and the guy giv­ing the talk is hav­ing com­put­er prob­lem­s.

Lois Lane Re­ports from PyP­I. and GitHub

Gyro grows some legs

It was just a few days ago that I start­ed an ex­per­i­men­tal wi­ki project called Gy­ro ... it's al­ways fun when a project just grows fea­tures or­gan­i­cal­ly. It does this, so it makes sense to make it do that, and then this oth­er thing is easy, and so on.

So, here is what hap­pened with Gy­ro:

• Fed­eri­co Cin­golani made it run on dock­er
• I added some fea­tures:
  • UI for cre­at­ing new pages
  • UI for delet­ing pages
  • Sup­­port for mul­ti­lev­el pages (so you can have "foo" and "foo/bar")
  • Au­­to­­com­­ple­­tion with ti­­tles in search
  • Bread­­crumbs so you can ac­­tu­al­­ly fol­low the mul­ti­lev­el pages
  • Lots of code cleanup
  • Themes (via Bootswatch)
  • Cus­­tom fonts (via Google We­b­­Fonts)
  • Au­­to­­mat­ic link­ing for Wik­i­­Words if you like that kind of thing

And, I pub­lished it as a Google Chrome Ex­ten­sion ... so you can now have a wi­ki on your chrome. If you saw how it worked be­fore, you may won­der how it be­came an ex­ten­sion, since those are pure Javascrip­t. Well... I made it have plug­gable back­end­s, so it can ei­ther use the old­er San­ic-based python API or use Lo­cal­Stor­age and just save things in­side your brows­er.

The be­hav­ior is iden­ti­cal in both cas­es, it's just a mat­ter of where things are saved, and how they are re­trieved. The goal is that you should not be able to tell apart one im­ple­men­ta­tion from the oth­er, but of course YM­MV.

And since I was al­ready do­ing a chrome ex­ten­sion ... how hard would it be to run it as an elec­tron "desk­top" ap­p? Well, not very. In fac­t, there are no code changes at al­l. It's just a mat­ter of pack­ag­ing.

And then how about releasing it as a snap for Ubuntu? Well, easy too, just try snap install gyro --beta

Is it fin­ished? Of course not! A non ex­haus­tive list of miss­ing MVP fea­tures in­clude:

• Im­port / Ex­port da­ta
• A sync­ing back­end
• Gen­er­al UI pol­ish (wid­get fo­cus, kbd short­cut­s)
• Bet­ter er­ror han­dling
• Gen­er­al test­ing

But in any case, it's nice to see an app take shape this fast and this pain­less­ly.

History

Facu­batis­ta: ralsi­na, yo, vos, cerveza, un lo­cal-wik­i-server-he­cho-en-un-­solo-.py-­con-in­ter­faz-web en tres ho­ras, pen­sa­lo

Facu­batis­ta: ralsi­na, you, me, beer, a lo­cal-wik­i-server-­done-in-one-.py-with­-we­b-in­ter­face in three hours, think about it

So, I could not get to­geth­er with Facu, but I did sort of write it, and it's Gy­ro. [1]

Technical Details

Gy­ro has two part­s: a very sim­ple back­end, im­ple­ment­ed us­ing San­ic [2] which does a few things:

• Serve stat­ic files out of _stat­ic/

• Serve tem­plat­ed mark­down out of pages/

• Save mark­down to pages/

• Keep an in­dex of file con­tents up­dat­ed in _stat­ic/in­dex.js

The oth­er part is a web­page, im­plem­nt­ed us­ing Boot­strap [3] and JQuery [4]. That page can:

• Show mark­­down, us­ing Show­­down [5]

• Ed­it mark­­down, us­ing Sim­­pleMDE [6]

• Search in your pages us­ing Lunr [7]

And that's it. Open the site on any URL that doesn't start with _static and contains only letters and numbers:

• http://lo­­cal­host:8000/My­­Page : GOOD

• http://lo­­cal­host:8000/My­Dir/My­­Page: BAD

• http://lo­­cal­host:8000/__­­foo­bar__: BAD

At first the page will be sort of empty, but if you edit it and save it, it won't be empty anymore. You can link to other pages (even ones you have not created) using the standard markdown syntax: [go to Foo­Bar](­Foo­Bar)

There is re­al­ly not much else to say about it, if you try it and find bugs, file an is­sue and as usu­al patch­es are wel­come.

---