Archive for the 'planet python' Category

Implementation of Dijkstra’s Single-Source Shortest-Paths in JavaScript

I’m working on a project where the client wants a cool sliding navigation effect. We’re implementing this with JavaScript/AJAX/DHTML.

One of the constraints is that pages can only be reached via certain other pages. For example, if you’re on the /portland/contact page and want to go to the /seattle/contact page, you’ll first slide up to /portland, then over to /seattle, then finally down to /seattle/contact.

After a while, it occurred to me that there were some similarities with another project I’ve been working on off and on for the last few years, byCycle.org, which is a bicycle trip planner ala Google Maps.

I had written a Python version of Dijkstra’s Single-Source Shortest-Paths (SSSP) for byCycle.org. That’s available on PyPi as Dijkstar (so named because it also does has the potential to do A*). I figured it wouldn’t be too hard to port the Python version to JavaScript, and it wasn’t.

There were a few snags, though. Most of it was just syntactic and semantic differences between the two languages. The biggest issue was that I use “heapq“ in the Python version to maintain the costs to previously visited nodes in sorted order. JavaScript has no priority queue implementation that I could find, so I came up with a different solution that involves updating an Object (AKA “dict“) with costs to newly visited nodes and sorting the keys to pick the next node to visit. I’m assuming/hoping the underlying sort implementation is highly optimized.

Interestingly, I think I found at least one bug in the Python version, although I’ve been using that version for a couple years now with no known problems, so it must only be applicable in certain edge (no pun intended) cases (or maybe it’s due to some difference in the languages–need to take a closer look). I think the JS version came out cleaner, too.

If anyone’s interested, I’m releasing this under an MIT license. For now, you can get it from here. Note that it depends on the util module that you can get from here. The util module contains some other Python-inspired JavaScript, in particular a couple of functions for generating namespaces and classes. I might write another post about that at some point.

Decluttering the Desk

Today I read a post about decluttering your desk. Right now, I can’t find that post, but this page links to a bunch of similar and related articles. Those links are on the Zen Habits site, which I’ve started reading lately. Lots of interesting food for thought there.

So, I’m pretty tidy already, and my desk wasn’t that much of mess to begin with, but there were still quite a few unnecessary items accumulated there, like a printer I’m never going to use, a stapler I use maybe once a month, etc. I only spent about ten minutes decluttering, but the benefits were huge. So huge that I had to write it again in bold AND italics.

Along with office type stuff, I cleared off some knick knacks–little bits of art and what have you.

Did I mention the huge effect all this had? Wow. There’s a sense of openness and space. There’s more room to work and breathe. There’s also less distracting visual clutter.

This is only an anecdote, but I was more productive today than I’ve been in a while.

I recommend giving this a try. What do you really need on your desk and in your immediate working environment? What I’ve found, at least for myself, is that it’s actually not much. I’m guessing everyone’s got at least a couple items that can be tossed or sold or given away. Things that only get used once in a while can be put out of sight in an easy to reach place. Things that are visually interesting (photos, say) eventually become visually distracting, especially when there’s too much of it.

Of course, there’s no precise recipe, and everyone’s got their personal preferences. For me, just thinking about this, or anything, consciously is what’s important. The “right” changes naturally precipitate from awareness.

Ruby on Rails… Revisited

Updated with links and a couple typo corrections.

I’ve been working on a fairly big Web site project lately. My partner and I initially decided to use Django to build the site, mainly because I’m a Python “expert” and Django is (apparently) the #1 Python Web framework. We were also lured by the easy admin interface.

After trying to use Django and not really enjoying it, I tried switching to Pylons because I’ve had a good amount of experience with it in the building of byCycle.org. It’s gone through two fairly major releases since then, and so have a bunch of the libraries that tend to get used with it, like SQLAlchemy, Elixir, etc.

I was having a hard time with the Pylons docs, and so I ended screwing around with Grok (which actually looks fairly interesting) and even took a look at the Zope 3 site. I’m sure Zope is really awesome or whatever, but it might as well suck. Every time I look at that site, I’m just like “WTF! This shit has been around for like five years!” Anyway, I might just not be smart enough for Zope.

This led us back toward Rails (even if it is a ghetto). I used Rails a bit last year but never did anything too serious with it. Diving into it today was quite a pleasure. There are issues to be sure, but overall I’m enjoying it by far over any of the other options we had tried. I’m also enjoying learning/relearning Ruby.

If Pylons had good docs, we’d probably be using that.

So, I don’t know if this is a particularly useful post, since I didn’t get into much in the way of reasons (what, i have back this up?!). This subject’s been hashed and rehashed, but I just wanted (needed) to make a qualitative statement about my/our experience, which, of course, is purely personal.

Taxes

I heard just yesterday about the so-called “economic stimulus payment.” I figure if I hadn’t heard about it, maybe there are others out there that also haven’t.

The simple version is that anyone who made over $3,000 in 2007, has a valid SSN, and files a federal tax return this year will receive between $300 and $1200 (individual -> married) from Uncle Scam. Sounds like free money to me, as it only takes a few minutes to fill out a 1040EZ.

Of course, it’s actually more complicated than that. See this and this for more info.

WordPress 2.5

I haven’t seen any posts about WordPress 2.5 on Planet Python yet, so I thought I’d make a quick mention of it. RC2 came out today. I installed it on a test site, and it’s looking pretty sweet. Definitely worth a look. The admin interface is much improved–easier to navigate, reduced clutter, and a new, lighter feel. Also, amongst other things, photo galleries are now built in.

Anyway, I just wanted to mention that I think it’s worth a look. For details, check out these posts over on the WordPress blog:

http://wordpress.org/development/2008/03/25-sneak-peek/
http://wordpress.org/development/2008/03/wordpress-25-rc2/

I Need Glasses

I’m finally sure I need glasses. I’ve probably needed them for a while, but I definitely do now. I’ve been finding lately that when I work at the computer for any length of time, I start to get a low level headache, my eyes hurt, I have a hard time concentrating, my productivity drops, and I sometimes get irritable. Sometimes when I leave the computer, my vision is a bit blurry. In the morning, I can barely read the clock from about eight feet away.

This has been going on for a while now, but it hasn’t been so pronounced, and I hadn’t realized exactly what the problem was. Lately, things that were once clear aren’t so clear any more.

I guess one solution would be to stop staring at computer screens as much, but so much of my work (all really) requires me to. The realistic solution is a combination of getting glasses, taking more breaks, and maybe doing some eye exercises. Getting out into nature should help too, as there the eye isn’t focussed at a fixed distance for long periods of time.

Before it became so obvious that I need glasses,  I would sometimes wonder what my problem was. It’s interesting how low intensity health issues can have such an effect on our general feeling of wellbeing. That’s one reason it’s important to stay healthy. It’s also a reason to keep an eye on things and get things checked out early–before they turn into major issues.

The one thing that really worries me about all this is picking out the frames. That could take forever.

Erlang Linked List Exercise

Yesterday, my copy of Programming Erlang arrived in the mail1. w00t! I’m already part way through chapter three. I don’t know what it is about this language – maybe it’s all hype and a passing fad – but I haven’t been this interested in learning a new language since I started with Python over two and half years ago.

The day before yesterday, I took a shot at implementing a linked list in Erlang. I had one basic rule, which was that I wasn’t allowed to use the built in list type. Getting started was fairly difficult, but once I started to “get it” (e.g., pattern matching, recursion), the task got much easier.

Previously, I had only played around in the shell, so this is the first Erlang module I’ve written. Erlang modules are similar to Python modules, though to make functions available outside an Erlang module, they must be explicitly exported. I haven’t yet come across whether Erlang modules can be organized into packages, although I imagine there must be some kind of higher level system for organizing Erlang programs.

Recently, I did a similar exercise with Python as way to experiment with Behavior Driven Development. That version uses the familiar “destructive assignment” operation throughout. Erlang allows single assignment only, so I had to think about the problem in a different way. For example, an item can’t be appended to a list by manipulating a couple of object references as in Python—instead I used recursion to build up a new list.

After I get further into the book, it will be fun to come back to this implementation and see how it can be improved given a better understanding of the language. I’m also looking forward to exploring Concurrency Oriented Programming in depth. With Python, I tend to not think in terms of concurrency, though I’m sure I’ve got code that could be improved by using it.

Finally, here’s the code. It was written in Emacs, which has a nice Erlang mode. There are two other IDEs available, one based on Eclipse and the other on NetBeans.

-module(linkedlist).
-export([new/0,
     head/1,
     tail/1,
     append/2,
     nth/2,
     last/1,
     length/1,
     is_empty/1,
     main/0
    ]).

-record(list, {head, length=0}).
-record(item, {data, next}).

new() ->
    #list{}.

new_item(Data) ->
    #item{data=Data}.

head(List) ->

    List#list.head.

tail(List) when List == #list{} ->
    undefined;
tail(List) ->
    Length = linkedlist:length(List) - 1,
    #list{head=next(head(List)), length=Length}.

append(Data, List) when List == #list{} ->
    List#list{head=new_item(Data), length=1};
append(Data, List) ->
    Item = append_item(Data, head(List)),
    NewLength = linkedlist:length(List) + 1,
    List#list{head=Item, length=NewLength}.

append_item(Data, Item) when Item#item.next == undefined ->
    Item#item{next=new_item(Data)};
append_item(Data, Item) ->

    Item#item{next=f(Data, next(Item))}.

next(Item) ->
    Item#item.next.

% Get the Nth item from List
% N: Index of item to get
% List: List to get from
nth(N, List) when N < 1 ->
    undefined;
nth(N, List) ->
    nth(N, 1, head(List)).

% N: Index of item to get
% I: Current index
% Item: #item in List corresponding to index I
nth(N, I, Item) when I == N ->
    Item;
nth(N, I, Item) ->

    nth(N, I + 1, next(Item)).

last(List) ->
    nth(linkedlist:length(List), List).

length(List) ->
    List#list.length.

is_empty(List) ->
    List == new().

p() ->
    io:format("\n").
p(Object) ->
    erlang:display(Object).

main() ->

    L = new(),
    test_list(L, "New list"),
    L1 = append(data1, L),
    test_list(L1, "List with one item"),
    L2 = append(data2, L1),
    test_list(L2, "List with two items"),
    L3 = append(data3, L2),
    test_list(L3, "List with three items"),
    ok.

test_list(List, Description) ->
    io:format("~s~n", [Description]),
    Length = linkedlist:length(List),
    p({'list', List}),
    p({'head', head(List)}),
    p({'tail', tail(List)}),
    p({'first', nth(1, List)}),
    p({'nth', nth(Length, List)}),
    p({'last', last(List)}),
    p({'length', Length}),
    p({'is_empty', is_empty(List)}),
    p(),
    ok.

1 A new blade and O-ring for my old-fashioned Oster blender came also. Double w00t!

Erlang for Python Programmers?

I’ve been hearing about Erlang for several months now. It has seemed interesting, but up until recently I hadn’t felt any great need to look into it.

Last week, however, my interest was piqued, and I spent much of the weekend reading about Erlang and playing around in its interactive shell. That continued into this week when I decided to buy the new book.

While I’ve been sitting around with “ants in my pants” waiting for the book to show up, I’ve been wondering if there’s an Erlang for Python Programmers tutorial floating around anywhere (I haven’t come across one yet). I’m thinking I might try to put something like that together. I think that will be an interesting challenge as there’s not necessarily an obvious one-to-one correspondence between the two languages.

Here’s how I initially learned about and got interested in Erlang:

These things made it easier to get beyond the yeah-that-looks-neat stage and into the ok-what-the-hell-I’ll-give-it-a-try stage:

  • It was easy to install with apt-get on Ubuntu (`sudo apt-get install erlang`)
  • It has an interactive shell (`erl`)
  • There’s good introductory documentation at erlang.org

Here’s what’s keeping me interested:

  • It’s a functional language, which makes it more interesting in some ways than learning, say, Ruby—in fact, I wouldn’t really need to learn much to start using Ruby, whereas using Erlang will require new ways of thinking about program structure (in terms of functions and processes)
  • I like the term “Concurrency Oriented Programming”
  • It’s been used to build some massive real-world systems
  • There’s a Web framework
  • It has a planet: Planet Erlang
  • There’s a recent book by one of the original creators of the language
  • It seems to be hot amongst alpha geeks, and I’m a wannabe alpha geek

See erlang.org for pointers to more info.

Erlang is the first functional language I’ve really gotten into (for whatever reason), so regardless of whether I ever end up using it in the real world, the learning experience will be valuable.

Fun With Python BDD

I was testing my algorithm/data structures chops by implementing a Linked List from scratch using no reference material (not very difficult, I know), and I must say that using BDD was incredibly helpful. I’m calling it “BDD” instead of “TDD” just because the test class names describe a context and the method names in the test classes are behavior-oriented, inspired by the style found on the RSpec site.

I didn’t write the tests/specs up front like you’re “supposed” to. Instead, I thought about what I was trying to do, came up with an initial idea for the implementation, started writing code, and then added a test to make sure I was on the right track. This worked out pretty well. I also used the tests during refactoring. Writing the tests first might have worked out better, but I can’t say for sure.

Now, for some real fun, implement a linked list based on only the test code below.

class LinkedList(object):
    def __init__(self, *values):
        self._size = 0
        if values:
            for v in values:
                self.add(v)
        else:
            self.head = self.tail = None

    def is_empty(self):
        return len(self) == 0

    def add(self, val):
        node = Node(val)
        if self.is_empty():
            self.head = node
        else:
            self.tail.next = node
        self.tail = node
        self._size += 1

    def get(self, index, get_prev=False):
        if index < 0:
            index = self._size + index
        if index < 0 or index >= self.length:
            raise IndexError
        prev = None
        for i, node in enumerate(self):
            if i == index:
                if get_prev:
                    return node, prev
                else:
                    return node
            prev = node

    def remove(self, index):
        node, prev = self.get(index, get_prev=True)
        if node is self.head:
            self.head = node.next
        else:
            prev.next = node.next
        self._size -= 1
        return node

    def pop(self):
        return self.remove(self.length - 1)

    def __len__(self):
        return self._size
    length = property(__len__)
    size = property(__len__)

    def __iter__(self):
        curr = self.head
        while curr is not None:
            yield curr
            curr = curr.next
        raise StopIteration

    def __str__(self):
        return ', '.join([str(node.value) for node in self])

class Node(object):
    def __init__(self, val, next=None):
        self.value = val
        self.next = next
import unittest
class Test_A_New_Linked_List(unittest.TestCase):

    def test_given_no_values_should_be_empty(self):
        list_ = LinkedList()
        assert list_.is_empty()

    def test_given_no_values_should_have_size_0(self):
        list_ = LinkedList()
        assert list_.size == list_.length == 0

    def test_given_values_should_not_be_empty(self):
        list_ = LinkedList(1, 2, 3)
        assert not list_.is_empty()
        assert [node.value for node in list_] == [1, 2, 3]

    def test_given_4_values_should_have_size_4(self):
        list_ = LinkedList(1, '2', 3, '4th value')
        assert list_.size == 4

class Test_An_Empty_List(unittest.TestCase):

    def test_should_have_size_1_after_add(self):
        list_ = LinkedList()
        val = 15
        list_.add(val)
        assert list_.head.value == list_.tail.value == val
        assert len(list_) == list_.length == list_.size == 1

    def test_should_raise_an_index_error_on_get(self):
        list_ = LinkedList()
        self.assertRaises(IndexError, list_.get, 1)
        list_ = LinkedList(1, 2, 3)
        list_.remove(0); list_.remove(0); list_.remove(0)
        self.assertRaises(IndexError, list_.get, 0)

    def test_should_raise_an_index_error_on_remove(self):
        list_ = LinkedList()
        self.assertRaises(IndexError, list_.remove, 1)
        list_ = LinkedList(1, 2, 3)
        list_.remove(0); list_.remove(0); list_.remove(0)
        self.assertRaises(IndexError, list_.remove, 0)

    def test_should_raise_an_index_error_on_pop(self):
        list_ = LinkedList()
        self.assertRaises(IndexError, list_.pop)

class Test_A_Non_Empty_List(unittest.TestCase):

    def test_should_increase_its_size_by_1_on_add(self):
        list_ = LinkedList(1, 2, 4, 5)
        starting_size = list_.size
        list_.add(13)
        assert len(list_) == list_.length == list_.size == (starting_size + 1)

    def test_should_return_a_node_on_get(self):
        list_ = LinkedList(1, 2, '3rd value')
        node = list_.get(0)
        assert isinstance(node, Node)
        assert node.value == 1
        node = list_.get(1)
        assert isinstance(node, Node)
        assert node.value == 2
        node = list_.get(2)
        assert isinstance(node, Node)
        assert node.value == '3rd value'

    def test_should_decrease_its_size_by_1_on_remove(self):
        list_ = LinkedList(1, 2, 4, 5)
        starting_size = list_.size
        list_.remove(0)
        assert len(list_) == list_.length == list_.size == (starting_size - 1)

    def test_should_decrease_its_size_by_1_on_pop(self):
        list_ = LinkedList(1, 2, 4, 5)
        starting_size = list_.size
        list_.pop()
        assert len(list_) == list_.length == list_.size == (starting_size - 1)

Google Maps Encoded Polylines

Update 6/30/07: Fixed links to glineenc.py so that it’s actually accessible.
Update 7/15/08: Fixed links to glineenc.py again because of change to HTTPS on Trac site.

Here’s some Python code (complete with unit and doc tests) for converting a series of latitude/longitude points (i.e., a polyline) to the Base64 encoding that Google Maps understands. It’s particularly useful for long and/or complicated lines.

It’s based on the algorithm listed here and the JavaScript code here [page disappeared].

This site gives some more insight into it and has a pretty cool example of a fractal line here.

Here’s the code:

Previously, I had pasted the Python code right into this article, but I recently made a bunch of revisions and it was way too long. Here’s a link to the code instead:

glineenc.py on Trac

Please note that this code is still in somewhat of a rough state. I have plans to polish and package it up, but for now, I’m using it as is and it’s working quite well (you’ll have to be patient to click that link as it takes ~20-30 seconds to generate the route, even though the line drawing itself is almost instantaneous).

JavaScript that uses results from `glineenc` looks something like this (assuming you’ve returned some JSON, say, with `encoded_points` and `encoded_lines` keys):


map.addOverlay(new GPolyline.fromEncoded({
  color: "#0000ff",
  weight: 4,
  opacity: 0.8,
  points: result.encoded_points,
  levels: result.encoded_levels,
  zoomFactor: 32,
  numLevels: 4
}));

`points` is the encoded lat/long points. `levels` indicates which zoom levels each point should displayed at; there is one character per point. See the links above for a more complete explanation.