All posts by Wyatt

Saddle Mountain

Date: July 11, 2012
Elevation: 3283 feet
Distance: 5.4 miles
Time to top: 50 minutes
Weather: Hot (~90 degrees), sunny, hazy
Gear: Trail running shoes, small Camelbak pack

The trail is very steep pretty much the whole way; it’s shaded for about the first half and then open and rocky to the top. It was really hot, and I got a bit sunburned. The view from the top would probably be pretty spectacular on a non-hazy day.

View from Saddle Mountain

There’s a nice, shady spot to hang out at the end of this other trail (top left of image below), which is just a little lower than the peak. I hung out there for about an hour.

What I Learned from the apache.org Break-in

Here’s my initial take-away after reading about the recent apache.org XSS-initiated break-in:

  • If you’re going to allow caching of credentials (Subversion or otherwise) on a server, don’t use an account that shares credentials with any superuser account. Personally, I can’t think of a good reason for these credentials to be cached in the first place (except on a development machine). As an aside, by default, Mercurial doesn’t do this; I suppose the fact that every ‘svn commit’ is also a push makes this more “necessary” with Subversion.
  • If you have an organization-wide login (say a Windows login that is automatically sync’ed with Subversion, your enterprise RDBMS, and who knows what else[1]), if at all possible use a different  password on any server where you’ve got superuser access.
  • All superuser accounts on servers should have different passwords; at a minimum, if you use a common password for superuser accounts across servers, don’t use this password for other accounts.
  • Use Trac instead of Jira. [2]

That’s a bare minimum; I’m still thinking about how vulnerable the organization I work for might be. Most of this probably seems obvious, but I’m betting that these and other less-than-best practices are extremely common.

Ref: https://blogs.apache.org/infra/entry/apache_org_04_09_2010

[1] This is a purely hypothetical scenario. ;)

[2] Just kidding?

Eclipse for Python Web Development on Ubuntu

Updated 14 July 2009 for Eclipse 3.5. Now using Aptana to install PyDev.

Updated 5 March 2010 for Eclipse 3.5.2 and Subversion 1.6.

Updated 20 July 2010 for Eclipse 3.6

Updated 1 August 2011 for Eclipse 3.7 and Aptana Studio 3

Overview

So, I think I’ve finally decided that I prefer Eclipse for Python Web development over NetBeans. (I prefer Wing IDE over both for straight Python development, but that’s another post.) Eclipse’s Python support, via PyDev, seems more advanced and NetBeans has some annoying issues. Eclipse also seems a bit snappier, at least on my machine (YMMV, blah blah).

Eclipse takes a bit more effort to set up, but once you’ve done it a couple times, it’s pretty straightforward. The hard part is keeping track of the links to the Eclipse update sites and remembering a few odd bits of configuration. This document gives details on installing Eclipse for Python Web development with Aptana, PyDev, and Subclipse.

The instructions here are Linux/Ubuntu-centric, but the instructions for getting to the Platform Binary–the smallest possible Eclipse download, as far as I can tell–are applicable to all platforms.

Download Eclipse

If you go to the Eclipse downloads page, you’ll see packages for Java and C++ along with some other options. If you’re doing ONLY Python development, you might wonder which version to download. These versions install cruft I don’t want or need. I finally found what I think is the smallest possible Eclipse package, the so-called Platform Runtime Library.

Install Eclipse

Extract the downloaded package: tar xvzf eclipse-platform-3.7-linux-gtk.tar.gz. I rename the resulting eclipse directory to eclipse-3.6, move it to ~/.local, and create a symlink in ~/.local/bin to the eclipse executable: ln -s ~/.local/eclipse-3.7/eclipse ~/.local/bin/eclipse

Configure Eclipse

Edit ~/.local/eclipse-3.7/eclipse.ini. Find the line containing “-vmargs”. Add a new line directly below that: -Djava.library.path=/usr/lib/jni. Save and close. Fire up Eclipse.

Install Aptana Studio

Install PyDev

PyDev is installed as part of the Apatana Studio install.

Install MercurialEclipse

Eclipse update URL: http://cbes.javaforge.com/update

Supervisor now supports Python 2.6

I’ve been using Supervisor for a few months now to manage some app servers at work. Good stuff; solid. My only gripe with it was that it didn’t run properly under Python 2.6[1,2]. I had to install Python 2.5 just to run Supervisor, which isn’t a super big deal but is kind of annoying. The latest release, 3.0a7, fixes this by including a “patched version of Medusa to allow Supervisor to run on Python 2.6.”

Note: I had to `wget http://dist.supervisord.org/supervisor-3.0a7.tar.gz` and easy_install that due to a network timeout[3]. I assume that’s because Supervisor is super popular and everyone’s upgrading.

[1] It did run under 2.6 but issued an error when starting up.
[2] This was due to changes in the Python 2.6 stdlib, not a problem with Supervisor itself.
[3] Suggestion: If possible, remove link from PyPI to http://supervisord.org/ so easy_install doesn’t get stuck there.

Implementation of Dijkstra’s Single-Source Shortest-Paths in JavaScript

I’m working on a project where the client wants a cool sliding navigation effect. We’re implementing this with JavaScript/AJAX/DHTML.

One of the constraints is that pages can only be reached via certain other pages. For example, if you’re on the /portland/contact page and want to go to the /seattle/contact page, you’ll first slide up to /portland, then over to /seattle, then finally down to /seattle/contact.

After a while, it occurred to me that there were some similarities with another project I’ve been working on off and on for the last few years, byCycle.org, which is a bicycle trip planner a la Google Maps.

I had written a Python version of Dijkstra’s Single-Source Shortest-Paths (SSSP) for byCycle.org. That’s available on Bitbucket as Dijkstar (so named because it also does has the potential to do A*). I figured it wouldn’t be too hard to port the Python version to JavaScript, and it wasn’t.

There were a few snags, though. Most of it was just syntactic and semantic differences between the two languages. The biggest issue was that I use “heapq“ in the Python version to maintain the costs to previously visited nodes in sorted order. JavaScript has no priority queue implementation that I could find, so I came up with a different solution that involves updating an Object (AKA hash) with costs to newly visited nodes and sorting the keys to pick the next node to visit. I’m assuming/hoping the underlying sort implementation is highly optimized.

Interestingly, I think I found at least one bug in the Python version, although I’ve been using that version for a couple years now with no known problems, so it must only be applicable in certain edge (no pun intended) cases (or maybe it’s due to some difference in the languages–need to take a closer look). I think the JS version came out cleaner, too.

If anyone’s interested, I’m releasing this under an MIT license. You can get it from here. Note that it depends on the util module that you can get from here. The util module contains some other Python-inspired JavaScript, in particular a couple of functions for generating namespaces and classes. I might write another post about that at some point.

YUI TreeView with Ruby on Rails

Here’s some code I’m using to generate a dynamic tree view using an acts_as_tree model with slug and title fields, the TreeView widget from YUI, and a Rails helper. I chopped out some of the code for clarity, so all this does is create a menu with the titles from the model, but the basic idea is there to expand on.

Rails view/JavaScript

<script type="text/javascript">
  var page_tree;
  page_tree_init = function () {
    page_tree = new YAHOO.widget.TreeView('page_tree');
    var root = page_tree.getRoot();
    <% generate_page_nodes(@root) {} %>
    page_tree.draw();
  };
  page_tree_init();
</script>

Ruby

def generate_page_nodes(node, &block)
  parent = node.parent
  node_name = node.slug.gsub('-', '_')
  parent_node_name = parent.nil? ? 'root' : parent.slug.gsub('-', '_')
  js = <<-JS
    var #{node_name} = new YAHOO.widget.MenuNode('<span class="node_title">#{node.title}</span>',
                                                 #{parent_node_name});
  JS
  concat(js, block.binding)
  children = node.children
  children.each { |c| generate_page_nodes(c, &block) } unless children.empty?
end

Ruby on Rails… Revisited

Updated with links and a couple typo corrections.

Update: It wasn’t long before the project got too complex on the back end (SOAP blech) for my limited Ruby knowledge. I switched it back to Python/Pylons and never looked back. The Pylons => Rails migration was straightforward. I guess I could have pushed through with Ruby/Rails, but with deadlines looming, it made more sense for me to go with what I knew best. Being familiar Python and its ecosystem was far more pertinent than the deficiency of any particular library. There’s probably another blog post or two in here…

I’ve been working on a fairly big Web site project lately. My partner and I initially decided to use Django to build the site, mainly because I’m a Python “expert” and Django is (apparently) the #1 Python Web framework. We were also lured by the easy admin interface.

After trying to use Django and not really enjoying it, I tried switching to Pylons because I’ve had a good amount of experience with it in the building of byCycle.org. It’s gone through two fairly major releases since then, and so have a bunch of the libraries that tend to get used with it, like SQLAlchemy, Elixir, etc.

I was having a hard time with the Pylons docs, and so I ended screwing around with Grok (which actually looks fairly interesting) and even took a look at the Zope 3 site. I’m sure Zope is really awesome or whatever, but it might as well suck. Every time I look at that site, I’m just like “WTF! This shit has been around for like five years!” Anyway, I might just not be smart enough for Zope.

This led us back toward Rails (even if it is a ghetto). I used Rails a bit last year but never did anything too serious with it. Diving into it today was quite a pleasure. There are issues to be sure, but overall I’m enjoying it by far over any of the other options we had tried. I’m also enjoying learning/relearning Ruby.

If Pylons had good docs, we’d probably be using that.

So, I don’t know if this is a particularly useful post, since I didn’t get into much in the way of reasons (what, i have back this up?!). This subject’s been hashed and rehashed, but I just wanted (needed) to make a qualitative statement about my/our experience, which, of course, is purely personal.

Erlang Linked List Exercise

Yesterday, my copy of Programming Erlang arrived in the mail1. w00t! I’m already part way through chapter three. I don’t know what it is about this language – maybe it’s all hype and a passing fad – but I haven’t been this interested in learning a new language since I started with Python over two and half years ago.

The day before yesterday, I took a shot at implementing a linked list in Erlang. I had one basic rule, which was that I wasn’t allowed to use the built in list type. Getting started was fairly difficult, but once I started to “get it” (e.g., pattern matching, recursion), the task got much easier.

Previously, I had only played around in the shell, so this is the first Erlang module I’ve written. Erlang modules are similar to Python modules, though to make functions available outside an Erlang module, they must be explicitly exported. I haven’t yet come across whether Erlang modules can be organized into packages, although I imagine there must be some kind of higher level system for organizing Erlang programs.

Recently, I did a similar exercise with Python as way to experiment with Behavior Driven Development. That version uses the familiar “destructive assignment” operation throughout. Erlang allows single assignment only, so I had to think about the problem in a different way. For example, an item can’t be appended to a list by manipulating a couple of object references as in Python—instead I used recursion to build up a new list.

After I get further into the book, it will be fun to come back to this implementation and see how it can be improved given a better understanding of the language. I’m also looking forward to exploring Concurrency Oriented Programming in depth. With Python, I tend to not think in terms of concurrency, though I’m sure I’ve got code that could be improved by using it.

Finally, here’s the code. It was written in Emacs, which has a nice Erlang mode. There are two other IDEs available, one based on Eclipse and the other on NetBeans.

-module(linkedlist).
-export([new/0,
     head/1,
     tail/1,
     append/2,
     nth/2,
     last/1,
     length/1,
     is_empty/1,
     main/0
    ]).

-record(list, {head, length=0}).
-record(item, {data, next}).

new() ->
    #list{}.

new_item(Data) ->
    #item{data=Data}.

head(List) ->

    List#list.head.

tail(List) when List == #list{} ->
    undefined;
tail(List) ->
    Length = linkedlist:length(List) - 1,
    #list{head=next(head(List)), length=Length}.

append(Data, List) when List == #list{} ->
    List#list{head=new_item(Data), length=1};
append(Data, List) ->
    Item = append_item(Data, head(List)),
    NewLength = linkedlist:length(List) + 1,
    List#list{head=Item, length=NewLength}.

append_item(Data, Item) when Item#item.next == undefined ->
    Item#item{next=new_item(Data)};
append_item(Data, Item) ->

    Item#item{next=f(Data, next(Item))}.

next(Item) ->
    Item#item.next.

% Get the Nth item from List
% N: Index of item to get
% List: List to get from
nth(N, List) when N < 1 ->
    undefined;
nth(N, List) ->
    nth(N, 1, head(List)).

% N: Index of item to get
% I: Current index
% Item: #item in List corresponding to index I
nth(N, I, Item) when I == N ->
    Item;
nth(N, I, Item) ->

    nth(N, I + 1, next(Item)).

last(List) ->
    nth(linkedlist:length(List), List).

length(List) ->
    List#list.length.

is_empty(List) ->
    List == new().

p() ->
    io:format("\n").
p(Object) ->
    erlang:display(Object).

main() ->

    L = new(),
    test_list(L, "New list"),
    L1 = append(data1, L),
    test_list(L1, "List with one item"),
    L2 = append(data2, L1),
    test_list(L2, "List with two items"),
    L3 = append(data3, L2),
    test_list(L3, "List with three items"),
    ok.

test_list(List, Description) ->
    io:format("~s~n", [Description]),
    Length = linkedlist:length(List),
    p({'list', List}),
    p({'head', head(List)}),
    p({'tail', tail(List)}),
    p({'first', nth(1, List)}),
    p({'nth', nth(Length, List)}),
    p({'last', last(List)}),
    p({'length', Length}),
    p({'is_empty', is_empty(List)}),
    p(),
    ok.

1 A new blade and O-ring for my old-fashioned Oster blender came also. Double w00t!

Erlang for Python Programmers?

I’ve been hearing about Erlang for several months now. It has seemed interesting, but up until recently I hadn’t felt any great need to look into it.

Last week, however, my interest was piqued, and I spent much of the weekend reading about Erlang and playing around in its interactive shell. That continued into this week when I decided to buy the new book.

While I’ve been sitting around waiting for the book to show up, I’ve been wondering if there’s an Erlang for Python Programmers tutorial floating around anywhere (I haven’t come across one yet). I’m thinking I might try to put something like that together. I think that will be an interesting challenge as there’s not necessarily an obvious one-to-one correspondence between the two languages.

Here’s how I initially learned about and got interested in Erlang:

These things made it easier to get beyond the yeah-that-looks-neat stage and into the ok-what-the-hell-I’ll-give-it-a-try stage:

  • It was easy to install with apt-get on Ubuntu (`sudo apt-get install erlang`)
  • It has an interactive shell (`erl`)
  • There’s good introductory documentation at erlang.org

Here’s what’s keeping me interested:

  • It’s a functional language, which makes it more interesting in some ways than learning, say, Ruby—in fact, I wouldn’t really need to learn much to start using Ruby, whereas using Erlang will require new ways of thinking about program structure (in terms of functions and processes)
  • I like the term “Concurrency Oriented Programming”
  • It’s been used to build some massive real-world systems
  • There’s a Web framework [link died]
  • It has a planet Planet Erlang [link died]
  • There’s a recent book by one of the original creators of the language
  • It seems to be hot amongst alpha geeks, and I’m a wannabe alpha geek

See erlang.org for pointers to more info.

Erlang is the first functional language I’ve really gotten into (for whatever reason), so regardless of whether I ever end up using it in the real world, the learning experience will be valuable.