Configure Eclipse with Python and Django on OS X

I've recently been doing some development using the Python web application framework Django. To make my life easier, I configured Eclipse to work with Python and Django however I soon discovered the many disparate resources on the web are either dated or incomplete. Thus, the main goal of this post is to consolidate those resources into one comprehensive and easy to follow post that I'll try to keep as succinct as possible while maintaining clear instruction. Also, I'll assume that you already have Eclipse installed and if not, go get it - what are you waiting for? I'm using Eclipse 3.5.1 but I believe these instructions will work for 3.4.x as well with some slight modifications.

Also, these instructions assume the following:

  • For simplicity's sake we're going to use SQLite for Django database - Python 2.5 and above come with SQLite support so no install necessary.

  • No previous versions of Pydev modules or Django on your system.

  • We'll be using the built-in versions of Python inherent in Leopard (python version - 2.5.1) or Snow Leopard(python version - 2.6)



Installing Python Support - Pydev plugin


The Pydev plugin from aptana gives you syntax highlighting, code complete, etc for Python in the Eclipse environment and in my experience work exceptionally well.

Open Eclipse and perform the following:

1) Help->Install New Software. In the "Work With" form section, input the following site to download the Pydev modules: http://pydev.org/updates.

2) Select the Pydev plugin modules in the checkbox.


3) Click through, accepting the licensing agreements and wait for the module to install. Restart Eclipse. Open Eclipse->About and click on the Pydev Icon - we want to get the latest version of Pydev which is version 1.5.0.125.1989166 at the time of this post. If you have an earlier version use the automatic update feature of Eclipse to update just the Pydev module (Help->Update Software)

Install Django


4) Download Django 1.1.1, open a Terminal, navigate to the directory where you downloaded Django, and run the following commands:


tar -xvf Django-1.1.1.tar.gz
cd Django-1.1.1
sudo python setup.py install


Yep, it's as easy as that.

Configure PyDev to use Django


4) After installing Django, we need to ensure that the newly installed directories are visible to the Eclipse python interpreter. In the toolbar under Eclipse->Preferences->Pydev, click the Interpreter-Python option and the following window will appear:




5) Click 'New' and populate Interpreter Name with any value you want (I called it 'OS X Python'), and populate the Interpreter Executable with the path to your python executable - the default path on OS X is /usr/bin/python.



6) Eclipse will attempt to locate the libraries for Python and will display a pre-populated list of directories to add to the Eclipse PythonPath - i.e., the modules and binaries necessary for Python to run. Critical here is to select the directory where Django files were extracted when we installed it earlier. So make sure to select '/Library/Python/2.x/site-packages', as in the screenshot below. After you click 'OK' you'll be taken back to the preferences screen. Click 'OK' again and Eclipse will reparse the site packages.



Sanity Check


7) As a quick sanity check, create a new PyDev project and then a new Python file within that project space. Test a django import statement to see if code completion is working. It should look something like the screenshot below:




Create a New Django Project


8) Open Terminal navigate to a directory where you want to store your first Django project. Once there create a new directory for your project, navigate into the folder and enter the following command:

django-admin.py startproject PROJECTNAME
It's important to create the top level directory and place the new project files into a separate directory, as we'll see in a few minutes. If you navigate into the directory you created you'll see that Django has created the prepopulated files __init__.py, manage.py, settings.py, and urls.py within that directory. Now we have to import these into Eclipse.

9) In Eclipse, click 'File->New->Pydev Project' Name the project then uncheck 'Use Default' since we're using code that's already created for us in the previous step. Instead, click 'Browse' and navigate to the location of the top level directory you created previously. Uncheck 'Create default src folder' and click 'Finish'.



In my case, I created a top level directory named Djangoprojects and created the Django files with the django-admin.py command inside a directory named Django.

10) Within Eclipse, right-click the project you've created and click 'Properties->Pydev-PYTHONPATH->Add Source Folder' and select the project folder you created.



Now in the package explorer for your newly created project you'll see that your __init__.py, manage.py, settings.py, and urls.py are showing as a package.

Set up the Django Testserver



11) Now we have to configure Eclipse to start the the Django test server when we run our program. Right-click your new project in the Package explorer-> Properties->Debug Configurations(or Run/Debug Settings in newer versions of Eclipse). Click 'New' and select 'Python Run'. Press the new configuration button at the top, then give the new configuration a name (I call mine DjangoTest), under Project click 'Browse' and point to our new project, and under Main Module point to the manage.py of our new project. It should look as follows:



Under the Arguments tab, for Program Arguments, type in 'runserver'. Refer to the screen below:



Now right-click your project, then 'Debug As->Python Run' then browse to the manage.py file within your project to launch, and click 'Ok', if all went well you'll get info in the console indicating that the pydev debugger is running:



Point your browser to http://localhost:8000/ and you should see the following page:



Unfortunately, you'll likely find that you're unable to shut down the server from Eclipse using the square red button near the console. Also, it won't shut down when you close Eclipse so you'll have to get in the habit of shutting down the Python process using Activity Monitor.

Debugging with Django and Python


12) Nearing the end of the process now. We have to point Eclipse to the working directory within our workspace for it to be able to access the database (SQLite in our case). So we go back to the Python perspective and right click our project->Debug As-> Debug Configurations(or Run/Debug Settings in newer versions of Eclipse)-> Arguments Tab. Under 'Working Directory' we click 'Other' then 'Workspace' and browse to our inner folder (In my case the Django folder within Djangotest). Bear in mind that if your code resides in a directory outside the Eclipse workspace you'd have to use the 'File System' button and point to the appropriate directory in your filesystem. Refer below:



The final debug configuration should appear as the one below:



13) Lastly we need to configure Eclipse to have the python debug folder 'pysrc' in the PYTHONPATH variable. Select Eclipse in the menu toolbar -> Preferences ->Pydev->Interpreter-Python->New Folder. Browse to the plugins folder of your Eclipse install, in my case located on the path '/Applications/eclipse/plugins/org.python.pydev.debug_1.5.0.1251989166/pysrc. Click apply and Eclipse will again reconfigure the PYTHONPATH to include the debug extensions. Be careful here if you updated your pydev modules as described there will be a debug folder for the previous version and you want to make sure you're using the latest and greatest. The config should look like the screenshot below:



In order to simplify starting and stopping the debug server, right-click the toolbar Eclipse toolbar (within the Eclipse window) where there aren't buttons and select 'Customize Perspective'. Click the 'Command Groups Availability' and check the Pydev Debug switch. This will place the start and stop debug server options directly within the Eclipse toolbar. It should like like this:



Now, when adding breakpoints to your code you'll be able to debug in Eclipse as you're used to with Java. One note: in order to see the debugging perspective within Pydev you'll have to close the Pydev perspective and reopen it so it can reload the modules we've added - otherwise you'll never see the debug option. In order to test this place a breakpoint in your python code and run the debugger. The python debug perspective will subsequently open and you'll be able to step through your code, see running values of variables, etc.

14) Celebrate. You've completed installation of Python and Django support in Eclipse and this was no easy task. I'd suggest periodically looking for updates to the Pydev extensions as they are regularly improved by the good folks at aptana. The following is a list of several resources I used to compile this post in the event that you need additional help:

Blackberry Code Signing - Setting up Eclipse to use Keys

During the course of development on the Blackberry platform you're bound to call functions from RIM's set of restricted APIs. The APIs with controlled access tend to be the most interesting as they touch core OS modules such as messaging, invoking the browser, access to mail, adding menu items, etc. The point of controlling access via code signing keys is you'll be interfacing with system critical functionality - RIM needs to be able to map applications to specific developers in the event your code is wreaking havok with the OS or is performing something unethical. Although the keys cost $20 US per set you should count yourself lucky to be developing on a platform where you can run your application as a background process unlike Apple app development where you're running in a sandbox. This means that you'll be able to freely add to the functionality of the phone, permitting you to extend and improve RIM' built-in features.




Restricted Blackberry APIs


When you're testing on the simulator Eclipse will give warnings that you're calling these restricted APIs but will still permit you to invoke the simulator. Ideally, you'll want to install your application on an actual phone and this is where obtaining the code signing keys come into play. Below is an overview of the Blackberry packages that require code signatures:

  • net.rim.blackberry.api.browser - gain access to the Blackberry Browser application through getDefaultSession() and getSession() functions

  • net.rim.blackberry.api.invoke - provides functionality for your application to access and invoke internal applications such as Camera, Calendar, Calculator, Maps, etc.

  • net.rim.blackberry.api.mail - interface to mail features including sending, receiving, and accessing email.

  • net.rim.blackberry.api.mail.event - interface to listener functionality for events, mail, folder changes, services, etc.

  • net.rim.blackberry.api.menuitem - permits adding and registering custom menu items to application.

  • net.rim.blackberry.api.options - add system-wide option menus for multiple applications to access.

  • net.rim.blackberry.api.pdap - if your application must conform to the Personal Digital Assistant Profile(PDAP) speification you must implement this interface.

  • net.rim.blackberry.api.phone - as the name suggests, the PhoneListener interface along with the Phone and PhoneCall classes permits access to the Blackberry Phone application to send and receive calls along with changing phone options.

  • net.rim.blackberry.api.phone.phonelogs - interface for attaching listeners to phone log events.

  • net.rim.device.api.browser.field - interface to BrowserContent (URL rendering) and events to open, close, and monitor browser fields.

  • net.rim.device.api.browser.plugin - API to add support for custom MIME types to the RIM browser.

  • net.rim.device.api.crypto - API for encryption and decryption of data, digital signing, authentication of data.

  • net.rim.device.api.io.http - provides API for request methods, header requests, and push input streams for use with TCP/IP connections.

  • net.rim.device.api.notification - API for indicating events to the user via tone, vibration, or LED as well as dialogs or interaction screens on the RIM device.

  • net.rim.device.api.servicebook - provides access to the device service book containing information that configures how the phone will interact with the enterprise infrastructure and enables specific services for the device.

  • net.rim.device.api.synchronization - interface to backup/restore both wired and over the air.

  • net.rim.device.api.system - not all system classes/interfaces require code signing keys however those that do include: interface for receiving audio events, application management(ApplicationManager), access to the backlight(Backlight), requests for the Device to power down, go into storage mode, set time and date(Device), API to access the device display properties (Display), access to store application object data on the phone (PersistentStore), and more.



Apply For Signature Keys and Install


The first step in the process is to complete the web form application including providing your credit card details and a personal 10 digit PIN number that you create. Think of it in the same way as the PIN number for your ATM card - it protects you from someone else acquiring your key and using it to sign their application. Once you submit the form, within 2 weeks you'll receive three emails from Blackberry, each containing a different client key - RBB, RRT, and RCR.

The instructions that come with the keys are comprehensive and I didn't run into trouble but I've included the Eclipse version below for reference. It is a two-step process for making a signed application. First you have to register the keys with RIM using the following process:

BlackBerry JDE Plug-in for Eclipse Users:

1) Save all 3 .csi files in the same directory (each one will be sent in a separate email message).
2) Start Eclipse.
3) Click on the BlackBerry menu and choose Install Signature Keys.
4) Select one of the 3 .csi files saved in step 1 and click Open.
5) Click "Yes" to create a new key pair file.
6) Type a password for your private key of at least 8 characters, and type it again to confirm. This is your private key password, which protects your private key. Please remember this password as you will be prompted for it each time signing is attempted or a signature key is installed.
7) Move your mouse to generate date for a new private key.
8) In the Registration PIN" field, type the PIN number that you supplied on the signature key order form.
9) In the Private Key password field, type the password created in step 6.
10) Click Register .
11) Click Exit .
12) Repeat this process for the other csi files.

One undocumented aspect of this part of the setup is that you'll have to copy the files 'sigtool.db' and 'sigtool.csk' from your main Research in Motion folder to your Eclipse plugin folder. On my system (Windows 7 64-bit) the originals were located at C:\Program Files(x86)\Research In Motion\BlackBerry JDE 4.5.0\bin and the path to the Eclipse plugin was C:\Program Files(x86)\Eclipse\plugins\net.rim.eide.componentpack4.5.0_4.5.0.164.5.0\components\bin. So long as these files are in the correct place you'll have the code signing keys installed and all warnings on use of restricted APIs within Eclipse will be eliminated within the workspace.

One Set of Keys on Multiple Machines?


Once you install the keys on your box you cannot move them to another machine without going through RIM support and requesting new keys. As I understand it, this process invalidates your original keys. The bottom line is that you cannot use one set of keys across multiple machines so chose the box where you install the keys wisely.

Request for Code Signature from Blackberry Signing Authority


The second step is to force your .cod file to be signed by requesting the code signature from the Blackberry Signing Authority Tool. This is done by navigating to the directory where your compiled application resides and double-clicking the cod file you'd like to request signing. If the install of your keys went correctly the signature tool will appear that looks like the one below:



Since the application that I'm requesting signing for is large, you'll notice that the cod file is broken into many separate parts and each must be signed. When you click request, a popup will appear asking for your private key that you set up during the first step.



Enter your password and the request will be sent to RIM's signing authority. Keep your eye on the status column of the signature tool to see if you successfully requested the signing. If you were successful, the status will show "Signed" as below:




That's all there is to it. Now you can directly provision your application and test directly on the actual hardware. An extremely valuable reference to code signing can be found here.

Set $PATH Environment Variable in OS X - Snow Leopard

Slightly off Blackberry topic but recently I performed a clean install of Snow Leopard on my Mac Mini and the other day found myself needing to modify the $PATH variable to reconfigure some of my favorite commands via Terminal. The last time I'd done this was many years ago but I finally got the syntax down correctly and thought it could help some of you out there including those on Leopard and even Tiger - if you're using the Bash shell.

What we need to do since it's a fresh Snow Leopard install is create a .bash_profile file in our home directory with the additional paths we'd like to add. If you're not working with a new install you can check if there's a .bash_profile file that already exists by navigating to your home directory and using the 'ls -a' command to display hidden files. If one already exists you would edit this file with the directory paths you would like to add to the environment. Just add additional 'PATH=/path/you/want/to/add/:' statements after the export statement but before the final ':$PATH'.

An alternative to the .bash_profile is to edit the file 'paths' in the /etc directory with the directory paths you'd like to add however this seems less intuitive to me, especially if you have multiple users on your system and don't want them to have access to particular commands from Terminal.

In my case, the .bash_profile file didn't exist in my home directory in Snow Leopard since I'd performed a clean install so opened a new Terminal session(opens in your home directory) and ran the following command:



echo 'export PATH=/path/you/want/to/add/to/env:$PATH' >> ~/.bash_profile


Running this command creates the file '.bash_profile' and adds your new path along with the current directory paths that reside in the $PATH variable. For example, if I wanted to include subversion commands like svn, svnadmin, svndumpfilter, etc. located (in my case) in the /opt/subversion/bin directory, I'd simply issue the following command:


echo 'export PATH=/opt/subversion/bin/:$PATH' >> ~/.bash_profile


You can test if this works by opening up a new terminal window, typing the 'env' command and looking for the newly added directory values in the output.

Blackberry User Interface Design - Build a Title Screen









I know, I know - The video embedded here shows an iPhone application and as Blackberry developers we despise the iPhone (mainly because iPhone developers have it easy developing apps for one platform with a uniform API and screen size. We're jealous) There's a point to watching that video so take a minute. Making professional Blackberry applications isn't necessarily easy given the complexity of the J2ME environment and vastness of the Blackberry API.

Lots of little tricks with the API are necessary in order to make a rich end user experience. I've found that devoting the time to designing the user interface up front is time well spent. In fact recently I read a great book called "Getting Real" written by the fantastically successful 37Signals team, most famous for creating the online project management software Basecamp and in geek circles for being the creators of the Ruby on Rails framework. I highly recommend buying the book but if it's out of reach then at mininum subscribe to the team's Signal vs Noise blog that is a wealth of design, business, and general startup information.

According to the 37Signals team, programmers should not start designing and building backend functionality first - instead they are proponents of designing the interface before you even start programming. They have a really good point that the interface is the product - it's what people see first and it's what's going to sell your app on Blackberry's app store over your competitors. It seems obvious but if two apps perform similar functionality yet one has an intuitive interface and just plain "looks pretty" it will sell in vast quantities over the less functional app. So in case you didn't get it by now, invest your time in the UI as, in my opinion, it will only benefit you in the end.

If you have any doubt about how much thought can go into the user experience on a mobile application and haven't watched it yet, take a look at the video embedded in this post demonstrating the evolution of the interface for the Convert iPhone application from initial design to final product - amazing isn't it? I have a feeling the TapTapTap team comes from the same design school of thought as the 37Signals guys.



Although "Getting Real" is focused on building web applications, 37Signals UI design philosophy of keeping features/design simple can be extended to other applications including the Blackberry. Yes, the Blackberry has a limited set of Fields for UI elements including the list below, in no particular order and with direct links to the 4.5 API documentation. I suggest you familiarize yourself with these classes and their apis even if you're a novice Blackberry dev. The beauty of these built-in classes is that they can be extended and modified and of course the paint function of any of these elements can be overridden thus, you are limited only by your imagination. In future tutorials I'll show you how to build custom Fields by extending these classes - the coding section of this tutorial will strictly center on building a title screen.



At any rate, besides coming here to read my wonderfully written prose you may have also wanted to learn how to make a title screen for your application. The title screen is important as it identifies your app with a unique look and if your app takes awhile to load, it tells the user that something is going on - otherwise the user clicks your app and proceeds to stare at the home screen while the clock/timer is displayed, something like this:





This is pretty unintuitive so one of the first things I'll do is design an eye-catching title screen with graphics that pop. It really adds to the professionalism of your app. As far as coding the title screen, it's relatively easy to write, we just have to initiate a thread that will push the title screen onto the stack and display it while the application loads. First you have to implement the Runnable interface so we can override the run function for the standard main function that creates the application and enters the event dispatcher, as follows:


public class MainScreenUI extends UiApplication implements Runnable {

public static void main(String[] args){


MainScreenUI theApp = new MainScreenUI();
theApp.enterEventDispatcher();

}
.....//more Class code here



Once this is done you'll create the run function in the main application class, in my example that's the class MainScreenUI and it's constructor is written as follows:


public MainScreenUI(){

TitleScreen ts = new TitleScreen();
pushScreen(ts);
Thread t = new Thread(this);
t.start();


public void run(){
Application.getApplication().invokeLater(new Runnable()
{
public void run()
{
FirstScreen fs = new FirstScreen();
pushScreen(fs);
}
});


So what's going on here? Essentially, MainScreenUI is created in the main() function. the constructor for MainScreenUI is called and it pushes an instance of the TitleScreen (not shown) onto the display stack - TitleScreen is simply a class that extends FullScreen and draws our titlescreen bitmap we've designed in Photoshop, Illustrator, or InkScape via the overridden paint function. We create a Thread named t and start it by calling the start() function.

The thread performs whatever functionality we've designed into the run() function above. In the example we create a new Runnable and add it to the stack of threads to be invoked by the Application - here it's Application.getApplication.invokeLater(...) and we're overriding both the MainScreen run() and the Application run() methods. An instance of our main application entrypoint, FirstScreen(not shown), is created and then is pushed onto the display stack once complete. If your application loads quickly you may want to add a sleep(x) function(where x is some numeric value to wait) before creating the main application screen to give the user time to at least see the titlescreen.

As you can see it's relatively painless but does add some flash to your app.

Python and XML - Simple Example to Parse an XML document

A little background before I show you how to do a simple example to pull node data out of XML using Python. I've used Perl for years and absolutely love it for accomplishing a wide variety of tasks ranging from complex object-oriented solutions to simple text parsing. The power and simplicity of Perl is astounding but secretly I enjoy programming in it because the MIS folks I work with can't wrap their heads around the arcane syntax and who would want MIS guys monkeying around with real code? I'm kidding of course but Perl users must admit that their code is not easily maintainable.

Recently I've had reason to use Python to set up the backend of a Blackberry mobile project and I must say that I really like it. Initially I had trouble simply typing in code as muscle memory forces me to use bracketing for code blocks. You can imagine the frustration having to constantly delete brackets, so I quickly solved this problem by downloading and installing the Pydev Eclipse plugin found here. Armed with code completion, syntax highlighting, and code analysis my development time has been reduced significantly.

Setup


An initial process I needed to automate using Python was simple parsing of XML content and I couldn't find a quick and simple example of how it's done (hence, I thought I'd help other out with this small tutorial). After some experimentation I determined that importing the minidom module would suitably accomplish what I wanted to do and is extremely lightweight so we'll use it for the example code. For the purposes of this article I'll use the following sample XML file courtesy of Microsoft and edited to save some space:




Gambardella, Matthew
XML Developer's Guide
Computer
44.95
2000-10-01
An in-depth look at creating applications 
with XML.


Ralls, Kim
Midnight Rain
Fantasy
5.95
2000-12-16
A former architect battles corporate zombies, 
an evil sorceress, and her own childhood to become queen 
of the world.





Parsing the XML


What we want to accomplish is parse out the child nodes of each book to get the data element of each for processing. With Python it's simple - import the DOM (includes the parse, and Node modules), read in the xml file using the parse() method, and then iterate through the childnodes. For simplicity's sake in the example I'll just print the data for each node to output. Here's the Python code to do just that:


from xml.dom import *
xmlDoc = parse("library.xml")
    for node1 in xmlDoc.getElementsByTagName("book"):
        for node2 in node1.childNodes:
            if node2.nodeType == Node.ELEMENT_NODE:
                print node2.childNodes[0].data 


As you can see, with just six lines of code we've read the entire XML file into memory using DOM, parsed it, pulled out the book elements and printed all the childnodes for each book to output. The check if the node is an ELEMENT_NODE is critical since we do not want to pull the TEXT_NODEs and iterate through them for this example. If you pull that test out the code will attempt to get childNodes from the TEXT_NODE and will fail with "IndexError: tuple index out of range" since the text node doesn't have childNodes.

If you need to pull out the attributes from the Element you can iterate through them by using the attributes dictionary class member of the node. In this case, if node2 had attributes(and it doesn't in my example) you could assign a variable the attributes dictionary 'attributes = node2.attributes' then iterate through the attributes using the 'keys()' method on attributes.

Perl Counterexample


To parse the same XML in Perl you'd have to write the following:
use XML::DOM;

my $file = 'library.xml';
my $parser = XML::DOM::Parser->new();
my $xmldoc = $parser->parsefile($file);
foreach my $book ($xmldoc->getElementsByTagName('book')){
   foreach my $tag ( $book->getChildNodes() ) {
      if($tag->getNodeType == ELEMENT_NODE){
         print $tag->getFirstChild->getNodeValue;		
      }
   }
}



Looks similar and in reality it is mostly identical code but in my opinion it is definitely not as clean as the Python code. Don't get me wrong - I love Perl and am not saying that other languages beat it out - if that were the case I'd have posted a better example to stoke the flames beneath the Python vs Perl zealots. Nonetheless, I must say I'm beginning to enjoy Python quite a bit.


Wrap up


If you came here looking for something other than a simple way to parse XML using Python, such as adding to the ridiculous flame wars that go on between Perl & Python evangelists, look here, here, and here for more on the debate. If you want a pretty good reference for using XML with Python look here

Side Note: Monty Python Rules


By the way, I always thought Python was named after the snake. However, according to the source itself it's really named after Monty Python's flying circus which really strikes a chord with me since I love several of the Python movies, particularly the Holy Grail. Maybe I should change the picture at the top of this post to this:

top