High Earth Orbit » Howto

Migrating a site from PHP to Rails

Andrew — Thu, 12 Oct 2006 10:53:45 +0000

I learn best by picking a project and learning a new technology by applying it to the project. This is how I originally learned PHP, MySQL, and various web technologies when developing the Detroit Curling Club site/portal. Looking back, it’s amazing I got done what I did with what I knew. I implemented a Wiki, CMS, user management, uploading, RSS, and so forth, from the ground up, with no frameworks .

However, the result also means the code is, shall we say, “spaghetti”. And is really no fun to maintain and add to. Since then, I’ve learned Ruby and Rails and really enjoy working in the language and framework and wanted to reimplement the site. With so many other projects going on, I was concerned about the time necessary to get the site migrated.

Actually, it was rather easy. by stroke of fortune. Here is how the work went.

On the catwalk…

By similar mindset, when I designed the original database schema, I used “rails-esque” naming: tables are plural, columns are simple singular names. Join tables are alphabetical of the two tables they are joining. I just had to migrate the names of a couple of tables and columns to mee the rails standard configuration. I could have overriden the table/column names, but didn’t want to deal with maintaining that. I really wanted the site to work like it was originally written with Rails in mind.

You’ve obviously created a Rails application by now, so I’ll skip that. Now, I have my database (with backups). Setup your config/database.yml to point to the database and then run rake db:schema:dump to create the schema.rb of the database. If you want, you can copy this to a migration file.

Next, install Dr. Nic’s Magic Models, which automagically give you your model files, associations, basic validations, and so forth with no coding. Magic just begins to describe it. You can verify this by bringing up script/console and toying with your models and associations.

Testing is good

A good idea at this point is to load Selenium, a web user-interface testing application, and walk over your original site to create user interface requirements based on current site design. This should stand as your base-line specification. In the end, your Rails app should act like your current application.

You can also check out the work of Josh Susser who has a ruby script that will validate all the records in your database using your model validations.

Define your application test specification using RSpec and BDD. Again, you already have an existing web application to help define your desired functionality.

The View, the View

Now you have a your base application done. You should start migrating your views. Fill in your layout/application.rhtml with the base layout from your original HTML files.

You’ll now need to start actually defining your controllers for various functionality: calendar, users, leagues, etc. Along the way, run your tests you defined with RSpec and verify operation. This is where the actual work comes in. Overall though, I find the design of Rails to make this very easy going. I just have to deal with various legacy functionality.

I’m also looking more at Unobtrusive Javascript for Rails (ujs4rails) at how to handle graceful degradation (or is it graceful upgrading) to browsers that may not support javascript.

Work in Progress

That all said, migrating the site has been very easy. So easy I’m looking at migrating other projects to Rails. Of course, deploying Rails applications to shared hosts is not nearly as easy as deploying PHP ones. Especially with applications that are meant for other users to easily install on their own hosts. But the ease of development and maintenance is not easily overlooked.

If only Apache supported a built-in Rails module.

TextMate Bundles

Andrew — Sun, 01 Oct 2006 20:33:14 +0000

I’m learning the power that is TextMate (like Emacs, but prettier). TextMate really is setting itself up to be a rather decent OS for editing. From within TextMate I can blog, keep a todo list, program in any number of languages, WYSIWYG webpage editing, and calendaring. Heck, mix TextMate and Quicksilver and who *needs* Finder. I picked up the new Pragmatic Programmer’s TextMate book to learn more of the underpinnings and how to stop repeating myself with simple tasks.

Like my thoughts on why plugins make, TextMate has excellent support via bundles (over 124 of them). However, the only way to get to these is via subversion. This is nice because it keeps me up to date, but is a little annoying having to get the subversion command right and path all the time.

So I wrote a small shell script to grab a bundle, get_tmbundle:

#!/bin/bash
svn --username anon --password anon co http://macromates.com\
/svn/Bundles/trunk/Bundles/$1.tmbundle ~/Library/Application Support/TextMate/Bundles/$1.tmbundle

I can then $ get_bundle AppleScript to get a new bundle installed.

You could also use the getBundle Bundle, but that seems like cheating. Of course, getBundle also has an AutoUpdater and is baked into TextMate as a bundle itself (how deep does the Rabbit Hole go?)

Technorati Tags: mac os x, macosx, programming, subversion, textediting, textmate

Daily del.icio.us blog posting

Andrew — Thu, 24 Aug 2006 23:08:25 +0000

By popular demand (ok, one request), I am putting up instructions on how to have your del.icio.us links posted daily to your blog.

I like this, as feed readers then have a succinct list of interesting sites to show (rather than a sidebar, who goes to a blog anymore?). A list of links is often more interesting than some half-written paragraph of diatribe just to get to the point. (case in point).

How to setup del.icio.us to post daily links

Log into del.icio.us
On the right-side, under “experimental”, choose “daily blog posting”
Click “add new thingy” (yes, it really says “thingy”, those crazy del.icio.users
Fill in the form as appropriate, here is an example:
- job_name: DailyLinks
- out_name: (blog user – e.g. delicious)
- out_pass: (blog password for that user)
- out_url: http://highearthorbit.com/xmlrpc.php
- out_time: 3 (approx time to post, in GMT)
- out_blog_id: (leave this blank)
- out_cat_id: (category id, look at the id in your blogging software)
Click “Submit Query”

Sometime within the appointed hour (they’re spaced out so neither del.icio.us’s servers, or ping services, are flooded ‘on the hour’) your new links will be posted.

For this blog, I created a new user, gave that user Author privileges, and created a specific category, bookmarks

No LaTeX, never fret

Andrew — Wed, 12 Jul 2006 20:43:41 +0000

So you’re writing a paper with citations and you’re wishing you had LaTeX’s beautiful self-referencing ability to your list of references. However, for whatever reason, you’re stuck using Microsoft Word. And clicking through Insert->Reference->Cross-reference…, then scroll to find your article, and set to paragraph number, etc. has got you really down.

Bet you didn’t know about field codes. Instead, just put your cursor in your citation brackets (or whatever you use), press -F9, and then REF NameYear \n.

... demonstrated in on an airship [{REF Turner02 \n}].

The \n specifies that you want the paragraph number of the reference, which equates to the ordered-list number. Later, in your references section, you will Insert->Bookmark and properly name your citations (again, I use NameYear). You then need to hit Alt-F9 turn ‘hide’ all the field codes in your doc and update and they’ll all be linked.

It’s not pretty, it’s not LaTeX (not that LaTeX is pretty either), but it works.

GeoNames supports reverse geocoding

Andrew — Mon, 26 Jun 2006 13:21:04 +0000

GeoNames is YAG (yet another geocoder), but behind the curtains lie many cool features. The most unique of which is a reverse geocoder.

Reverse geocoding is converting Latitude & Longitude to a place name. This is the other side of the mirror from traditional geocoding, which converts a place name into latitude & longitude. Why would someone want to reverse geocode you ask? With reverse geocoding you can convert your GPS tracklogs into meaningful locations easily, or allow users to click on a map and get back actual location of where they’re clicking.

The GeoNames API provides an interface for getting nearby postal codes, country, or most specific: place names.

Example

HighEarthOrbit offices: http://ws.geonames.org/findNearbyPlaceName?lat=42.4266&lng=-83.4931&style=full

returns:


  
    Northville
    42.43111
    -83.48333
    5003956
    US
    United States
    P
    PPL
    city, village,...
    populated place
    6360
    252
    MI
    Michigan

The whole geo-enchilada

Lastly, to make you really feel warm and good inside, the GeoNames database is provided for direct download under a Creative Commons Attribution license. Yum, free data.

See the GeoNames Blog post about it.

Bluelogger GPS

Andrew — Wed, 03 May 2006 17:03:00 +0000

For the past several months I’ve been playing with the Delorme Bluelogger GPS. It’s a very nice GPS receiver in its own right, but has the unique feature (for BT receivers) of including onboard logging. This is an great feature as it allows me to turn on the unit, toss it in my bag (somewhere on top, so it can still get a view of the sky) and forget about it. I don’t need to grab multiple devices, such as the receiver and a PDA or computer in order to receive and store my GPS waypoints.

This article will give a short overview of how to use the BlueLogger for a variety of applications. I use it primarily for geolocating photographs, but it’s also nice for any location-based activity.

The Bluelogger comes with the following:

Bluelogger device
Carrying case (with belt loop)
Car charger
Wall charger
Charging stand (can work with either car or wall charger)
Bluelogger Windows software

Connecting to GPS

To connect to the Bluelogger, you will need a bluetooth adapter. Many computers now come with bluetooth built-in or as an add-on option. If this isn’t the case, I would recommend the D-Link DBT-120. It’s probably the only D-Link product I can recommend, but I’ve had great luck with them, and never run into any device that it hasn’t worked with (and I have had problems with other BT adapters, especially on my Mac).

Once you have a bluetooth adapter, you will need to setup a connection to the device by pairing them. See your devices’/operating systems’ manuals on how to do this.

A very slick option is to run gpsd, which is a service-daemon running in the background that allows multiple connections to the single GPS device. Normally, only 1 software instance can connect at a time. With GPSd, you can “serve” your location. What would be really cool is to have GPSd be able to connect in with Geolocation by IP or Wifi as well as an actual GPS device to seamlessly switch between location technique.

GPSUtility is a nice, compact, graphical GPS application for Mac OS X. It can connect either directly to the GPS bluetooth port, or via gpsd. You can view location, satellite strengths, verbose NMEA output, and speed.

KisMac, while not a GPS-specific applicaiton, has excellent GPS support. KisMac is actually a wireless stumbler, which can also mark the latitude, longitude, and strength of detected networks and plot these on a map.

Storing & Viewing tracks

The Bluelogger software (currently Windows only) can export the tracks as GPL files. GPSBabel can convert these to a more useful format, such as GPX, which an XML format for GPS data.

Since using the bluelogger usually entails turning it on and off often, the entire track log will contain many separate trips. GPSBabel supports splitting up tracks based on a time separation. Each segment will be a self-contained track.

The following example will convert a GPL file to a GPX file, and make a individual track for any separation of 4 hours between points.


$ gpsbabel
  -i gpl -f Track_2005_11_23.gpl
  -x track,pack,split=4h,title="LOG # %c"
  -o gpx -F Track_2005_11_23.gpx

Displaying tracks

There are several options for displaying your tracks:

GPX Tracks

GPXLoader

GPX Tracks HTML file

GPS and Nokia 770

ThoughtFix has a fairly comprehensive tutorial on setting up GPSDrive with a Bluetooth GPS receiver. They went with the i-Blue High Sensitivity Bluetooth GPS, which looks like a nice unit, but lacks logging.

Resources

GreaseRoute – Mapping the web

Andrew — Tue, 02 May 2006 02:22:24 +0000

I’ve been fairly quiet for the past couple of weeks. I’ve been focused on some projects, including entries to the MapQuest OpenAPI developers’ competition.

One of my entries, which may or may not actually be considered a “mashup”, is a Firefox GreaseMonkey script called GreaseRoute that creates map and route icons for an Microformat formatted adr or geo locations. These are often used in an hcard.

One of the cooler parts is that the extension automatically determines the location of the person viewing the page. Using the HostIP IP-to-Location database, the user’s location is automatically entered as the starting location. So when a user is viewing the webpage of, for example, a store or business, they can click the “route” link and get turn-by-turn directions to that location. No more copy & pasting street/city/state to a form or asking the person next to you “where are we?”

Microformats are a stepping stone on the way to the semantic web. They are a simple way to encode useful information for viewing and mining by users and other applications (ala mashups).

Example adr

The proper format of the adr is as follows:


 1517 N. Main St.
 Box 203
 Royal Oak,
 MI
 48067
 US

which will then be displayed as:

1517 N. Main St.

Box 203

Royal Oak,
MI
48067

Example geo

Instead of an address, one can instead just encode the latitude and longitude:

N 37° 24.491
W 122° 08.313

The Result

GreaseRoute has 2 version, the lite version and the embed version. The lite version displays a map and route icon next to the addresses found on the webpage:

The embed version displays a “route” link. When clicked, a pop-up map is displayed between the users location and the encoded address.

Give GreaseRoute a try and let me know what you think.

You can also go over to Userscripts.org, the GreaseMonkey script repository, and rate them and leave comments.

Creating inline (embedded) PNG images

Andrew — Sun, 23 Apr 2006 14:55:53 +0000

Sometimes it’s necessary, or nice, to be able to directly embed an image in code. Like a modern form of ASCII art, you then don’t have to distribute icons and images with your source, because the image is in the source itself. The method uses Base64 encoding. See the RFC for Data URI.

“Base64″ format used by MIME-encoded documents such as electronic mail messages with embedded images and audio files.

To convert a PNG (for example) image to Base64, you will need some utility. base64 is an open-source converter that does just this.

Do the configure, compile jig:

$ ./configure
$ make
$ sudo make install

Then encode your image:
base64 -e my_image.png my_image.png.base64

Open the my_image.png.base64 file in a text editor and then copy it into your code. An example is shown below, written in Javascript.


const png_image_src = 'data:image/png;base64,'+
'iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAABGdBTUEAANbY1E9YMgAAABl0' +
'RVh0U29mdHdhcmUAQWRvYmUgSW1hZ2VSZWFkeXHJZTwAAAGAUExURcDYv1C8WdPrtIDAeWqk' +
'Oqnd2VLWaOX0x7ftwZzZlIjGxr7gxanLkdfu1FumybHSsJ3Cutbyx4S9fm2paMrq0Lfo1L/a' +
'o6PHiozL11aZJpnW2oG0ZLTcm5zjvovbvM3qtcror2q5vUaIRHLGZWWkR0aQGSR/BYy5baHY' +
'imrbd/z+/HO9bPb69m29xHy8c2OeMUGJPZDThZPPi3ivdGazzZbBgCuCEd3x2jaICs7ksqLZ' +
'mTp/N6zwtWu/YLDWvZHHeqLS0affzKfvsWDYpn7CvJHJh0+cSGWbY6PQka/Xkvj894W2a4DQ' +
'qIrDhF+hQ8fexJDN24PYgYGzWnXJy6zXpp3Fm5jCg37NdI++dWejWmCwVnSrT6nuyTuLELvh' +
'pV2gO6DSwI3qnd3wv6rQxK/fz1OZxrTvtaLim3rRr+bv5TuCN+jw557FiK7NlWzRkHDGhHLd' +
'fnHWmm2zx7Xe1m21zW65z9Ts0HTHaHrGb3vJb3yvVbTaxVGUTITLtV6w0////94DUpYAAACA' +
'dFJOU///////////////////////////////////////////////////////////////////' +
'////////////////////////////////////////////////////////////////////////' +
'//////////////////////////////8AOAVLZwAAAQxJREFUeNpiqAcCc87wcuVKTl4Qm6G+' +
'XstKOU6eu9ouxNbICySgZajBzluWUuzgkOyqpwMUsNJIEuQVYWUNkOCysxeqZzBXZk+z4RCV' +
'kiopquVUcPNn4JSRd4oRkZAwUSwsKPAQN2YIZ0pPFBUpNQnWZGNj0xOrYShPCtSUdeTTlXNm' +
'0WcUtjRgUE4Kz5Plq0vVzWAJkoy0zGKokDeU4+JjYWHRr+KJVhVTYjDylMn34WPJzeXhiRBX' +
'sXBnKIti0tZW1M9VV/eWlg6LV2Oo943wZHaR5BEXz/EOUzHTYajXEfZWSJBUVY2NN1UxywR5' +
'TsfYIFRAQIDfws8sG+xbICGsZG2tpKamBWQDBBgAU049f2kd9aAAAAAASUVORK5CYII=';

var my_link = document.createElement('a');
my_link.title = "Go to example.com"
my_image = document.createElement('img');
my_image.src = png_image_src;
route_image.border = 0;
my_link.href  = "http://example.com";
my_link.appendChild(my_image);

document.body.insertBefore(my_link, document.body.lastChild);

Javascript XML parsing for GML and you

Andrew — Sun, 23 Apr 2006 14:11:15 +0000

You find the need to determine the locations of your visitors, and you wonder how to do this? Surely it’s impossible. Alas, it is not. Services like HostIP provide the ability to locate your users given their IP address. The slick way to do this is via javascript. Unfortunately, the HostIP API doesn’t provide a JSON interface.

But of course, the response is in XML, so you’re also wondering how to parse this XML response for something meaningful? The answer is the DOMParser and namespaces. There is a really nice function, getElementsByTagNameNS that allows you to parse out elements given a namespace. I kept trying getElementsByTagNameNS("gml","name"), which didn’t work. The answer seems to be that you need to include a valid uri for the namespace.


var dom = new DOMParser ( ) .parseFromString ( responseText,
                            "application/xml" ) ;

var location = dom.getElementsByTagNameNS (
   "http://www.opengis.net/gml", "name" ) [1].firstChild.data
    if location == " ( Unknown city ) "
        return "";

Publish iCal to Google Calendar

Andrew — Thu, 13 Apr 2006 21:26:43 +0000

Let me attempt to redeem my prior remarks about Google Calendar by offering this bit of how to publish your iCal (via iCal or Mozilla Sunbird) to Google. This was inspired by How to subscribe to Google Calendar in iCal.

Create your calendar

You have numerous options for creating your calendar: iCal (Mac), Mozilla Sunbird (Mac, Windows, Linux), Outlook2iCal (Windows), etc.

Setup a public server to host your calendar

You have several options to publish your iCal calendar:

Apple .Mac provides iCal hosting, but costs $$ (or £, ₪, 圓, ﷼, etc.)
Setup WebDav on your own server to host calendars
iCal Exchange is a free calendar sharing host
Just FTP/Automator your iCal file to a public server

Publish your calendar

Using one of the prescribed methods above, you will then want to actually publish your calendar to the host (or upload via FTP/SSH/SFTP/carrier pigeon on 5.25″ floppy disk). You can turn on “Publish Changes Automatically” to have your changes on your computer automatically uploaded to Google Calendar.

Subscribe to your calendar in Google Calendar

Do the following:

Manage Calendars
then under “Other Calendars”, click “Add calendar”
Fill in the URL of your published calendar – this is probably supplied by the hosting server (like iCal Exchange, .Mac, or you can figure it out for your own server)
Click “Ok” and wait a little bit for it to be uploaded

Bask in the warming, glowing warmth…

You’re now done and have a Google Calendar version of your local iCal calendar. Unfortunately, at this point, Google Calendar doesn’t let you edit this “public” calendar, though you can add events from this calendar to one of your Google Calendar calendars (if that makes sense, you’re astute). Now you can just publish when you’ll be available to stop at the local pub for a brew!

Spatial programming with Ruby on Rails

Andrew — Mon, 20 Feb 2006 01:50:20 +0000

Many of my projects employ a spatial aspect. I am fascinated by location, context, community, and of course, the technology that drives this all. I created my travelogue a while ago, but it’s rather dated, buggy, and not fun to maintain or update since it’s all written in PHP as I was learning PHP.

I’ve really gotten “rolling with rails” (pun intended). One of the factors that really excites many developers is an active community. This can be seen by observing the Mac community (hyperlink any number of weblogs, fan sites, howtos, books, et al. here), car clubs, kennel clubs, etc. Ruby and Rails are both pumping out huge numbers of plugins, add-ons, and howtos that further fuels the fire.

However, there aren’t currently any good spatial libraries or support within RoR. Specifically, I would like to store general geometry information (areas, points, lines) in a database, create the model in Rails, and then query the model with code like: @user.get_within_radius(:users, 50).

Databases and GIS

My biggest difficulty is that my shared host (Dreamhost) does not provide PostgreSQL databases, only MySQL. As anyone in the GIS field knows, PostGIS, an extension module for PostgreSQL is the requirement for proper spatial database storage and retrieval. Instead of storing locations as a pair of floats, true geometry in the form of POINT, POLYGON, CURVE, and others are used.

MySQL Spatial Extensions are began to emerge in MySQL 4.1. However, as this article on GIS and Spatial Extensions with MySQL points out, many of the core functions are missing (most notably Distance()). This is true even in the current MySQL 5.0. The result is having to do more complex queries to get distance information:

SELECT
  c.cab_driver,
  ROUND(GLength(LineStringFromWKB(LineString(AsBinary(c.cab_loc),
                                             AsBinary(a.address_loc)))))
    AS distance
FROM cab c, address a
WHERE a.address = 'Foobar street 110'
ORDER BY distance ASC LIMIT 1;

In the meantime, there are several stop-gap solutions. Bryan Wood (glytch.com) put together a Ruby library file that provides a small number spatial operation utilities, and is useful for operating on the “simple database scheme” of a latitude, longitude pair of float columns. The operations are performed by doing a simple SQL query where the latitude and longitude are within max/min bounds. This can’t handle radius or spherical operations quickly, but is good for a rough estimate.

For an actual spherical measurement, you can do the following in your ruby code (via GeocodeAmerica):

@places = Place.find_by_sql ["select p.* from places p where
     ((3963.0 * acos(sin(p.latitude/57.2958) *
       sin(?/57.2958) +  cos(p.latitude/57.2958) * cos(?/57.2958) *
        cos(?/57.2958 - p.longitude/57.2958))) < ?)",
        lat, lat, lon, distance]

Extending Ruby on Rails

A better solution would be to develop a model that provides a GIS interface to a PostGIS or other OGC-style database. This requires extending the Ruby on Rails model to arbitrary column data types, and then providing the data types as getters/setters.

Given the SQL create cod:

CREATE TABLE address (
  address CHAR(80) NOT NULL,
  address_loc POINT NOT NULL,
  PRIMARY KEY(address),
  SPATIAL KEY(address_loc)
);

class Address < ActiveRecord::Base
        def address_loc=(textrepresentation)
           write_attribute(:address_loc,
           Address.find_by_sql(["SELECT GeomFromText(?)
as value", textrepresentation]).first.value)
        end

        def address_loc
          Address.find_by_sql(["SELECT AsText(address_loc) as value
FROM addresses WHERE id = ?", id]).first.value
        end
end

Geocoding

Geocoding is the act of converting a location name (major name, street address, etc.) into latitude and longitude (or whatever your preferred numerical location representation may be).

Ruby Geocoding Resources

Floating text below an absolutely positioned div

Andrew — Mon, 23 Jan 2006 02:41:55 +0000

This is probably odd to most people, but for a project I had to put a caption below an image on a webpage. But I wanted the image to be fixed in position and the caption would automatically flow and remain centered no matter how long the string was. I knew the size of the image (original div) but not the text string.

The trick lies in embedding the caption div within another div. Then the outer div should have a left: equal to half the image (or other div)’s width. The relevant CSS is:


outercaption {
	position: relative;
	left: 13px;
	}
.caption {
	text-align: center;
	position: relative;
	left: -50%;
	white-space: nowrap;
	}

Which will work with this HTML code.






Short

Here is an example. You can even put in your own text to see how it is dynamically sized.