12.26.07

Realizing AtomPub entries from XML using ROXML

Posted in Ruby, blog, rails at 11:00 am by Robert Horvick

I have done some digging and there does not seem to be any plugins or gems (or even active projects on RubyForge) that are providing support for an AtomPub server. I did find a sample based on Camping but it was not really what I was looking for. It supports a subset of AtomPub and not in a way that is really friendly for what need.

So I set out on my own.

An atom pub entry can be as simple as this:

<atom:entry xmlns="http://www.w3.org/2005/Atom">
  <atom:title>ATOM Post Test</atom:title>
  <atom:id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</atom:id>
  <atom:content>Some text.</atom:content>
</atom:entry>

Or quite a bit more complex (lifted from http://tools.ietf.org/html/rfc4287):

<entry>
  <title>Atom draft-07 snapshot</title>
  <link rel="alternate" type="text/html"
    href="http://example.org/2005/04/02/atom"/>
  <link rel="enclosure" type="audio/mpeg" length="1337"
    href="http://example.org/audio/ph34r_my_podcast.mp3"/>
  <id>tag:example.org,2003:3.2397</id>
  <updated>2005-07-31T12:29:29Z</updated>
  <published>2003-12-13T08:29:29-04:00</published>
  <author>
    <name>Mark Pilgrim</name>
    <uri>http://example.org/</uri>
    <email>f8dy@example.com</email>
  </author>
  <contributor>
    <name>Sam Ruby</name>
  </contributor>
  <contributor>
    <name>Joe Gregorio</name>
  </contributor>
  <content type="xhtml" xml:lang="en"
    xml:base="http://diveintomark.org/">
      <div xmlns="http://www.w3.org/1999/xhtml">
        <p><i>[Update: The Atom draft is finished.]</i></p>
      </div>
  </content>
</entry>

I started with just caring about title and content. Everything else I could infer from authentication (author, anyway).

With this thought, I created some simple xpath queries to find the title and content. That was working out. I could get the atom response created and the post serialized to the database. So with just a few lines of code (beyond the scaffolding) I was able to use Live Writer to post via atompub to my blog engine.

But I wasn’t happy with just winking away the atompub protocol in favor of getting done quickly.

So I sat down and really started reading the atom publishing spec (and the a related atom rfc). I have a lot of work to do.

I started using XmlSimple to parse the XML. That was ok, except some things don’t map well to hashes and it did not get me closer to serializing atom entries to xml (for the feed later on).

So I moved down to REXML and that was working ok. But I still wasn’t happy. I felt like I was writing too much monkey code and not doing making progress on my real goal.

Tinally I found ROXML - and now I’m happy. I’ll cut to the chase and just show the code:

require 'rubygems'
require 'roxml'

class AtomBase
  include ROXML

  def to_s
    self.to_xml
  end
end

class AtomAuthor < AtomBase
  xml_name "author"
  xml_text :name
  xml_text :uri
  xml_text :email
end

class AtomContributor < AtomBase
  xml_name "contributor"
  xml_text :name
  xml_text :uri
  xml_text :email
end

class AtomLink < AtomBase
  xml_name "link"
  xml_attribute :rel
  xml_attribute :type
  xml_attribute :href
  xml_attribute :length
end

class AtomContent < AtomBase
  xml_name "content"
  xml_attribute :type
  xml_attribute :lang
  xml_attribute :base
  xml_text :text, nil, ROXML::TEXT_CONTENT
end

class AtomSummary < AtomBase
  xml_name "summary"
  xml_attribute :type
  xml_attribute :lang
  xml_attribute :base
  xml_text :text, nil, ROXML::TEXT_CONTENT
end

class AtomEntry < AtomBase
  xml_name "entry"
  xml_text :title
  xml_object :link, AtomLink, ROXML::TAG_ARRAY
  xml_text :id
  xml_object :summary, AtomSummary
  xml_text :updated
  xml_text :published
  xml_object :author, AtomAuthor
  xml_object :content, AtomContent
  xml_object :contributor, AtomContributor, ROXML::TAG_ARRAY
end

The major gap is that this does not support xhtml content and summary fields. Those fields contain embedded XML and I have not figured out how to get ROXML to stop navigating the tree and make the object property return the inner xml.

I did try hacking ROXML to support an XML_CONTENT tag. It solved about 70% of the problem but it was not working exactly as I wanted after about a half hour and I felt like I was shaving a yak. I can come back to that later on. For now that the atom bits were are neatly wrapped up behind a class with a known bug. I’m ok with that.

I haven’t reviewed the atom syndication spec to make sure I’m doing everything right but it’s working with all the samples I’ve thrown at it (including the samples from the spec) and Live Writer seems to be having a field day with it. So that’s good.

Anyway - that’s where I’m heading.

I have the feeling that I’m reinventing the wheel. I hope not poorly.

Permalink 5 Comments

12.25.07

Just what the world doesn’t need … another blog engine.

Posted in Ruby, blog, rails at 9:36 pm by Robert Horvick

Nuby on Rails recently asserted that “every beginning Rails developer should write their own blog software. It’s a great learning experience and you can try things that aren’t possible with just an app running on localhost.”

I can buy into this.  It’s the canonical Rails sample app.  I’m new to Ruby and Rails.  Maybe I should do this.  And I don’t mean a little toy that I do in 15 minutes and then throw away.  I mean the blogging platform I use for my primary blog.

So I sat down and tried to list out what I would want my blog to look like …

1. I don’t want to spend time writing code for a text editor to write blog posts
2. I want people to be able to subscribe to my content
3. I want caching (not that I plan to get dugg - but it’s good learning)
4. I want to be able to easily customize the look/feel
5. I want to support tagging and providing post views based on those tags
6. I want to support uploading media files (images, etc)
7. I want to be able to import this blog into it.

So I spent some time researching those issues and this is what I’ve come up with:

1. I will post via atom publishing -there are already blog tool that support this (Microsoft Live Writer, for example - I’m sure there are others).
2. I will provide atom feeds.
3. Rails has lots of caching examples.  I’ll start with file system based caching and move upwards from there.
4. I won’t provide full theme support but I’ll have a generic post format and include user defined stylesheets (and possible javascript).
5. acts_as_taggable until shown why not.
6. atompub can do this.
7. I think I can use atom feeds to do this.

Also …

As mentioned I’m hosting with SliceHost.com on a 256 slice so I need to keep it trim.  I may go to production on Sqlite3 (if I do the caching right this shouldn’t be a problem - and that would keep 4-10% of my limited RAM free).

Also my goal is to write as little new code as possible.  Plugins and gems whenever possible.

I figure it will take about a month to have something I can show to the world and not worry about hiding my face.

Over the next few weeks I’ll be blogging about what I learn that seems interesting.  And plenty that isn’t, I’m sure.

Permalink Comments

11.29.07

Use escape_javascript when rendering model data into a js string

Posted in Ruby, rails at 5:00 am by Robert Horvick

To give users a better view into some data I want to render local points of interest with current deals onto a Google map.  In the marker popup I’d like the user to be able to drill down further into the details.  To do this I want a hyperlink to the deal details.

 The link_to I want is:

link_to "Details", { :controller => "deal", :action => "details", :id => deal.id }

 The problem is that the rendered <a> should go into a blob of javascript such as:

       GEvent.addListener(        marker,"click",        function() {    

          var myHtml = "<b><%= h deal.Restaurant.name %></b><br/><%= h deal.description %></br><%= link_to "Details", { :controller => "deal", :action => "details", :id => deal.id } %>";    

          map.openInfoWindowHtml(marker.getLatLng(), myHtml);    

        }    

      );

That fails because the string literal has embedded double quotes.

Since ‘h’ (alias for html_escape) exists I figured there had to be a javascript version as well - and action view did not disappoint.

The updated code is:

      GEvent.addListener(        marker,"click",        function() {    

          var myHtml = "<b><%= escape_javascript h(deal.Restaurant.name) %></b><br/><%= escape_javascript h(deal.description) %></br><%= escape_javascript link_to("Details", { :controller => "deal", :action => "details", :id => deal.id }) %>";    

          map.openInfoWindowHtml(marker.getLatLng(), myHtml);    

        }    

      );

The javascript now renders correctly and the link to the details page functions as expected.

I’m still shooting in the dark - but I think I’m facing the target.

Permalink Comments

11.28.07

Getting RSpec working on Windows with InstantRails

Posted in Ruby, rails, rspec, testing at 6:00 am by Robert Horvick

Since bringing a copy of my current project down from heroku to test some features I decided to give RSpec a try at the same time.

My gut feeling was to run “gem install rspec”.  So, for the record, that is NOT the way to properly install rspec on a rails site.

To use rspec on a rails site you need to install it as a plugin.

The documentation page on installing RSpec indicates that I should do this be running the following commands from my ruby shell:

ruby script/plugin install svn://rubyforge.org/var/svn/rspec/tags/CURRENT/rspecruby script/plugin install svn://rubyforge.org/var/svn/rspec/tags/CURRENT/rspec_on_rails

When I did this nothing happened.  No output whatsoever. 

I tried running the next command on the install guide (output provided):

>ruby script/generate rspec 

Couldn't find 'rspec' generator 

I figured the problem was not having svn installed (let me take a moment here to say that the plugin script could do a better job of indicating this - silent failure smells a lot like success).

I installed downloaded the win32 client binaries from the subversion site http://subversion.tigris.org/downloads/1.4.5-win32/apache-2.2/svn-win32-1.4.5.zip and extracted them to c:\svn  (so c:\svn\bin now contains svn.exe).

In my ruby command shell I added c:\svn\bin to the path (set PATH=%PATH%;c:\svn\bin) and now re-running the plugin installation succeeded (thousands of lines of status scroll by).

Next I ran the following (with output):

C:\InstantRails\rails_apps\kidseatfree>ruby script\generate rspec 

      exists  spec 

  create spec/spec_helper.rb 

      create  spec/spec.opts 

  create previous_failures.txt 

      create  script/spec_server 

      create  script/spec

Finally I ran “rake spec” - which showed I had a missing gem dependency:

C:\InstantRails\rails_apps\kidseatfree>rake spec 

(in C:/InstantRails/rails_apps/kidseatfree) 

C:/InstantRails/rails_apps/kidseatfree/vendor/plugins/rspec/lib/spec/runner/formatter/base_text_formatter.rb:37:in `colour=': You must gem install win32console to use colour on Windows (RuntimeError)

So I ran “gem install win32console” and now rake spec gave the following (I already had an rspec test written so there was something to do):

Errno::ENOENT in 'Deal with fixtures loaded should have one record' 

No such file or directory - C:/InstantRails/rails_apps/kidseatfree/config/../spec/fixtures/deal

So I copied my fixtures from test\fixtures to spec\fixtures and re-ran:

C:\InstantRails\rails_apps\kidseatfree>rake spec 

(in C:/InstantRails/rails_apps/kidseatfree) 

..Finished in 0.166 seconds 

2 examples, 0 failures

Permalink 3 Comments

11.27.07

Getting an exported heroku project working

Posted in Programming, Ruby, heroku, rails at 6:00 am by Robert Horvick

After exporting a project from heroku to do some local work there were two things I needed to do:

1.  Create config/database.yml - it is good that this file is not exported.  It contains no details that are useful outside of the heroku environment.  But don’t forget you need it (if you don’t know what to put there - create an empty rails project and grab that one).
2. create the databases referenced in config/database.yml
3. Execute “rake db:sessions:create” - this will create a migration that is necessary to enable SQL session storage.
4. Execute “rake db:migrate” - this will execute all of your migrations as well as the SQL session migration.

After this my project was working.

Permalink Comments

11.26.07

heroku.com + script.aculo.us = broken ajax

Posted in Programming, Ruby, ajax, heroku, rails at 4:55 pm by Robert Horvick

Update: As I expected the guys are heroku turned around a fix very quickly.  I tested autocompletion tonight and it worked perfect.  heroku + script.aculo.us = working like a champ :) 

They say that constructive criticism (e.g. complaints) should occur in a sandwich of positive thoughts.  Generally I think that’s a bunch of PC granola-farming nonsense.  People need to hear what people need to hear.  If I suck - tell me.  If I’m lucky you’ll tell me why.  If I’m really lucky you may offer advice on how to improve.

But knowing that something sucks is the baseline.

But I’m going to break my rule and go with the PC warm-fuzzy approach.

Positive: Heroku kicks ass

Rails site online fast, syntax highlighting editor, backup/restore, nginx, postgres … love it.  I’ve been able to make a lot of progress in this environment (and being able to work on the site from multiple computers is very nice).

Negative: Heroku injects some code into every response

This breaks ajax behaviors by altering partials (and includes prototype.js which breaks script.aculo.us) … in short ajaxy rails is broken.

When I make an ajax request I expect to get back something like:

<ul class="restaurants">

  <li class="restaurant">    <div class="name">rest name</div>

  </li>

  <li class="restaurant">

    <div class="name">O'Charley's</div>

  </li>

  <li class="restaurant">

    <div class="name">Sunday Deal</div>

  </li>

  <li class="restaurant">

    <div class="name">Monday Deal</div>

  </li>

  <li class="restaurant">

    <div class="name">Sunday Cheap</div>

  </li>

</ul>

But what I actually get back is:

<!-- heroku toolbar -->

<script src="http://heroku.com/javascripts/prototype.js” type=”text/javascript”></script>

<script src=”http://heroku.com/javascripts/effects.js” type=”text/javascript”></script>

<script src=”http://heroku.com/toolbar/heroku_toolbar.js” type=”text/javascript”></script>

<link href=”http://heroku.com/toolbar/heroku_toolbar.css” media=”screen” rel=”Stylesheet” type=”text/css” />

<script type=”text/javascript”>HerokuToolbar.html = ‘<div id=”heroku_toolbar”><a href=”http://heroku.com” id=”heroku_logo”></a><a href=”http://edit.kidseatfree.heroku.com/?uri=%2Fdeal%2Fauto_complete_for_restaurant_name” id=”heroku_back_button”></a><a href=”javascript:HerokuToolbar.toggle()” mce_href=”javascript:HerokuToolbar.toggle()” id=”heroku_inspect_button”></a><img id=”heroku_spinner” src=”http://heroku.com/toolbar/images/spinner.gif” style=”display: none”><span id=”heroku_login_info”><a href=”http://heroku.com/myapps”>robert.horvick@gmail.com</a> | <a href=”http://heroku.com/logout”>logout</a></span></div>’</script>

<span class=”heroku_marker heroku_start” style=”display: none”>/mnt/home/userapps/347/app/views/deal/_restaurants.rhtml</span>

<ul class=”restaurants”>

  <li class=”restaurant”>    <div class=”name”>rest name</div>

  </li>

  <li class=”restaurant”>

    <div class=”name”>O’Charley’s</div>

  </li>

  <li class=”restaurant”>

    <div class=”name”>Sunday Deal</div>

  </li>

  <li class=”restaurant”>

    <div class=”name”>Monday Deal</div>

  </li>

  <li class=”restaurant”>

    <div class=”name”>Sunday Cheap</div>

  </li>

</ul>

<span class=”heroku_marker heroku_finish” style=”display: none”></span>

Ah crap.  Now my autocomplete doesn’t render … oh but it gets worse.

Reincluding prototype.js causes the scriptaculous extending of Ajax.Autocompleter to be lost and now autocomplete events don’t fire (in fact FireBug shows an error because Ajax.Autocompleter is not a constructor).

Bug reported.

Positive: The folks at heroku kick ass

I have reported quite a few issues - some bugs, some feature ideas, some random rants about usability or business models.  But in every case these guys have turned around fixes or offered workarounds in a matter of hours. 

The issue I just described - I’m confident that they will be able to work past it quickly or provide some sort of workaround for the specific case of partials.

So there you go.

A feedback sandwich.

Now I’m hungry.

Permalink Comments

11.25.07

A rails form that creates multiple models at once

Posted in MySQL, Programming, Ruby, rails at 6:10 pm by Robert Horvick

I want to create a simple site that allows users to add event, restaurant and location details through an ajax-ish interface.  Ultimately I have a design in mind but to get there I need to be able to submit details of multiple models through a single form (there are about 10 fields over 4 models that will need to be defined).

I wanted to use the action view “form_for”, a single submit and have the submission automatically map the fields to their proper object so that my controller method could look like this (sans error handling):

def create
    if request.post?
@deal = Deal.new(params[:deal])
@deal.Restaurant = Restaurant.new(params[:restaurant])
@deal.Location = Location.new(params[:location])
      @deal.save
    end
  end

Long story short - what you need to do is use the “fields_for” call within the form to get the model information. The form in the view looks like:

<% form_for :deal, :url => { :action => :create } do |form| %>
  <% fields_for :restaurant do |r| %>
Restaurant: <%= r.text_field :name %><br/>
  <% end %>    

  <% fields_for :location do |l| %>
Address 1: <%= l.text_field :address1 %><br/>
Address 2: <%= l.text_field :address2 %><br/>
City: <%= l.text_field :city %><br/>
State: <%= l.text_field :state %><br/>
Zip: <%= l.text_field :zip %><br/>
  <% end %>    

  Details: <%= form.text_area :description, :rows => 3 %>
  <%= submit_tag %>
<% end %>

In this example three models are used - Deal, Restaurant and Location.

The form is based on Deal (since I want the submit to go to Deal::create and I defined two fields_for blocks - one for Restaurant and Location.

The params.inspect output of the form data is:

Parameters: {”restaurant”=>{”name”=>”rest name”}, “deal”=>{”description”=>”Kids can eat whatever they want in under 3 minutes.”}, “commit”=>”Save changes”, “location”=>{”city”=>”some place”, “address1″=>”address one”, “zip”=>”12345″, “address2″=>”", “state”=>”NC”}}

Notice that restaurant, deal and location are all their own hash which can be passed to the new method on the respective models.

Worked like a champ.

Details on the schema is:

    create_table :locations do |t|
      t.column :address1, :string, :limit => 60
      t.column :address2, :string, :limit => 60
      t.column :city, :string, :limit => 40
      t.column :state, :string, :limit => 2
      t.column :zip, :string, :limit => 10
      t.column :phone, :string, :limit => 16
t.column :lng, :string, :limit => 30
      t.column :lat, :string, :limit => 30
      t.column :restaurant_id, :int
    end    

    create_table :restaurants do |t|
      t.column :name, :string, :limit => 60
    end    

    create_table :deals do |t|
      t.column :restaurant_id, :int
      t.column :location_id, :int
      t.column :description, :text
    end

And the models relationships:

class Deal < ActiveRecord::Base
  belongs_to :Restaurant
  belongs_to :Location
...
end    

class Location < ActiveRecord::Base
  belongs_to  :Restaurant
end    

class Restaurant < ActiveRecord::Base
  has_many  :Location
  has_many  :Deal
...
end

Permalink 11 Comments

11.21.07

Setting data on hidden fields during form submit using AJAX (Google maps geocode example)

Posted in Programming, Ruby, rails at 7:30 pm by Robert Horvick

Using heroku I’m working on a small rails app that needs to integrate with google maps. 

The Plan 

I want to do is allow a user to provide a location which I will store and later be able to render on a google map. 

Now, to use Google Maps to store location data you want to be using geocodes (i.e. longitude/latitude values).  There are a few reasons for this:

1. Ultimately adding a marker requires a geocode.
2. Google limits how many address-to-geocode conversions a site can perform (I’ve seen daily and weekly limits - they were 15,000 and 50,000 respectively [which doesn't add up - but whatever]).
3. Converting from address to geocode takes a second or longer.  Why make the viewer wait when the information only needs to be gathered once?

So the plan was to do the following:

1. Accept the data in a standard address form.
2. Contact the google web service for geocode creation
3. Cache the geocode with the location record in SQL
4. Subsequent requests would use the cached record

 Reality Bites 

The problem is that heroku does not allow outgoing connections so I can’t perform #2.  So I need to get the information from the caller without actually making them enter it.  What I came up with was:

1. Get the data from the user
2. Perform the geocode lookup in javascript on form submit
3. Note the geocodes on a pair of hidden form fields
4. Submit using javascript

Get the data from the user

Let’s get right to it.  This is the form code in the rails view (index.rhtml):

    <%= form_tag( {:controller => “location”, :action => “create”},

                  {’id’ => ‘frmAddress’, ‘method’ => ‘post’, ‘onSubmit’ => ’showAddress(this); return false;’} ) %>

     <p>

        <%= text_field_tag( :address ) %>

        <%= hidden_field_tag( :lat ) %>

        <%= hidden_field_tag( :lng ) %>

        <%= submit_tag(”Go”) %>

      </p>

      <div id=”map_canvas” style=”width: 500px; height: 300px”></div>

    <%= end_form_tag %>

Time for a little breakdown:

    <%= form_tag( {:controller => “location”, :action => “create”},
                  {’id’ => ‘frmAddress’, ‘method’ => ‘post’, ‘onSubmit’ => ’showAddress(this); return false;’} ) %>
 

We are submitting the form to the location controller’s “create” action.  We will submit using the post method.  The ‘id’ isn’t needed in this example - ignore it.  When the submit button is clicked we will call the showAddress javascript function and pass the form as the parameter.  So what’s with return false?

OMG - this is so important.  I can NOT stress this enough.

When submitting a form via javascript you must (MUST) have this return false.  If you don’t the form will submit twice - once via the javascript submit and once via the normal form post.

Say it with me - if javascript is submitting the form I will return false.  I will return false!!!

By the way - Agile Web Development With Rails had this information on 537 (form_remote_tag and remote_form_for documentation).       

        <%= hidden_field_tag( :lat ) %>
        <%= hidden_field_tag( :lng ) %>

These are the hidden fields for the longitude and latitude.

Everything else is plain-jane form stuff.

Lookup, record, submit

function showAddress(frm) {   
      if (geocoder) {
        geocoder.getLatLng(
          frm.address.value,
          function(point) {
            if (!point) {
              alert(address + ” not found”);
            } else {
              frm.lat.value = point.lat();
              frm.lng.value = point.lng();
              frm.submit();
            }
          }
        );
      }
 

This is mostly the stock google sample code except that I am storing away the latitude and longitude in the hidden form fields and I changed the function to take the form instead of the address value.

Also I submitted the form.

Now in the controller’s create method I can access the parameters like:

  def create
    @lat = params[:lat]
    @lng = params[:lng]
    @address = params[:address]
  end
 

And display them in the view like:

  Long: <%= @lng %><br/>
  Lat: <%= @lat %><br/>
  Address: <%= @address %><br/>
 

So it’s not the server-side solution I had hoped for but it gave me a chance to try something AJAX-ish and to make use of a more complex form submission approach.

Alright … now what have I done wrong?

Permalink 3 Comments