08.02.08

(Resolved) Error installing merb on Ubuntu (hint: install ruby-full)

Posted in Ruby, merb at 9:32 am by Robert Horvick

While installing merb I hit the following error:

Building native extensions.  This could take a while...
ERROR:  While executing gem ... (Gem::Installer::ExtensionBuildError)
ERROR: Failed to build gem native extension.

ruby extconf.rb install merb --include-dependencies
extconf.rb:1:in `require': no such file to load -- mkmf (LoadError)
from extconf.rb:1

Gem files will remain installed in /var/lib/gems/1.8/gems/hpricot-0.6.161 for inspection.
Results logged to /var/lib/gems/1.8/gems/hpricot-0.6.161/ext/fast_xs/gem_make.out

The issue is that compiling extensions requires ruby1.8-dev on Ubuntu.  I installed ruby-full instead using:

> sudo aptitude install ruby-full

And was able to get through the installation without issues (selecting the “ruby” version of each dependency that required a choice).

Permalink Comments

07.16.08

Fixing “undefined method `exitstatus’ for nil:NilClass”

Posted in Ruby at 3:12 pm by Robert Horvick

I’m trying to get some more practical experience with Ruby so I’ve started converting a few batch files that automated instrumenting some .NET assemblies for code coverage.  Basically it comes down to cleaning out some directories, running a child process in a loop, and pushing some binaries out to a sandbox test area.

It seemed like a good opportunity to also get some experience writing rake tasks.

The interface right now is:

coverage coverage:all    # Perform a complete code coverage pass from scratch
coverage coverage:clean  # Clean the coverage temp directory and sandbox
coverage coverage:cover  # Instrument the assemblies for code coverage
coverage coverage:merge  # Merge the test results
coverage coverage:run    # Run the test suite
coverage coverage:start  # Start the coverage service
coverage coverage:stop   # Stop the coverage service

During development I was doing something I thought was pretty simple - executing a process in a loop for each file in a FileList.

C:\ruby\coverage>rake coverage:cover
"E:\Program Files/Dev Suite}/instrument.exe" "foo.dll" /COVERAGE "/OUTPUTPATH:C:/output/coverage"
rake aborted!
undefined method `exitstatus' for nil:NilClass
C:/ruby/coverage/rakefile:36
(See full trace by running task with --trace)

See the problem?  I didn’t … (now it’s all I see!)

I re-ran rake with –trace and got a little more detail.  The top of the stack is:

rake aborted!
undefined method `exitstatus' for nil:NilClass
c:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.1/lib/rake.rb:899:in `sh' 

That makes sense … I’m calling sh.

In rake.rb line 899 is for this method:

def sh(*cmd, &block)
    options = (Hash === cmd.last) ? cmd.pop : {}
    unless block_given?
      show_command = cmd.join(" ")
      show_command = show_command[0,42] + “…”
      # TODO code application logic heref show_command.length > 45
      block = lambda { |ok, status|
        ok or fail “Command failed with status (#{status.exitstatus}): [#{show_command}]”
      }
    end
    rake_check_options options, :noop, :verbose
    rake_output_message cmd.join(” “) if options[:verbose]
    unless options[:noop]
      res = system(*cmd)
      block.call(res, $?)
    end
  end

 
This is where I spent 5 minutes not making the mandatory mental leap. That’s how long it took to realize that

block = lambda { |ok, status|

is creating a block object which will be called later at the block.call.

So status.exitstatus is “$?” and “$?” is the exit status of the child process … and it’s NilClass so that must mean the process never executed … which means I need to start with the most obvious culprit: the command line (specifically the executable path).

So go back to the origional error message … now do you see it? (if you don’t - there is a “}” after “Dev Suite”

C:\ruby\coverage>rake coverage:cover
"E:\Program Files/Dev Suite}/instrument.exe" "foo.dll" /COVERAGE "/OUTPUTPATH:C:/output/coverage"
rake aborted!
undefined method `exitstatus' for nil:NilClass
C:/ruby/coverage/rakefile:36
(See full trace by running task with --trace)

It sucks that rake is not resiliant to this. So I updated my local copy to check for status.nil? like so:

      block = lambda { |ok, status|
        ok or fail "Command failed with status (#{status.nil? ? "nil" : status.exitstatus}): [#{show_command}]”
      }

Summary …

If you are running a rake and get the message “undefined method `exitstatus’ for nil:NilClass” it (probably) means that the program it was trying to run was not found.

Permalink 2 Comments

07.08.08

Accept The Facts: Your Test Suite is a Roach Motel

Posted in Ruby, rspec, testing at 10:13 am by Robert Horvick

Adam @ Heroku had a recent blog post where he discussed his recent experience with a bug in rush.

“A bug is a situation where the code has been specified to behave one way, and instead it behaves another. It does not include things that users or developers may like the software to do, but that it is not specified to do currently.”

It’s not clear to me if Adam is arguing then that he hit a bug in his product code or spec.

I would argue the bug was not in the product code.  Using Adam’s words - since the failing line had no spec it had “no functionality that you can rely on.”  This is what he observed.

The code was correct as per the spec - the bug was the missing spec.

On that specific line of code Adam was unintentionally practicing HDD (Hope Driven Development).

Adam makes good points about the benefits of BDD/TDD and the importance of using tools like rspec.

Reading Adam’s experience brought out a rant I let fly once in a while.  This rant isn’t about Adam or his recent experience.

It’s about a trend in software development that has been grinding on me for years.

We distrust product code while blindly trusting test code.

That is a huge problem.

The “blindly trusting test code” part.  Go ahead and distrust product code all day long.  That’s what it’s there for.

Of the two code bases, product and test, only one of them has any test coverage- and yet it’s the other that we treat as golden.

Using BDD/TDD this makes no sense.  We know (in the “deep in my heart I truly believe” sense) that BDD/TDD reduces the number of bugs in our product code.  It’s a core belief.   We wouldn’t do it otherwise. 

I have a few other core beliefs:

• You can’t find bugs with missing specs
  • Stumbling across a bug is dumb luck (See: Hope Driven Development).
  • What is the process for identifying missing specs? Does using BDD/TDD suddenly make you smarter or more creative? Are you able to divine user scenarios you didn’t know of before? Maybe a few - but generally we all still live within our same limitations of vision, insight and experience that we had before.
• You won’t find the right bugs with wrong or broken specs.
  • “Duh”
• BDD/TDD practitioners have more test bugs than product bugs
  • Assertion: BDD/TDD advocates write twice as much test code as product code
  • Assertion: Everyone’s “defects per LOC” count remains individually consistent regardless of whether they are writing product or test code.
• 100% code coverage doesn’t mean that much
  • Without code coverage you know you have missing tests. But just because the code succeeds in one scenario with one set of data in a single story doesn’t mean anything beyond exactly that. 100% coverage is not the goal - it’s the starting point.

Your test suite is a roach motel.  It’s filled with bugs and there isn’t much any of us can do about it until we change the way we approach specs.

Change is good!

When we move to BDD/TDD we are shifting a portion of our HDD from the product code to the test code.  It turns out that in doing so we also shift a good portion of the bugs from the product to the tests as well - that is great!  Keep on doing that.

But we need to stop putting so much faith in our specs and start thinking about how we can improve them. 

We need to take a hard look at where specs come from and find a better way. 

User stories and scenarios, customer requirements and our imaginations are good - but they are bound by our own personal (or pair, or team) limitations.

There’s Patterns in Them-Thar Hills

Take debit/credit as an example.  This is a very common pattern used in almost every non-trivial system.

The RSpec homepage uses it as their Story example (banking domain).  Sites like Digg use it for digging/burying.  Shopping carts use it to add and remove items from carts.

Any time you want to allow a balanced change (accounts stay balanced throughout the transfers, burying and digging have balanced effects and prevent duplication, etc) you are in this pattern.

If 10 people sat down they would come up with 10 different set of closely related specs.  Some parts would be domain specific to banking or social networking - but some would be straight-up debit/credit logic. 

Some of the specs would be wrong - forget about those.

Some of the debit/credit specs would be good.  Some would be very good.  They would not just check for happy-path balancing but also invalid sources and targets (transferring to an invalid account or voting on a non-public post), they would spec failure rollback scenarios (transactional behaviors), they would check auditing.  There would be checks for a single user in multiple sessions performing simultaneous operations.

The union of the good is approaching the ideal debit/credit spec pattern.

In the process of approaching the ideal the spec would become opinionated about your debit/credit design (plugins, anyone?)

Let’s capture that ideal.

Let’s tell people about it.

Let’s make it easy to discover and use.

Or not. 

Permalink Comments

05.31.08

Installing Merb on Windows is a lot easier!

Posted in Ruby, merb, windows at 7:32 pm by Robert Horvick

It’s been a while since I’ve triedthe Merb-on-Windows-from-scratch so I started from and found that it has gottens a lot easier since the last time I tried.

It’s now a two step process:

1. Install the Ruby One-Click Installer
2. Follow the instructions on at http://merbivore.com/index.html

But for those of you who want the blow-by-blow account…

1. Download Ruby One-Click Installer[ruby186-26.exe]
2. Install - select all defaults.
3. Run “RubyGems Package Manager”
   1. (Start -> All Programs -> Ruby-186-26 -> RubyGems -> RubyGems Package Manager)
   2. this brings up a command window
4. - Execute “gem install merb –include-dependencies”
   1. select “1. ParseTree 2.1.1 (i386-mswin32)”
   2. select “1. mongrel 1.1.5 (x86-mswin32-60)
   3. Wait for 10 minutes or so while the rdoc is installed…
5. Execute: merb-gen app  my_application
6. Execute: cd my_application
7. Execute: merb
   1. wait a few moments for it to start up…
8. open your browser of choice to http://localhost:4000

Success!

I want to extend a major kudos to the folks working on Merb for improving the Merb-on-Windows story so significantly.

Permalink 6 Comments

12.27.07

Resolved: SQLite3::SQLException: SQL logic error or missing database

Posted in Ruby, sqlite3 at 11:00 am by Robert Horvick

After doing some cleanup I reset my Sqlite3 databases and tried to run tests to make sure that everything was still ok. The command I ran was:

$ rake test

and the output from each test was a failure with the message:

34) Error:
test_should_update_user(UsersControllerTest):
SQLite3::SQLException: SQL logic error or missing database

So I did the following:

1. Check that test.sqlite3 exists.
2. Rebuild test.sqlite3 using “rake db:reset”
3. retry … still failed.
4. Try loading fixtures using “rake db:fixtures:load”

C:\InstantRails\rails_apps\atompublog>rake db:fixtures:load
(in C:/InstantRails/rails_apps/atompublog)
rake aborted!
SQLite3::SQLException: no such table: atom_entries: DELETE FROM atom_entries WHERE 1=1

(See full trace by running task with --trace)

Ah ha!

That table shouldn’t exist (and it doesn’t…) so there should not be a fixture loading into it.

I checked my fixtures and there was one named “atom_entries.yml”. After deleting that file the fixtures loaded, the tests ran and I was confident that my cleanup changes did not break anything.

A quick google search shows that a lot of people have hit this problem so hopefully this post helps some of them track down one of the potential causes.

Permalink 10 Comments

12.26.07

Realizing AtomPub entries from XML using ROXML

Posted in Ruby, blog, rails at 11:00 am by Robert Horvick

I have done some digging and there does not seem to be any plugins or gems (or even active projects on RubyForge) that are providing support for an AtomPub server. I did find a sample based on Camping but it was not really what I was looking for. It supports a subset of AtomPub and not in a way that is really friendly for what need.

So I set out on my own.

An atom pub entry can be as simple as this:

<atom:entry xmlns="http://www.w3.org/2005/Atom">
  <atom:title>ATOM Post Test</atom:title>
  <atom:id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</atom:id>
  <atom:content>Some text.</atom:content>
</atom:entry>

Or quite a bit more complex (lifted from http://tools.ietf.org/html/rfc4287):

<entry>
  <title>Atom draft-07 snapshot</title>
  <link rel="alternate" type="text/html"
    href="http://example.org/2005/04/02/atom"/>
  <link rel="enclosure" type="audio/mpeg" length="1337"
    href="http://example.org/audio/ph34r_my_podcast.mp3"/>
  <id>tag:example.org,2003:3.2397</id>
  <updated>2005-07-31T12:29:29Z</updated>
  <published>2003-12-13T08:29:29-04:00</published>
  <author>
    <name>Mark Pilgrim</name>
    <uri>http://example.org/</uri>
    <email>f8dy@example.com</email>
  </author>
  <contributor>
    <name>Sam Ruby</name>
  </contributor>
  <contributor>
    <name>Joe Gregorio</name>
  </contributor>
  <content type="xhtml" xml:lang="en"
    xml:base="http://diveintomark.org/">
      <div xmlns="http://www.w3.org/1999/xhtml">
        <p><i>[Update: The Atom draft is finished.]</i></p>
      </div>
  </content>
</entry>

I started with just caring about title and content. Everything else I could infer from authentication (author, anyway).

With this thought, I created some simple xpath queries to find the title and content. That was working out. I could get the atom response created and the post serialized to the database. So with just a few lines of code (beyond the scaffolding) I was able to use Live Writer to post via atompub to my blog engine.

But I wasn’t happy with just winking away the atompub protocol in favor of getting done quickly.

So I sat down and really started reading the atom publishing spec (and the a related atom rfc). I have a lot of work to do.

I started using XmlSimple to parse the XML. That was ok, except some things don’t map well to hashes and it did not get me closer to serializing atom entries to xml (for the feed later on).

So I moved down to REXML and that was working ok. But I still wasn’t happy. I felt like I was writing too much monkey code and not doing making progress on my real goal.

Tinally I found ROXML - and now I’m happy. I’ll cut to the chase and just show the code:

require 'rubygems'
require 'roxml'

class AtomBase
  include ROXML

  def to_s
    self.to_xml
  end
end

class AtomAuthor < AtomBase
  xml_name "author"
  xml_text :name
  xml_text :uri
  xml_text :email
end

class AtomContributor < AtomBase
  xml_name "contributor"
  xml_text :name
  xml_text :uri
  xml_text :email
end

class AtomLink < AtomBase
  xml_name "link"
  xml_attribute :rel
  xml_attribute :type
  xml_attribute :href
  xml_attribute :length
end

class AtomContent < AtomBase
  xml_name "content"
  xml_attribute :type
  xml_attribute :lang
  xml_attribute :base
  xml_text :text, nil, ROXML::TEXT_CONTENT
end

class AtomSummary < AtomBase
  xml_name "summary"
  xml_attribute :type
  xml_attribute :lang
  xml_attribute :base
  xml_text :text, nil, ROXML::TEXT_CONTENT
end

class AtomEntry < AtomBase
  xml_name "entry"
  xml_text :title
  xml_object :link, AtomLink, ROXML::TAG_ARRAY
  xml_text :id
  xml_object :summary, AtomSummary
  xml_text :updated
  xml_text :published
  xml_object :author, AtomAuthor
  xml_object :content, AtomContent
  xml_object :contributor, AtomContributor, ROXML::TAG_ARRAY
end

The major gap is that this does not support xhtml content and summary fields. Those fields contain embedded XML and I have not figured out how to get ROXML to stop navigating the tree and make the object property return the inner xml.

I did try hacking ROXML to support an XML_CONTENT tag. It solved about 70% of the problem but it was not working exactly as I wanted after about a half hour and I felt like I was shaving a yak. I can come back to that later on. For now that the atom bits were are neatly wrapped up behind a class with a known bug. I’m ok with that.

I haven’t reviewed the atom syndication spec to make sure I’m doing everything right but it’s working with all the samples I’ve thrown at it (including the samples from the spec) and Live Writer seems to be having a field day with it. So that’s good.

Anyway - that’s where I’m heading.

I have the feeling that I’m reinventing the wheel. I hope not poorly.

Permalink 5 Comments

12.25.07

Just what the world doesn’t need … another blog engine.

Posted in Ruby, blog, rails at 9:36 pm by Robert Horvick

Nuby on Rails recently asserted that “every beginning Rails developer should write their own blog software. It’s a great learning experience and you can try things that aren’t possible with just an app running on localhost.”

I can buy into this.  It’s the canonical Rails sample app.  I’m new to Ruby and Rails.  Maybe I should do this.  And I don’t mean a little toy that I do in 15 minutes and then throw away.  I mean the blogging platform I use for my primary blog.

So I sat down and tried to list out what I would want my blog to look like …

1. I don’t want to spend time writing code for a text editor to write blog posts
2. I want people to be able to subscribe to my content
3. I want caching (not that I plan to get dugg - but it’s good learning)
4. I want to be able to easily customize the look/feel
5. I want to support tagging and providing post views based on those tags
6. I want to support uploading media files (images, etc)
7. I want to be able to import this blog into it.

So I spent some time researching those issues and this is what I’ve come up with:

1. I will post via atom publishing -there are already blog tool that support this (Microsoft Live Writer, for example - I’m sure there are others).
2. I will provide atom feeds.
3. Rails has lots of caching examples.  I’ll start with file system based caching and move upwards from there.
4. I won’t provide full theme support but I’ll have a generic post format and include user defined stylesheets (and possible javascript).
5. acts_as_taggable until shown why not.
6. atompub can do this.
7. I think I can use atom feeds to do this.

Also …

As mentioned I’m hosting with SliceHost.com on a 256 slice so I need to keep it trim.  I may go to production on Sqlite3 (if I do the caching right this shouldn’t be a problem - and that would keep 4-10% of my limited RAM free).

Also my goal is to write as little new code as possible.  Plugins and gems whenever possible.

I figure it will take about a month to have something I can show to the world and not worry about hiding my face.

Over the next few weeks I’ll be blogging about what I learn that seems interesting.  And plenty that isn’t, I’m sure.

Permalink Comments

12.19.07

Merb on Cygwin: “No such file or directory - /tmp/mysql.sock” (resolved)

Posted in Ruby, merb at 8:56 pm by Robert Horvick

When trying to perform a migration (rake db:migrate) I hit the following error:

$ rake db:migrate
(in /home/Robert/ruby/merb_apps/blog)
svn: '.' is not a working copy
Connecting to database...
Thu, 20 Dec 2007 01:48:09 GMT: loading gem 'merb_activerecord' from config/dependencies.rb:16 ...
rake aborted!
No such file or directory - /tmp/mysql.sock

(See full trace by running task with --trace)

After a few minutes of googling and some trial/error I discovered that changing “localhost” in the database.yml file to the loop back address 127.0.0.1 resolved the issue. Apparently when you specify an IP address the :socket: value is not needed. Since the default merb database.yml content includes the line:

:socket: /tmp/mysql.sock

This is something that other Windows users may need to modify.

This is my updated config/database.yml file:

:development: &defaults
:adapter: mysql
:database: blog_development
:username: root
:password:
:host: 127.0.0.1

:test:
<<: *defaults
:database: blog_test

:production:
<<: *defaults
:database: blog_production

Permalink 2 Comments

12.17.07

Manually downloading and installing gems.

Posted in Ruby, merb at 11:57 pm by Robert Horvick

I’m getting ready to dive deeper into merb and wanted to install merb_activerecord. Unfortunately when I tried this I hit the following error:

$ gem install merb_activerecord
Updating metadata for 32 gems from http://gems.rubyforge.org
................................
complete
Building native extensions.  This could take a while...
ERROR:  While executing gem ... (Gem::RemoteFetcher::FetchError)
OpenURI::HTTPError: 404 Not Found reading http://gems.rubyforge.org/gems/activesupport-2.0.2-.gem

So I tried “gem update activesupport” and hit the same error.

I went out to the ActiveSupport RubyForge page found that 2.0.2 was released today. So I went to http://gems.rubyforge.org (which redirected to http://gems.rubyforge.vm.bytemark.co.uk/gems) and the latest version available in 2.0.1. So I assume this is just an issue of needing to wait for data to shuffle around.

But I’m impatient.

So I downloaded the gem file from http://rubyforge.org/frs/download.php/29344/activesupport-2.0.2.gem and installed as:

$ gem install activesupport-2.0.2.gem
Successfully installed activesupport-2.0.2
1 gem installed
Installing ri documentation for activesupport-2.0.2...
Installing RDoc documentation for activesupport-2.0.2...

When that was done I was able to successfully install merb_activerecord as follows:

$ gem install merb_activerecord
Successfully installed merb_activerecord-0.4.3
1 gem installed
Installing ri documentation for merb_activerecord-0.4.3...
Installing RDoc documentation for merb_activerecord-0.4.3...

The lessons here are that just-released gems might not be available via “gem install” even though they are a listed dependency of other gems and that manually downloading and installing gems is trivial.

Permalink 2 Comments

12.13.07

Getting Merb working on Cygwin requires a Rakefile patch

Posted in Ruby, cygwin, merb at 3:39 pm by Robert Horvick

When getting merb working on Windows I provided a patch to the merb folks that resolved a Rakefile compat problem (sudo was being used even when on Windows - where it does not exist).

I’m trying various configurations on Windows and found that in cygwin the platform name does not include “win32″ so the filter no longer worked (and cygwin does not include sudo).

I filed a ticket http://merb.devjavu.com/ticket/361 and attached the patch. It’s very simple:

$ cat cygwin.diffIndex: Rakefile
===================================================================
--- Rakefile    (revision 1087)
+++ Rakefile    (working copy)
@@ -18,7 +18,7 @@

CLEAN.include ['**/.*.sw?', '*.gem', '.config']

-windows = (PLATFORM =~ /win32/)
+windows = (PLATFORM =~ /win32|cygwin/)

SUDO = windows ? “” : “sudo”

And then it all goes well …

Robert@HORVICKLAPTOP ~/merbsrc
$ rake install
(in /home/Robert/merbsrc)
rake package
(in /home/Robert/merbsrc)
  Successfully built RubyGem
  Name: merb
  Version: 0.4.2
  File: merb-0.4.2.gem
 gem install pkg/merb-0.4.2 --no-rdoc --no-ri
Successfully installed merb-0.4.2
1 gem installed

Permalink 2 Comments