libxml2 for Nokogiri in JRuby

I’ve been playing with Nokogiri on JRuby 1.3.0 and this message gets displayed when I require ‘nokogiri’:


macbookpro:~ douglas$ jirb
irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'nokogiri'
HI. You're using libxml2 version 2.6.16 which is over 4 years old and has
plenty of bugs. We suggest that for maximum HTML/XML parsing pleasure, you
upgrade your version of libxml2 and re-install nokogiri. If you like using
libxml2 version 2.6.16, but don't like this warning, please define the constant
I_KNOW_I_AM_USING_AN_OLD_AND_BUGGY_VERSION_OF_LIBXML2 before requring nokogiri.
=> true

That can’t be, I’ve just upgraded libxml2 and libxslt in my MacPorts installation. Digging further, I don’t receive the message when doing the same thing on MRI. A quick look with lsof reveals that the shared libraries are being loaded from /usr/lib instead of /opt/local/lib (MacPorts is installed into /opt/local).


lsof -c java
----omitted----
java 390 douglas txt REG 14,2 290736 898139 /usr/lib/libexslt.0.dylib
----omitted----

lsof -c ruby
----omitted----
ruby 865 douglas txt REG 14,2 72156 1871557 /opt/local/lib/libexslt.0.8.13.dylib
ruby 865 douglas txt REG 14,2 218688 1871571 /opt/local/lib/libxslt.1.1.24.dylib
ruby 865 douglas txt REG 14,2 1251948 1866501 /opt/local/lib/libxml2.2.7.3.dylib
----omitted----

Nokogiri uses Ruby FFI to dynamically load native C code and FFI makes use of dlopen to do the actual loading of dynamic libraries. On OSX, dlopen searches for files specified by a couple of environment variables (LD_LIBRARY_PATH, DYLD_LIBRARY_PATH, DYLD_FALLBACK_LIBRARY_PATH), and the current working directory.

Setting LD_LIBRARY_PATH to /opt/local/lib worked for me. There may be differences in the environment variables used for dlopen on different platforms, so a look at the MAN pages would be a good idea if things don’t seem to work.

Posted in JRuby | 1 Comment

Using soap4r with .NET web services authenticated with WSSE

Recently I had to consume a .NET web service that uses WSSE based authentication. Thanks to the intarweb, I was able to get started with soap4r and insert WSSE authentication headers into my SOAP requests.The samples available on the tracker are also quite comprehensive, other than being light on explanation.

I did encounter an error when using the automatically generated client application:

ruby WSDLService_PortClient.rb
./defaultMappingRegistry.rb:5: uninitialized constant SOAP::Mapping::EncodedRegistry (NameError) from ./defaultDriver.rb:2:in `require'
from ./defaultDriver.rb:2
from ./WSDLService_PortClient.rb:2:in `require'
from ./WSDLService_PortClient
.rb:2

It appears that the soap library provided with Ruby 1.8.6 is being used instead of the soap4r gem. The version bundled with Ruby doesn’t have some classes referenced by the generated stubs (from wsdl2ruby) as well as a bunch of other functionality.

The solution is simple (after knowing what’s wrong of course): explicitly reference the gem so that Ruby knows which one to use.

require 'rubygems'
gem 'soap4r'

Just insert this snippet before the code that references the missing classes and you’re good to go.

Posted in Ruby | Comments Off

currency-convertor gem on github.com

I took a moment to extract some code that’s being used at work for getting exchange rates between 2 currencies into a simple gem called bianster-currency-convertor at github.com.

If you haven’t already added the gem repository at github.com, do take a look at the concise instructions before trying to install the gem.

Sample Usage

>> require 'currency_convertor'
=> true
>> CurrencyConvertor.lookup('USD', 'SGD')
=> 1.4165

Posted in Ruby | Comments Off

Painless Rails Deployment

Remember all that joy coding with Rails when you were a wee lad? Things seemed so blissful then, metaprogramming voodoo and that complex object relational mapping stuff were all taken care of behind the scenes.

Alas, whence it doth time to unleash your brilliant social networking app onto the unsuspecting public, your Ruby/Rails world came crashing down faster than FSJ can order a chai latte.

FastCGI, SCGI, Mongrel, Rack, Thin, Pound, Nginx, Apache.

Doing a virgin Rails application deployment would have been a trial by fire where you’ll trawl the Ruby and Rails forums looking for advice on which webserver is best suited to serve your future web application superstar or pray to the One and just deploy on faith.

You’ll settle for Mongrel or Rack if you were lucky enough to come across people who have been there before. Otherwise, be prepared to be regaled with war stories about how some web hosts *COUGH* Dreamhost *COUGH* arbitrarily shutdown your Rails processes for consuming a tad too much resources.

Wouldn’t it be nice if I could just do like what I did for PHP applications? Just upload the files onto the webserver and it just works(tm)?

Now you can! Well, technically only the people at Phusion can at this moment. Lai Hongli breaks the news about modrails, complete with requisite cool screencast.

I think this is a really cool development that makes life a whole lot simpler for people who just want to focus their energies on actual development rather than have to worry about how to configure the deployment environment properly. Even if this is Apache2 only now, it should be possible to port it over to other servers like Nginx, licence restrictions notwithholding.

Posted in Rails | 1 Comment

Specifying raising of errors in RSpec

RSpec allows a developer to specify that an error is raised within a block with the raise_error method. It’s a nice expressive way of saying that your code should fail when it needs to.

But my tiny brain has often been confused with using it at times, more so when the error class requires parameters for instantiation and when used in conjunction with the and_raise method on a stub or a mock

Consider the snippet below where my Widget depends on Thingamajig to do its funky thing in order to run. But Thingamajig is rigged to explode in a mass of funkyness and make Widget all useless.

[code]
describe Widget do
it "should re-raise errors as a Widget::UnrecoverableError" do
# expectations
thingamajig = stub(Thingamajig)
thingamajig.should_receive(:do_funky_thing).and_raise(Thingamajig::FunkyExplosion.new('The funky thang exploded yo'))

# our SUT
widget = Widget.new(thingamajig)

#verification
lambda { widget.run }.should raise_error(Widget::UnrecoverableError, 'The funky thang exploded yo')
end
end
[/code]

Do you notice the inconsistency between the way errors are declared in the expectation and the actual verification?

[code]
# expectations
thingamajig.should_receive(:do_funky_thing).and_raise(Thingamajig::FunkyThingExplosion.new('The funky thang exploded yo'))

#verification
lambda { widget.run }.should raise_error(Widget::UnrecoverableError, 'The funky thang exploded yo')
[/code]

The expectation on the stub, ‘thingmajig’ needs the exception instantiated first while verification requires the class name and parameters used to instantiate the error instance.

And no, doing it like this doesn’t work as expected:

[code]
lambda { widget.run }.should raise_error(Widget::UnrecoverableError.new('The funky thang exploded yo'))
[/code]

It’s an unfortunate impedance mismatch that might be caused due to the way Ruby handles the raising of errors.

Posted in Behaviour Driven Development, Ruby | Comments Off

Using cruisecontrol.rb with repositories without anonymous access

I decided to use cruisecontrol.rb for continuous integration of an application that I’m working on at Bezurk. So I downloaded the 1.2.1 release from RubyForge and proceeded to add the project repository to the local installation of cruisecontrol.rb.

douglas@macbookpro:~$ ./cruise add MyProject --url http://path.to/repository --username 'douglas' --password 'guessable'
douglas@macbookpro:~$ ./cruise build MyProject
Builder for project 'MyProject' started
Logging to: /Users/douglas/Development/Ruby/cruisecontrolrb-1.2.1/log/MyProject_builder.log
Build loop failed
BuilderError: svn: PROPFIND request failed on '/svn/my_project/trunk'
./script/../config/../app/models/subversion.rb:98:in `execute_in_local_copy'
./script/../config/../lib/command_line.rb:86:in `call'
./script/../config/../lib/command_line.rb:86:in `e'
./script/../config/../lib/command_line.rb:84:in `popen'
./script/../config/../lib/command_line.rb:84:in `e'
./script/../config/../lib/command_line.rb:71:in `execute'
./script/../config/../lib/command_line.rb:70:in `chdir'
./script/../config/../lib/command_line.rb:70:in `execute'
./script/../config/../app/models/subversion.rb:89:in `execute_in_local_copy'
./script/../config/../app/models/subversion.rb:85:in `chdir'
./script/../config/../app/models/subversion.rb:85:in `execute_in_local_copy'
./script/../config/../app/models/subversion.rb:44:in `latest_revision'
./script/../config/../app/models/project.rb:228:in `new_revisions'
./script/../config/../app/models/change_in_source_control_trigger.rb:8:in `revisions_to_build'
./script/../config/../vendor/rails/actionpack/lib/../../activesupport/lib/active_support/core_ext/symbol.rb:10:in `__send__'
./script/../config/../vendor/rails/actionpack/lib/../../activesupport/lib/active_support/core_ext/symbol.rb:10:in `to_proc'
./script/../config/../app/models/project.rb:223:in `collect'
./script/../config/../app/models/project.rb:223:in `revisions_to_build'
./script/../config/../app/models/project.rb:202:in `build_if_necessary'
./script/../config/../app/models/polling_scheduler.rb:13:in `run'
./script/builder:79
./script/builder:78:in `catch'
./script/builder:78
./cruise:14:in `load'
./cruise:14:in `builder'
./cruise:68:in `send'
./cruise:68
/opt/local/lib/ruby/1.8/fileutils.rb:121:in `chdir'
/opt/local/lib/ruby/1.8/fileutils.rb:121:in `cd'
./cruise:67

Hmm, what’s with the svn: PROPFIND error? Looking at the stracktrace doesn’t tell me alot about what’s going wrong here, let’s try logging errors to the console.

--SNIP--
douglas$ svn --non-interactive info --xml
douglas$ svn --non-interactive log --revision HEAD:20 --verbose --xml
svn: PROPFIND request failed on '/repository/trunk'
svn: PROPFIND of '/repository/trunk': authorization failed (http://svnhost.com)
--SNIP--

It happens that my repository does not have anonymous access and requires a subversion user account to do anything useful. So it should be obvious that cruisecontrol.rb is trying to get log info from the repository but subversion is quitting with authentication errors because no user credentials are being supplied.

I need to have cruisecontrol.rb make use of the –username and –password options when making queries to the repository when I give it the credentials for access.

My first stop is the app/models/subversion.rb. Only the checkout method uses the username and password instance variables. Subversion should only include the –username and –password options when executing svn commands when both the username and password instance variables are present.

[ruby]
test/unit/subversion_test.rb

def test_svn_command_uses_user_password_when_provided
	svn = Subversion.new(:username => 'jer', :password => "crap")

	svn.expects(:info).with(dummy_project).returns(Subversion::Info.new(10, 10))
	svn.expects(:execute).with(["svn", "--non-interactive", "log", "--revision", "HEAD:10", "--verbose", "--xml",
								"--username", "jer", "--password", "crap"],
								{:stderr => './svn.err'}).yields(StringIO.new(LOG_ENTRY))

	svn.latest_revision(dummy_project)
end

app/models/subversion.rb

def checkout(target_directory, revision = nil, stdout = $stdout)
	@url or raise 'URL not specified'

	options = [@url, target_directory]
	options < < "--revision" << revision_number(revision) if revision

	# need to read from command output, because otherwise tests break
	execute(svn('co', options)) do |io|
	begin
		while line = io.gets
			stdout.puts line
		end
		rescue EOFError
		end
	end
end

def svn(operation, *options)
	command = ["svn"]
	command << "--non-interactive" unless @interactive
	command << operation
	command += options.compact.flatten
	command += ['--username', @username, '--password', @password] if @username and @password
	command
end
[/ruby]

The username and password would then be injected into the project's Subversion instance in the cruise_config.rb file for each project.

[ruby]
Project.configure do |project|
    project.source_control.username = 'douglas'
    project.source_control.password = 'guessable'
end
[/ruby]

I’ve submitted a ticket along with a patch for this on cruisecontrol.rb’s tracker. Keep a lookout for it if you happen to encounter the same problem.

Posted in Agile, Ruby | 3 Comments

Serializing custom Ruby classes with YAML

I needed to be able to save an array of POROs (plain old Ruby objects) to the database on a Rails project that I’m currently working on. That should be easy, right?

This ought to do the job, yes?:
class MyModel < ActiveRecord::Base
serialize attribute_name
end

However, this didn’t quite work as advertised, the objects being returned were typed as YAML::Object instead of the actual class being serialized. Turns out that YAML is unable to find the reference to the actual class and so its falling back to using YAML::Object as the generic class type for deserialised objects. The answer to this: Rails ticket #7537.

Take a look at the YAML documentation for all the gory details.

Posted in Ruby | Comments Off

Writing tests first is…?

What would you do if your co-developers say “Writing tests first is too hard/slows me down/waste of time”?

  1. Evangelise test first development by surreptitiously providing publications on the topic
  2. Practice it on projects and try to test infect other people.
  3. Grow weary of going at it alone and move on.
Posted in Software Development | Comments Off

Behaviour Driven Development != Testing

Testing: Executing a program with the specific intent of uncovering errors.

Software Engineering: A Practitioner’s Approach – Roger S Pressman

The definition of testing by Pressman states that the purpose of performing software testing is to detect errors in a program. This encompasses a wide range of techniques such as black box/white box testing, basis path testing, fault based testing and at a more thorough level, control structure testing.

So where do the specifications in Behaviour Driven Development (BDD) fit into the picture? In this respect, BDD isn’t about testing at all. We write specifications to say that the software exhibits a specific behaviour when its in a certain state. The specifications serve to reinforce the notion that the program is working as expected under known conditions.

Having 100% coverage for code certainly does not mean that a program is free of errors, there are still edge cases that may be too difficult or complex to replicate with an automated test suite. Traditional QA testing is still very much relevant to software projects with BDD employed religiously throughout development.

This has been discussed at length before but it bears repeating: BDD is a design technique that gives you executable documentation of what functions the software is expected to provide.

Posted in Behaviour Driven Development | Comments Off

Visualising Log Activity with Ruby and OpenGL with glTail

Here’s an interesting combination of Ruby and OpenGL, glTail lets you visualise log activity in realtime. Check out the screencasts on the project page!

One thing to note at the time of writing, it assumes that the command for getting incoming log data, i.e “tail -f” is non blocking. So if you need to use “sudo tail -f”, the application won’t work properly for this situation. I might give it a shot at adding sudo support when I have some time to spare.

The quick spike I did for this suggests that I’ll need to use a shell to respond to password challenges. Incidentally, Capistrano already has this figured out, so i’ll be a good place to look for clues.

Posted in Ruby | Comments Off