Tuesday, February 12, 2008

ibm_db and Ruby/Informix interface comparison

Antonio Cangiano, a Software Engineer at IBM, recently posted in his blog an Essential guide to the Ruby driver for DB2, which included some samples of IBM's ibm_db Ruby driver. This driver supports not only DB2, but several IBM databases, Informix included.

As Antonio states in his guide, it was not meant to be thorough (that's what the API reference is for), but it certainly helped to get an idea of how ibm_db looks at work.

Antonio explains in a comment that ibm_db's interface was taken from PHP's DB2 driver, and that same interface was used for the IBM's Ruby and Python drivers.

That helps to explain why the code examples don't look Ruby-ish at all.

Here's one of the simplest examples:



require 'rubygems'
require 'ibm_db'

if conn = IBM_DB::connect("sample", "db2inst1", "mypassword")
sql = "SELECT * FROM EMPLOYEE"
begin
if stmt = IBM_DB::exec(conn, sql)
while row = IBM_DB::fetch_assoc(stmt)
puts "#{row['FIRSTNME']} #{row['LASTNAME']}: #{row['EMPNO']}"
end
IBM_DB::free_result(stmt)
else
puts "Statement execution failed: #{IBM_DB::stmt_errormsg}"
end
ensure
IBM_DB::close(conn)
end
else
puts "Connection failed: #{IBM_DB::conn_errormsg}"
end


And here's the equivalent, written with Ruby/Informix:



require 'informix'

Informix.connect("sample", "db2inst1", "mypassword") do |conn|
sql = "SELECT * FROM EMPLOYEE"
conn.cursor(sql) do |cur|
cur.open
cur.each_hash do |row|
puts "#{row['FIRSTNME']} #{row['LASTNAME']}: #{row['EMPNO']}"
end
end
end


The difference is obvious: blocks.

Blocks are one of the nicest features of Ruby. Matz, creator of Ruby, gave an attractive look to an old concept, which made it easy and desirable to use.

You can see blocks everywhere in Ruby as iterations, for resource management, event handling or as high order functions.

Blocks are a distinctive of Ruby. But still no blocks in ibm_db.

Fortunately Informix users have an alternative.




UPDATE: It will be even more succinct using the new features that will be included in the next release but that you can already try grabbing the code from the CVS tree.




require 'informix'

Informix.connect("sample", "db2inst1", "mypassword") do |conn|
sql = "SELECT * FROM EMPLOYEE"
conn.each_hash(sql) do |row|
puts "#{row['FIRSTNME']} #{row['LASTNAME']}: #{row['EMPNO']}"
end
end

Monday, February 11, 2008

Informix on Rails demo at IBM developerWorks

Some weeks ago, Akmal B. Chaudhri, Senior IT Specialist at IBM, completed a set of demos about working with Informix on Windows.

In part 6 of this set you can find an excellent guide, step by step, from scratch, about how to get Ruby on Rails working with Informix, using Ruby/Informix and Rails/Informix.

Thanks to Akmal for posting this demo and for giving (Ruby|Rails)/Informix a try.

Wednesday, December 05, 2007

Informix adapters for ActiveRecord and ruby-sequel

I have updated Rails/Informix to implement DECIMAL support and fix a pair of bugs that prevented the generation of db/schema.rb. I'm happy to say that on its 1.1.0 release, this adapter is mostly complete - indexes is still missing - and robust.

By the way, even though Rails 2.0 is out already, the ActiveRecord's Informix adapter gem was not published. I have tried to bring this issue to Jeremy Kemper's and Ruby on Rails Talk's attention without success yet. So, in the meantime, just donwload and manually install the distribution from Rails/Informix's project site.

Speaking of O/RM adapters, some weeks ago I found another Ruby O/RM called ruby-sequel which looks very nice to me. It has a nice interface that lets you write queries using Ruby and has just implemented an Informix adapter based on Ruby/Informix. You can take a look at some examples here.

Wednesday, January 24, 2007

One year later

Almost a year has passed since my last post and a lot of things have happened.

I didn't quit my job after all and that meant that I could still work on the Ruby extension and the Ruby on Rails adapter for Informix, which have become richer in features and more robust.


Ruby/Informix


Latest Ruby/Informix version is 0.6.2, and has gained the following features:

  • Use of BigDecimal for MONEY/DECIMAL columns
  • Resource management by means of blocks for statements, cursors and connnections
  • Support for SLOB status and storage characteristics information gathering
  • Thread safety
  • A test suite
  • Error, Warning, and AssertionFailure classes that replace RuntimeError when raising exceptions

Along with bug fixes, more documentation and new handy methods.

The last two features were contributed by Edwin Fine. He helped me to put together the resource management too. Thanks Ed.

In addition to the source distribution and the rubygem, a binary gem is provided for Microsoft Windows users.

There are still many things to do, for the sake of completeness. There's no rush to implement them right now, but if you have a especific request, I'd like to know about it.


Rails/Informix

Meanwhile, Rails/Informix reached 1.0.0, enabling migrations.

I'm glad to tell you that it was included in the Ruby on Rails repository and will be available as a rubygem when RoR hits 2.0. But you can use it now with RoR 1.x. Just follow the installation instructions from Rails/Informix's website.

To help you get started with Ruby on Rails and Informix, don't forget to visit the following links:



Have been useful to you?

Finally, I'm not making any money from Ruby/Informix or Rails/Informix, and never intended to do so. If not for them, I would be using Perl or Python at work. But my job wouldn't be as fun as it is right now.

But if any of them has been useful to you - like saving your sanity otherwise lost by using ESQL/C or SQR, or making your work more enjoyable and easier by using Ruby on Rails with your favorite DBMS, instead of, God forbid, Uniface! - and feel like giving something back, please consider making a donation, buying something from my Amazon wishlist, or sending a postcard of your country saying hi!

Sunday, December 17, 2006

Ruby/Informix 0.4.0 released

Ruby/Informix is almost full-featured and closer to 1.0.0. Don't hesitate to e-mail me if you have any suggestion, comment, criticism, feature request, bug report or question.

In this release there are two new major features added: multiple connections to databases and scroll cursors.

The interface to scroll cursors provided by Ruby/Informix resembles that of Array objects for accessing elements. Besides prev, next, first, last and current for traversing a scroll cursor, it counts with [] and slice methods that let you access a scroll cursor as an Array:



customers = db.cursor('select * from customers', :scroll=>true)
customers.open
customers[0] # => same as customers.first
customers[-1] # => same as customers.last
customers[2, 5] # => retrieves 5 records starting from the 3rd record
customers.drop


Just like Ruby Arrays, negative values for indices mean an offset from the end of the Array, being -1 the last element. Similarly, when using the [start, length] syntax, it retrieves an array of at most length elements starting from start. More details in the documentation.

Starting from this release, a Ruby gem will be provided for easily downloading, building and installing Ruby/Informix. Just type



gem install ruby-informix


from a command prompt on any platform supported.

Expect more exciting things to come for Ruby/Informix.



And just after releasing 0.4.0, there are news from IBM for officially supporting Ruby, according to Jerry Keesee, Director of the Informix Lab (read complete note here.) But this support won't be available until mid-2007.

Monday, November 27, 2006

Ruby/Informix 0.3.0 and Rails/Informix 0.1.0 released

Finally, the ActiveRecord adapter for IBM Informix Dynamic Server is ready for its first release. I have tested it succesfully on Solaris 9 and Windows XP with IDS 9.4 and IDS 10.

Eric Herber has been so kind as to cover Ruby/Informix at The Informix Zone as part of a review of database interfaces for popular scripting languages. The Informix Zone is an excellent resource for IBM's Secret Weapon, either if you're a long time Informix user or just curious about the exciting future that IBM is preparing for it.

Guy Bowerman also tested Rails/Informix succesfully and has posted a walkthrough, using Curt Hibb's "Rolling with Ruby on Rails" tutorial, using Informix 10 on Windows 2003. He explains
clearly how to get Ruby on Rails up an running with Informix. Don't forget to come back to Guy's blog for interesting articles about Informix development, among others.

Ruby/Informix 0.3.0 is released and this time includes initial support for Smart Large Objects (SLOB), you can create, read and write them.

Finally, it's sad that after getting the driver and the adapter working fine, it seems that I will be unable to rewrite that legacy, buggy, undocumented and insecure application (SIAPSEP) which started it all. I'm probably quitting my job in one month*.

Anyways, I hope everybody enjoys (Ruby|Rails)/Informix as much as I have. I'll try to keep adding features to them, while I still have access to Informix servers. After that I won't be able to maintain them any more. But the source is open, do whatever you want with them (it's BSD licensed after all!).

* I'll be free for hiring then, here's my résumé if you're interested.

Saturday, November 11, 2006

Ruby/Informix 0.2.1 released

This is a minor release after 0.2.0 that replaces calls to free() with xfree() to avoid crashes on Windows XP SP1.

I missed the announce of Ruby/Informix 0.2.0 which basically got new handy methods to fetch and iterate over records:

fetch_hash_many(n), fetch_hash_all
Fetch records as hashes. fetch_hash_many fetches n records at most, fetch_hash_all fetches all records as hashes. Both return an array.

each_by(n), each_hash_by(n)
Iterate by n records at a time.

fetch!, fetch_hash!, each!, each_hash!
Work the same as their counterparts without the "!" sign, except that these ones reduce the creation of objects, reusing the same ones in each call, which make them faster and memory savy.

I'm looking forward now to implement Smart Large Objects, which may help me with the ActiveRecord adapter.

Monday, April 10, 2006

Ruby/Informix 0.1.0 released!

Finally!, after a month and a half of working on it, as my day job permits, Ruby/Informix is ready for its first release. The Informix adapter for Ruby on Rails will probably follow soon!

Ruby/Informix is a Ruby extension for connecting to IBM Informix Dynamic Server, written in ESQL/C.

It provides a convenient interface for querying an Informix database the Ruby way and bringing the Informix power and flexibility to your Ruby programs.

This initial release has been heavily tested on Solaris/SPARC and has been reported to work on Linux Fedora and Windows XP. Please read the Changelog and README files for details.

Thanks to Matz for bringing the joy back to programming and IBM for maintaining this nice DBMS.

Home page: http://ruby-informix.rubyforge.org
Donwload: http://rubyforge.org/projects/ruby-informix/

Thursday, March 23, 2006

Informix adapter for Ruby on Rails, reloaded

Since my last post about the Informix extension for Ruby, I have been slowly adding features and testing it succesfully with real work at the office (thank you Rafael, Óscar and Andrey), at the point that we are already relying on it, for replacing SQR and ESQL/C (used for reports and batch database processes) with Ruby.

But the point of writing the extension was for having the foundation for an Informix adapter for Ruby on Rails ... for rewriting a legacy payroll application we are suffering from.

It wasn't much difficult to adapt the previous one based on Ruby ODBC to use the native driver. It was even easier to write than the rest of adapters, thanks to some methods implemented in the native driver ad-hoc for the job and to Informix's nobleness.

This adapter is faster than the previous one of course, and relies on less layers of software. I'm still cleaning it up, but you can mail me if you want to try it.

Friday, March 03, 2006

Informix driver for Ruby

I've been very busy at work, solving many legacy problems. One of them is the amount of unmaintainable, undocumented and buggy C programs for accessing an Informix database for generating ... reports. Another one is an insecure and unfriendly application made with an anachronic, unsupported and buggy application builder, with an Informix database as back-end.

I cannot describe enough how awful it looks, and behaves.

But I've always preferred to be part of the solution, not of the problem. It didn't take too much time to come to the conclusion that a web application would fit better, and that a better language for writing reports was needed.

Yes, a lot of work indeed. And not of my business by the way, I'm just "the DBA". But if solving problems is at your hand, why not do it? Besides, it will be fun.

This seemed to me as the opportunity to bridge my favorite language at the moment, Ruby, and Informix. Ruby is a very powerful and versatile language, easy to read and write, among other advantages. It has a Perl-like formats library [FormatR] for reports and a fabulous web development framework [Ruby on Rails].

The problem is there's no adapter for Informix for RoR, not even a native driver. I made a first attempt to write an adapter, using Ruby ODBC, but always missed a native driver for Informix.

Well, I got some time two weeks ago to write the driver, and thanks to the ease of writing Ruby extensions in C, you can find the preliminary results here. It is not complete, but already usable. Testers welcome. I'd like to hear from you either if it did or didn't work for you.

Other uses for binpatch

binpatch was conceived as a mean to create binary patches on one system with the intention of applying them on as many hosts as we need. It has helped me to maintain firewalls that wouldn't be able to patch by source (lack of disk space, CPU power, ...)

However, binpatch was recently being tried by Mike Erdely as a way to create a binary distribution for OpenSSH. With a creative approach, he built a "binary patch" to upgrade OpenSSH to 4.3 on any other server, avoiding the need to build OpenSSH on every system.

Friday, December 30, 2005

Matching ISO-8859-1 strings with Ruby

While refactoring a braindead legacy application, I needed to separate employees' last names and first names that historically have been stored in a single field in the employees table. Easy cake, even though Spanish names can be a bit more complicated than their English counterparts. Here is the regular expression:

# matches 'Gomez', 'de la Cruz', 'de los Santos', etc.
apellido = '((?:(?:de|del|la|las|los|y|san)\s+)*(?:\w|\#)+)\s+'

# matches the rest of the string after the last names
nombres = "(.*)"

# the complete regular expression
re = /#{apellido}#{apellido}#{nombres}/i

It's not perfect, but covers most of our cases, with only one failure.

These names were written in US-ASCII and thus without accented letters and 'Ñ' (did you notice the '#' in the regular expression?, it is replacing the 'Ñ'). But, what if they were used?

Ruby supports some encodings, including UTF-8 which would be enough for matching those characters. Unfortunately the database was created with the ISO-8859-1 encoding and converting it to UTF-8 was not an option because many programs and (very old) printers depend on ISO-8859-1.

Ruby supports ISO-8859-1 with its new regular expression engine code-named Oniguruma, but only in the development branch (1.9). Oniguruma will be included in Ruby 2.0.

There was one option left: converting the string from ISO-8859-1 to UTF-8 before passing it through the regular expression. This is done with the interface to the iconv library.

require 'iconv'

# We want to convert from ISO-8859-1 to UTF-8
c = Iconv.new('UTF-8', 'ISO-8859-1')

# This is an ISO-8859-1 string
fullname = "Núñez de los Santos María de Jesús"

# Converting
utf_fullname = c.iconv(fullname)

# We can test it, spliting the name into words:
utf_fullname.scan(/\w+/e)

Since \w now matches accented letters and 'ñ', the previous code splits fullname into words.

Notice the extra 'e' after the regular expression. It's an option, saying that Ruby should treat the string as UTF-8 encoded.

Sunday, December 18, 2005

Ruby and Tk on Solaris

Learning Ruby has been a pleasant and rewardable experience. Since May of this year it became one of my most valuable assets for solving problems at work, replacing Perl.

The need to understand how Ruby on Rails works started the quest. And it was such a joy programming in Ruby that I felt compelled to expand its use beyond scripting and web applications development.

Some scripts were asking for a GUI so it seemed like the next step to take. There are a hand of toolkits for which there are Ruby interfaces, like Qt, Gtk and FOX, but I choosed Tk to begin with. I made some tests on Windows, but I couldn't run them on Solaris. When requiring tk I got this error:

LoadError: No such file to load -- tcltklib

I wasn't able to find instructions on where to get and install the missing tcltklib file with Google. Hopefully this post will change this, helping those having the same problem.

The missing library is found in the ext/tcltklib directory, right inside the Ruby distribution. The installation is really simple, after reading the README file:

# ruby extconf.rb --with-tcl-include=/usr/sfw/include
# make && make install

Trivial. But easy to overlook.

Monday, December 12, 2005

Unlocking the Uniface repository. Part I.

Maintaining other people's code is ok if it is well designed, with clear and up to date documentation. But if the code is badly designed (if it ever was) with outdated or non-existent documentation, with the aggravating that it is made with an old out-of-the-market closed-source tool... well, it can be challenging.

That is my reality at work, maintaining an aberration, partially written in a tool from Compuware called Uniface, in its version 7.2.06, which is not supported any more.

Concerned about security, I started to audit this "software". And there I was, in front of around 1200 lines of code to replace to begin with. I can't, for obvious reasons, explain more about the flaw ;-) but I can tell you that these ~1200 lines scattered all over the source code had a pattern in common.

I can hear the Perl hackers saying: "just pass them through a regex". But Uniface 7.2.06 is cooked apart, as we use to say in Mexico, when speaking of source code storage.

A Uniface project's source code is not maintained as files in a filesystem. It is stored in a database in its own undocumented way, or not publicly available for that matter. I couldn't find any information about it on the web nor in the Uniface CDs. I bet I have to pay for some advanced training.

Uniface hackers would advise: "if you want to access the repository install the metadictionary". It's true that the metadictionary "decodes" the source code stored in the database, but I need to write a Uniface application that makes use of the new entities created by the metadictionary and still don't get my find & replace + regular expressions with it.

And you guessed it: Uniface 7.2.06 has no option to find & replace text outside a trigger (a piece of code assigned to a component, model, entity, etc...), let alone regular expressions. Heck, even this find & replace option is an undocumented feature. Go figure.

Exporting the source code to XML was no option, Uniface 7.2.06 doesn't eat its own dog food: it cannot load the XML file that itself generates. Exporting to and editing a TRX file is possible, but not supported by Compuware, and would require loading back the whole repository again.

I had no options left: to start modifying each of those ~1200 lines, one by one. Such is life.

Of course not.

Stay tuned for the second tech note about this topic and watch the Uniface repository in Eve's dress.

Sunday, December 11, 2005

Informix adapter for Ruby on Rails

Like many others recently, I've been touched by Ruby on Rails (RoR). It's indeed a nice framework to work with. I would talk about the beauty in its ORM implementation (ActiveRecord) or Ruby's dynamic nature that made Rails possible, but others have made it already, and way better than I can.

Rails can talk to many DBMS, but unfortunately not for the DBMS I have to use at work: Informix. Ok, it wasn't that bad actually: obstacles are opportunities. And this obstacle gave me the chance to write my first Rails adapter and at the same time gave Rails a chance to be ok'ed by the PHB.

The Informix adapter is usable already, but lacking a pair of features. I've made test applications without any problems whatsoever. Anyways, I will publish it when it is 100% ready. Testers are welcome!

binpatch 1.0.0 released

The first release of binpatch is out. Three years have passed since the first time binpatch saw the light, building binary patches for OpenBSD 3.1 and four years since the messages that started it all.

Back then binary patches for OpenBSD was a crazy idea. Nowadays a hand of different proposals have arised, each of them with its own particular approach. binpatch's approach is to be true to the OpenBSD philosophy of simplicity.

This release comes with documentation and a sample Makefile to help you create your own binary patches.

Home page: http://openbsdbinpatch.sourceforge.net/
Download: http://sourceforge.net/projects/openbsdbinpatch/