Analysis of library dependencies in the Ruby world…		Mon Oct 11 12:00:00 CEST 2004
		[link]
I have analyzed the dependencies of over 130 libraries and applications available in the Ruby Production Archive. This represents over 10% of RAA so maybe some conclusions can be drawn… The libraries most depended on are log4r and redcloth. We can see that the dependency chains are quite short, and that in general there are few dependencies. I think the latter is due to the inclusion of several well-known libraries into Ruby’s stdlib in 1.8. Take a look at the graph and try to extract some more information yourself… It represents only the libraries/applications that depend on, or are needed by, some other.

Ideas for the Ruby Production Archive — some self-critique		Mon Sep 06 12:00:00 CEST 2004
		[link]
At this stage rpa-base is working fine for many people (it is hard to know the real user base because rpa-base updates itself), and with well over 100 packaged Ruby libs/apps I’m probably second only to FreeBSD’s knu in terms of Ruby software packaged by a single individual :-). So it’s about time to analyze what I have so far and, in matz’s best style, see what sucks, and what I plan to do to address that. Inadequate support for package revisions, hidden pristine sources Currently, rpa-base supports package revisions (i.e. revisions of the packaging work itself for the same upstream sources), because it acknowledges that packaging can be non-trivial, but doesn’t leverage the fact that the upstream sources remain unmodified across minor revisions. Also, the pristine sources are only available implicitly as the initial commit to RPA’s version control repository. Once I complete the transition from .rps ports to a scheme involving upstream sources plus RPA patches, RPA will effectively work as a mirror of the projects in RAA and updates through rpa-base will be much faster (much lower BW usage). Insufficient support for binary packages Lack of signed packages/ports Digital signatures themselves are simple (I wrote a sample implementation months ago), the only complication being key management and the command line tools for RPA developers. Fortunately, RPA’s centralized nature makes the former much easier than in the case of RubyGems: end users only have to trust the signature of RPA and not those of all the upstream developers, so we don’t need a web of trust involving potentially all of the Ruby community for this to work. Platform specification and discovery This can be very tricky because the information reported by Ruby’s RUBY_PLATFORM doesn’t take us anywhere: we will at least need information concerning the ABI/runtime Ruby was built with, and probably much more if we’re linking against other libraries. This is something RubyGems doesn’t seem to address at all, despite having several binary gems in the repository, so I cannot just copy their ontology as I planned to :-( Too rigid versioning model Originally, it was decided that the best model to guarantee API compatibility would be full version sets, so that a developer would not specify the version of the library he wants directly, but rather the API he’s interested in, by specifying something like require 'rpa' RPA.version = "1.0" require 'foo' # uses foo with the API from RPA Release 1.0 if at all, since the good old require "foo" still works out of the box. All libraries within a release would be compatible, and only backwards-compatible updates would find their way into a RPA release. However, some libraries/applications just change too quickly, and some people have expressed the need for explicit cherry-picking of the libraries they depend on. I think that the latter should be the exception and not the primary means to specify API requirements, like RubyGems does — otherwise you end up having to change all the require_gem 'foo', ">=1.0", "<2" # ... in some other file ... require_gem 'bar', ">=2.0", "<3" # bar 2.x works with foo 1.x manually across several files, instead of changing a single RPA.version="x.y" in one. Anyway, since I’d hate to tell any developer that what he wants to do isn’t possible due to stupid limitations in my software, rpa-base will provide a way to do explicit cherry-picking for libraries, while keeping the general "repackager-friendly, transparent installs" model. I will probably generalize this all by adding explicit support for "package clusters": that way a RPA release would be but a rather large list of compatible ports. I will explore the way to merge FreeBSD’s and Gentoo’s model, probably by adding more "dependency primitives", which I will happily steal from Debian :).

All the packages you ever wanted… or not?		Sun Aug 22 12:00:00 CEST 2004
		[link]
I have invited the people from ruby-talk to request packages for the Ruby Production Archive. So far the response has been a bit disappointing; seemingly no Rubyist wants to use software packaged with some care, or maybe it’s just that the 114 packages currently available cover all needs? Come on, asking somebody else to do something for you isn’t that difficult, is it? If you prefer doing it yourself, just contact me and I’ll assist you :) But if I were you I’d just request more packages here. If you want to try the 100+ packages available at the moment, you can install rpa-base in a couple minutes. Full refund guarantee (it can uninstall itself), and remember installs are atomic. Update James Britt tried rpa-base on Win2K (and he had some problems that were fortunately solved quickly), and made me realize that the above could be read as implying that the packages in RPA are the only ones made "with some care". Of course I didn’t mean to imply that RPA packages are the only good ones. There are many good things out there, and in fact some of my packages might be buggy too, since the full QA process isn’t yet in place. So it’s not as good as it will get (hopefully :), but this doesn’t mean it’s not pretty OK on the average already… And if you don’t like a RPA package, you get a full refund (clean, transacted uninstall) ;-))

Recent updates to the Ruby Production Archive (RPA)		Sat Aug 14 12:00:00 CEST 2004
		[link]
More packages are being added to the preliminary Ruby Production Archive on a daily basis; here’s what I’ve packaged recently: PrettyException: a lib. to pretty-print exceptions in HTML format (very useful for the web. devels out there) MW::Template: a templating library needed by PrettyException I have also updated a number of packages to the latest releases; here are the ones I’ve done in the last few days: diff-lcs 1.1.0 rb-gsl 1.5.0 aeditor 1.5 sqlite 1.3.1 IOWA 0.9 Changes in rpa-base I have been a bit too busy maintaining the repository and answering to too many emails — if you’ve seen all the noise in ruby-talk, you might want to listen to the (sadly) only authoritative source of information on RPA at the moment: I will eventually try to explain the goals better and reply to some of the statements made a bit light-heartedly in ruby-talk. I could still manage to fix a few bugs that affected the bootstrap phase of rpa-base (e.g. it would fail to install itself under mingw) and implement a few additional features (notably better proxy specification). Tarballs with the ‘release critical’ fixes are available in Rubyforge. If you had rpa-base already installed, you can upgrade it with rpa update rpa install rpa-base as usual.

100+ Ruby libraries and applications in the Ruby Production Archive		Sat Aug 07 12:00:00 CEST 2004
		[link]
rpa-base 0.2.0 has been released. In order to prove that the Ruby Production Archive (RPA) approach is practical, I created over 100 packages: all of Rubyforge’s "top sellers" (Rails, Rake, RedCloth, Active Record, SQLite, Log4R, Copland, ruvi, to name a few) and many others: take a look at the full list of packaged software. This means that it is now possible to do rpa install instiki rake rails ruvi # or any of the other 100+ libs/apps and get these packages plus all their dependencies installed in one go, and atomically (no garbage left, guaranteed), on a number of platforms (rpa-base has been tested on OSX, FreeBSD, DragonFly BSD, Debian, Gentoo, Fedora, older RH, Win32: XP, 2K, cygwin and "Pragmatic installer", etc…) A number of movies show rpa-base in action, while installing Instiki, Rake, Rlimit (a C extension), Rails, etc… I have some funny videos where syck crashes and rpa-base recovers just fine :)… I am fully expecting other projects to mimic the usage of animations to showcase apps, which I originally stole from Rails, especially since I have created them with yet another small Ruby script. Call for help RPA is a very ambitious project and I could really use some help. Here are some of the areas that need to be worked on: packaging (new software and package maintenance) website development (should provide package indexes, QA section, bugtracking, etc) setting up a permanent repository infrastructure cross-compilation and build automation The 2 first ones in particular can be carried out with relative independence from rpa-base… I’d be really happy if you dropped a message to <batsman dot geo at yahoo dot com> (adding RPA to the subject will help get it past the spam filtering ;) or contacted me via IRC, batsman @ #ruby-lang on freenode.net.

ruvi, an editor written in Ruby		Mon Jul 19 12:00:00 CEST 2004
		[link]
I owe lypanov a link to ruvi, the übercool modal editor he’s writing in Ruby. He’s been adding the features from vim I use to make me switch… and he’s getting close. Multiple buffers plus sort of tabs, Ruby highlighting and autoindent, and :set ruler is coming soon too. I packaged ruvi using rpa-base some time ago but I fear nobody has installed it that way :-| He’s using the code coverage tool I developed for rpa-base (rcov: available with rpa install rcov), which creates reports like this one.

Defining methods with closures		Sun Jul 18 12:00:00 CEST 2004
		[link]
I had forgotten that there are still Rubyists who don’t know the class << self; self end.send(:define_method, :foo){ } idiom. I wrote a short explanation and some hints on how to use define_method. Module#define_method is my method of choice (har har) to define methods dynamically, since when I am meta-programming I usually need to define them as a closure anyway: this is especially useful when you create a new object, add some singleton methods to it that modify some data in your environment and then perform an instance_eval on a Proc; this is a way to define a restricted language to be used inside that block. Let’s see a stupid synthetic example: require "pp" class Foo def initialize(state = "initial") @state = [state] end def update(&block) magic_obj = Object.new # throw-away obj, only to capture method calls state = @state class << magic_obj; self end.send(:define_method, :do_magic) do |a| puts "This object allows do_magic operations only..." puts "i.e. we have restricted the allowable operations inside the block" state << a end magic_obj.instance_eval(&block) end end f = Foo.new f.update do # this block uses a different "vocabulary", i.e. it can behave as a new, # restricted language. This is more general than the <tt>yield self</tt> # technique used normally. do_magic "foo" end pp f I think I’ll probably write a bit more on this (and meta-programming in general) in my Ruby section. BTW, I use this technique (with a couple additional meta-programming layers though) in rpa-base, which allows me to define a IMHO very good language for the rpafied install.rb files: they are as descriptive and short as it gets; I designed the language first and then implemented it with heavy meta-programming in Ruby…

rpa-base 0.1.0 ‘kitanai’ released		Thu Jun 17 12:00:00 CEST 2004
		[link]
… so now you can rpa install instiki ruvi rake ri-rpa officially :) rpa-base.rubyforge.org The first public release of rpa-base 0.1.0 (‘kitanai’) is out, after around 4 months of hacking, over 500 revisions and tens of snapshot releases on #ruby-lang. The current ‘bogorepository’ (which is not the real thing, so this is not a proper RPA release) contains several interesting ports, including Instiki 0.9.1 (IIRC not yet available as a Gem ;), amrita, bluecloth, redcloth, ruvi, rake and rcov. Well, the latter cannot really be compared to the others but it’s pretty neat and will generate things like www.lypanov.net/+generated/coverage/ . I hope somebody will realize RPA’s importance and offer some help in the most important phase, which is just about to begin: Policy definition and mass packaging. Work on the infrastructure (repository, version control system…) will take place in parallel.

Keine leichte Geburt (no easy birth)		Wed Jun 02 12:00:00 CEST 2004
		[link]
Some people on #ruby-lang are aware of my involvement in RPA (see the manifesto), and I have been giving away snapshots of rpa-base on IRC, but it’s still mostly a secret; time to uncover it. Introducing rpa-base Short version of the README (the most important piece of documentation as everybody knows :) rpa-base is a port/package manager created to be the base for RPA’s client-side package management. You can think of it as RPA’s apt-get + dpkg. Some sexy features (see the link above for more) working now: sane dependency management: rpa-base installs dependencies as needed, keeps track of reverse dependencies on uninstall, and will remove no longer needed dependencies atomic (de)installs: operations on the local RPA installation are atomic transactions; the system has been designed to survive ruby crashes (OS crashes too on POSIX systems) handling C extensions: if you have the required C toolchain, rpa-base can compile extensions as needed rdoc integration: RDoc documentation for libraries is generated at install time (currently put in $prefix/share/doc/rpa0.0/portname) unit testing: when a library is installed, its unit tests are run; the installation is canceled if they don’t pass ri integration: ri data files are generated for all the libraries managed by RPA; you can access this information with ri-rpa Releasing I’ve been feeling it was ready for a preliminary release for a long time now (since around mid-March), but I’m notoriously bad at that and just kept hacking. Now that RPA is reported to work on win32, it’s definitely time for a tech-preview release. I’m soon applying for Rubyforge’s services and working on a FAQ and some minimalistic documentation.

Some tricks…		Tue Jun 01 12:00:00 CEST 2004
		[link]
I’ve found some ~150 .rb files in /tmp which illustrate a number of tricks (mostly solutions to problems ppl ask about in #ruby-lang); I’ll put some of them in the Ruby section. First take, capturing instance method definitions for fun and profit (clean AOP using closures & stuff).

Releasing often…		Mon May 31 12:00:00 CEST 2004
		[link]
I’m so bad at that. Over the last few months I’ve written some ~15000 lines of as of yet unreleased Ruby code. Since today (2004-05-31) is unexpectedly a Feiertag (I’ve never bothered to learn those dates, esp. if they’re related to some prominent sect), I’m updating my site and sort of releasing rdocsite 0.0.1rc3. Actually, I’m much worse at releasing early: I have reimplemented a couple things in RAA and elsewhere, minus the occasional braindeath, and leave them in my local svn repos as 0.0.1 forever, even though they’re sometimes better than the stuff declared to be >0.3.0 or worse, 1.0, out there. Of course, my versioning scheme is essentially sick, cause my stuff is usually feature-complete & not too buggy by 0.0.1. If I was releasing, the version numbers would increase faster since they’d reflect API changes, but as long as I don’t really release, I’m free to break the API as often as I like within the 1st tech preview or 0.0.1…

How I became a code fascist		Mon May 31 12:00:00 CEST 2004
		[link]
Code fascist is but a catchy expression to summarize the following: I’m tired of pointless discussions I’m sick of discussions at the absolute clueless level I hate with a passion discussions where clueless ppl think they’ve found the solutions everybody else was unable to find before code is the final argument in many technical discussions I’m seemingly getting increasingly identified with "do the Right Thing", that is, while I recognize the benefits of "worse is better", I do not like to procrastinate the Right solution and/or let others do the work I implicitly assumed when stating my goals (I have no problem with delegating, but I won’t "delegate" to abstract entities like the "community", or worse, "the community at some point in the future"). Therefore, I sometimes won’t release even when my code is objectively better than what’s out there. Mar 25 <dblack> to me it's part of the ruby inferiority complex Mar 25 <dblack> "let's do what java programmers feel comfortable with" Mar 25 <dblack> which is why it bothers me (along with the visual clutter it introduces) Mar 25 <batsman> I'm affected by another variant of the ruby (not Ruby) inferiority complex Mar 25 <dblack> namely? Mar 25 <batsman> I think the implementation is not as good as I'd like it to be :P Mar 25 <batsman> there's nothing in the language itself to be ashamed of Mar 25 <dblack> yeah Mar 25 <batsman> as for making Java programmers feel comfortable -> if that means turning Ruby into some crap syntax, no way [...] Mar 25 <batsman> too many discussions, too little code. if only people hacked some more and talked less :P ... Mar 25 <phubuh> maybe we should adopt the lisp community's practice of ritually chanting "MORE CODE" Mar 25 <phubuh> MORE CODE Mar 25 <phubuh> MORE CODE Mar 25 <batsman> esp. when the discussions consistently go back to the 'near clueless' level Mar 25 * batsman was about to add another line but then he realized it wouldn't be consistent w/ "too much discussion" & went back to coding [...] Mar 26 <batsman|off> MORE CODE DAMMIT [...] Apr 02 <batsman||-> MORE CODE! [...] Apr 16 <batsman> MORE CODE! [...] Apr 23 <batsman|off> MORE CODE! [...] May 02 <batsman> MORE CODE! [...] May 02 <batsman|away> .oO(MORE CODE!)

It’s a small world (or how geekness meets real life)		Fri Aug 29 12:00:00 CEST 2003
		[link]
Something quite funny happened to me 3 days ago. I had to take another routine flight from Stuttgart (Germany) to Madrid (Spain). Having slept too little, and stayed far too long in #ruby-lang, I forgot to check properly that I had no lethal weapons in my hand luggage. So, not really to my surprise, I had one. Actually, in my pockets. A 3cm long Swiss army knife. I had to put it in my backpack and check in again. The flight would take over 2H and I had some time before boarding, so I thought I’d need something to read and took ‘Design Patterns’ as a light reading. On the way to the plane, one American (you’d not expect to see many of them in a small city of Old Europe nowadays) saw my book and pointed "not many people would read this book for fun". And then right after "I used to work with John Vlissides in IBM" (he’s the fourth name in the Gang of Four). Now, isn’t the world small? We then talked shortly about design patterns being most helpful in statically typed languages. Indeed, most of them are not needed when using dynamic typing. And when you do need them, they are (exceedingly) easy to think of and implement. I was surprised to see that he knew Ruby. It was only much later that I remembered that he told me he was a Pythonist. This means (obviously) that we should have fought to death. Too bad we forgot it ;)

Language features and cross-pollination		Tue Jun 24 12:00:00 CEST 2003
		[link]
I rencently read Guido’s report on OSCON. The following caught my attention: for me the high point of the evening was Miguel de Icaza's excitement over Python's feature which makes text files iterators yielding the lines of the file in succession, so that you can write for line in open(filename): ...process line... Iterators were introduced in Python 2.2. Ruby has had them forever now. I then found this small walk-through for Perl6. They have introduced a new statement (given) which behaves as a switch were the values compared to decide on the nature of the comparison. Familiar? Humbleness The Ruby community is well aware of its position of inferiority with regard to Perl’s and Python’s. Matz has been quoted to say that Ruby is a "Smalltalk rip-off". We know RAA is so much smaller than CPAN. We are aware of the limitations of Ruby, yet we keep using it. Why? The Ruby community has often been described as the major asset of this language. Concepts such as happiness and stress reduction pervade ruby-talk (one of our highest traffic lists). This is consistent with matz’ ideals and turns out to be actively supported by the language itself. A language that makes people happy What features make a language a source of happiness for those using it? It turns out it’s not the features themselves, nor the expressive power. We all know Ruby has historically taken lots of things from several languages, mainly OO concepts from Smalltalk, things from Perl (regexps, some syntax) and a few things from other languages (yield mostly from CLU, continuations, closures…). For a long time matz took features from here and there, added some he thought of independently, and cared a lot about making a nice, relaxing, beautiful language, without falling into featuritis. The language evolved a little bit as time went on, but is still very close to its original embodiment. We certainly didn’t add object orientation after the 4th release (amongst other reasons, because Ruby is still at 1.8 as of now :-), but it was OO from the beginning), nor decided that we needed generators, iterators, true closures and things after 2.1… If you have ever programmed in Ruby, you’ll notice you’re using iterators, blocks, mix-ins… more often than not. This is no wonder, as they pervade the language, you find them in the core classes, in the modules, everywhere. As we used them all the time, we became able to make cool and beautiful things with them. And learned to appreciate them. And we were happy. Ruby pollinating other languages… The folks using Python and Perl have realized how happy Rubyists tend to be. And they looked for the reason. It seems the answers to their questions were the features (almost) nobody wanted in the first place, the things that remained hidden for a long time in cool, minority languages. So they began to add things to their languages. Meanwhile, matz keeps refusing new, "cool" features, if their usefulness isn’t proved, if the syntax isn’t Ruby-like, or if no suitable name can be found. Lots of perlers are looking for happiness, without realizing it. They hope they will find relief for the things that make them suffer now in Perl 5, sometime in the future (possibly 1 or 2 years away from now), in Perl 6. The new features ought to give them more power, and make them happier. I believe matz is right, the others are wrong. New features appearing in a languages don’t retroactively become a part of the existing code base. I read some time ago that full OO still had not made it into Python’s standard libraries: they still have non-OO things around. The obvious way to rethink libraries to use the new features and "change the language taste" is rewriting things from scratch (that’s what Perlers are doing anyway). The transition to Perl 6 will be an interesting moment. If Ruby is widely known by that time, several developers will have to choose amongst the following (and some more) options: a language with a long "usefulness history", but which became too bloated at some time, and which promises it will change to make them happier (by adding features and breaking previous code) a language that (from the very beginning) made programmers happier, and became increasingly useful as more people used it Lots of Ruby users today (most?) are Perl refugees. I believe there will be many many more when the transition to Perl 6 happens and people re-evaluate what each language offers. I think Ruby will have a massive number of new users if Rite becomes real soon (esp. if it works before Parrot). Everybody, go for the son-shi prize!

Ruby/Java integration		Thu Jun 19 12:00:00 CEST 2003
		[link]
It’s been some time (around 3 weeks and a half now) since I started to hack rjni. At first I didn’t think about what integration does really mean: I was just implementing (quite in a hurry) an idea that popped out during Euruko 03. So I just provided a simple means to use JNI from Ruby and a (IMHO neat) mapping of Java to Ruby concepts: static methods become class singleton methods, accessors are created for attributes, etc. I’ve been thinking of expanding the scope of rjni to cope with the following issues: type-safety: as of now, you can crash the interpreter by using JNI with wrong parameters. This could be prevented but there’s a performance hit that’s what you get for using low-level APIs instead of "magical reflection" signal handling: Ruby’s threading model uses signals to simulate threads (green threads). What are the implications when linking to the JVM? sub-classing Java classes in Ruby and using them from Ruby: already possible, but not clean enough as there’s a couple things to consider: method overloading in Java complicates things on the Ruby side Java attributes don’t become instance variables (you have to go through accessors, even inside instance methods) using Ruby from Java? Involves mapping Ruby’s C API through JNI then it becomes possible to subclass Java classes in Ruby and use them in Java… threads: Ruby’s interpreter is not thread safe; this will be a problem if using Ruby from Java becomes possible The signal handling issue is especially problematic since that’s quite tricky and possibly system- and JVM-dependent. Threads could prove to be difficult to get right, too. If you see how to solve these problems (or want to have some fun doing so :-) contact me at <batsman dot geo at yahoo dot com>.

Participation in ICFP 2003		Wed Jun 11 12:00:00 CEST 2003
		[link]
I participated in the ICFP 2003 contest (www.dtek.chalmers.se/groups/icfpcontest/), joining a team of Rubyists. Even though we didn’t submit an entry in the end, it was really fun, and very valuable (not to mention exhausting :). The main project lives (and dies) within a 72H time-frame, and involves tight communication between people who had (in our case) never met before, neither physically nor virtually. Problems arise naturally within a few hours, and it is indeed from mistakes that one can learn the most. What failed UPDATE (Jul 11 2003) You can get more detailed information on my experience at ICFP 03 in www.thekode.net/icfp03/lessons.html. Some of the things that happened to us include (I’m using the checklist from "The Pragmatic Programmer" as I write this :-): not getting the whole picture from the beginning insufficient planning communication problems incorrect use of prototypes inability to properly estimate devel. times over-engineering not fixing interfaces in time: they kept changing and we lost time readjusting we failed to see that a class/module/file is not (in this case) the right granularity level to achieve parallel development ad-hoc sub-teaming was sub-optimal

Building my site with rdocsite		Mon Jun 09 12:00:00 CEST 2003
		[link]
I first started to use RubLog because of the following sexy features: Rdoc markup file-based system: simply put a new file into the tree and everything will be updated written in Ruby!!! Now I see that what I really wanted was a Wiki-like tool to generate my site. Well, actually quite different from the standard wiki spirit: I don’t want to edit online, or to have concurrent edition, or anything; I only want to be able to create my pages easily, move nodes around without links breaking, have plugins automatically generate table of contents and such niceties, etc. So I decided to write my own tool. rdocsite These are the features I retained: RDoc markup templating system (RDoc’s :-) WikiNames, with some enhancements to allow hierarchies (a WikiName will link to the closest node in the hierarchical tree; name clashes are resolved sensibly). You can also specify (part of) the document hierarchy to find the WikiName in. meta-data: each node can have associated meta-data, which drives plugin operations and template expansion. Moreover, meta-data can be inherited along the hierarchy. And of course, still written in Ruby :-) The initial code took around 3 days. I’m now cleaning a bit, reworking the plugin system and thinking of new plugins, but the code is already functional and I use it to generate my website at www.thekode.net.