Analysis of library dependencies in the Ruby world…		Mon Oct 11 12:00:00 CEST 2004
		[link]
I have analyzed the dependencies of over 130 libraries and applications available in the Ruby Production Archive. This represents over 10% of RAA so maybe some conclusions can be drawn… The libraries most depended on are log4r and redcloth. We can see that the dependency chains are quite short, and that in general there are few dependencies. I think the latter is due to the inclusion of several well-known libraries into Ruby’s stdlib in 1.8. Take a look at the graph and try to extract some more information yourself… It represents only the libraries/applications that depend on, or are needed by, some other.

Ideas for the Ruby Production Archive — some self-critique		Mon Sep 06 12:00:00 CEST 2004
		[link]
At this stage rpa-base is working fine for many people (it is hard to know the real user base because rpa-base updates itself), and with well over 100 packaged Ruby libs/apps I’m probably second only to FreeBSD’s knu in terms of Ruby software packaged by a single individual :-). So it’s about time to analyze what I have so far and, in matz’s best style, see what sucks, and what I plan to do to address that. Inadequate support for package revisions, hidden pristine sources Currently, rpa-base supports package revisions (i.e. revisions of the packaging work itself for the same upstream sources), because it acknowledges that packaging can be non-trivial, but doesn’t leverage the fact that the upstream sources remain unmodified across minor revisions. Also, the pristine sources are only available implicitly as the initial commit to RPA’s version control repository. Once I complete the transition from .rps ports to a scheme involving upstream sources plus RPA patches, RPA will effectively work as a mirror of the projects in RAA and updates through rpa-base will be much faster (much lower BW usage). Insufficient support for binary packages Lack of signed packages/ports Digital signatures themselves are simple (I wrote a sample implementation months ago), the only complication being key management and the command line tools for RPA developers. Fortunately, RPA’s centralized nature makes the former much easier than in the case of RubyGems: end users only have to trust the signature of RPA and not those of all the upstream developers, so we don’t need a web of trust involving potentially all of the Ruby community for this to work. Platform specification and discovery This can be very tricky because the information reported by Ruby’s RUBY_PLATFORM doesn’t take us anywhere: we will at least need information concerning the ABI/runtime Ruby was built with, and probably much more if we’re linking against other libraries. This is something RubyGems doesn’t seem to address at all, despite having several binary gems in the repository, so I cannot just copy their ontology as I planned to :-( Too rigid versioning model Originally, it was decided that the best model to guarantee API compatibility would be full version sets, so that a developer would not specify the version of the library he wants directly, but rather the API he’s interested in, by specifying something like require 'rpa' RPA.version = "1.0" require 'foo' # uses foo with the API from RPA Release 1.0 if at all, since the good old require "foo" still works out of the box. All libraries within a release would be compatible, and only backwards-compatible updates would find their way into a RPA release. However, some libraries/applications just change too quickly, and some people have expressed the need for explicit cherry-picking of the libraries they depend on. I think that the latter should be the exception and not the primary means to specify API requirements, like RubyGems does — otherwise you end up having to change all the require_gem 'foo', ">=1.0", "<2" # ... in some other file ... require_gem 'bar', ">=2.0", "<3" # bar 2.x works with foo 1.x manually across several files, instead of changing a single RPA.version="x.y" in one. Anyway, since I’d hate to tell any developer that what he wants to do isn’t possible due to stupid limitations in my software, rpa-base will provide a way to do explicit cherry-picking for libraries, while keeping the general "repackager-friendly, transparent installs" model. I will probably generalize this all by adding explicit support for "package clusters": that way a RPA release would be but a rather large list of compatible ports. I will explore the way to merge FreeBSD’s and Gentoo’s model, probably by adding more "dependency primitives", which I will happily steal from Debian :).

All the packages you ever wanted… or not?		Sun Aug 22 12:00:00 CEST 2004
		[link]
I have invited the people from ruby-talk to request packages for the Ruby Production Archive. So far the response has been a bit disappointing; seemingly no Rubyist wants to use software packaged with some care, or maybe it’s just that the 114 packages currently available cover all needs? Come on, asking somebody else to do something for you isn’t that difficult, is it? If you prefer doing it yourself, just contact me and I’ll assist you :) But if I were you I’d just request more packages here. If you want to try the 100+ packages available at the moment, you can install rpa-base in a couple minutes. Full refund guarantee (it can uninstall itself), and remember installs are atomic. Update James Britt tried rpa-base on Win2K (and he had some problems that were fortunately solved quickly), and made me realize that the above could be read as implying that the packages in RPA are the only ones made "with some care". Of course I didn’t mean to imply that RPA packages are the only good ones. There are many good things out there, and in fact some of my packages might be buggy too, since the full QA process isn’t yet in place. So it’s not as good as it will get (hopefully :), but this doesn’t mean it’s not pretty OK on the average already… And if you don’t like a RPA package, you get a full refund (clean, transacted uninstall) ;-))

Recent updates to the Ruby Production Archive (RPA)		Sat Aug 14 12:00:00 CEST 2004
		[link]
More packages are being added to the preliminary Ruby Production Archive on a daily basis; here’s what I’ve packaged recently: PrettyException: a lib. to pretty-print exceptions in HTML format (very useful for the web. devels out there) MW::Template: a templating library needed by PrettyException I have also updated a number of packages to the latest releases; here are the ones I’ve done in the last few days: diff-lcs 1.1.0 rb-gsl 1.5.0 aeditor 1.5 sqlite 1.3.1 IOWA 0.9 Changes in rpa-base I have been a bit too busy maintaining the repository and answering to too many emails — if you’ve seen all the noise in ruby-talk, you might want to listen to the (sadly) only authoritative source of information on RPA at the moment: I will eventually try to explain the goals better and reply to some of the statements made a bit light-heartedly in ruby-talk. I could still manage to fix a few bugs that affected the bootstrap phase of rpa-base (e.g. it would fail to install itself under mingw) and implement a few additional features (notably better proxy specification). Tarballs with the ‘release critical’ fixes are available in Rubyforge. If you had rpa-base already installed, you can upgrade it with rpa update rpa install rpa-base as usual.

100+ Ruby libraries and applications in the Ruby Production Archive		Sat Aug 07 12:00:00 CEST 2004
		[link]
rpa-base 0.2.0 has been released. In order to prove that the Ruby Production Archive (RPA) approach is practical, I created over 100 packages: all of Rubyforge’s "top sellers" (Rails, Rake, RedCloth, Active Record, SQLite, Log4R, Copland, ruvi, to name a few) and many others: take a look at the full list of packaged software. This means that it is now possible to do rpa install instiki rake rails ruvi # or any of the other 100+ libs/apps and get these packages plus all their dependencies installed in one go, and atomically (no garbage left, guaranteed), on a number of platforms (rpa-base has been tested on OSX, FreeBSD, DragonFly BSD, Debian, Gentoo, Fedora, older RH, Win32: XP, 2K, cygwin and "Pragmatic installer", etc…) A number of movies show rpa-base in action, while installing Instiki, Rake, Rlimit (a C extension), Rails, etc… I have some funny videos where syck crashes and rpa-base recovers just fine :)… I am fully expecting other projects to mimic the usage of animations to showcase apps, which I originally stole from Rails, especially since I have created them with yet another small Ruby script. Call for help RPA is a very ambitious project and I could really use some help. Here are some of the areas that need to be worked on: packaging (new software and package maintenance) website development (should provide package indexes, QA section, bugtracking, etc) setting up a permanent repository infrastructure cross-compilation and build automation The 2 first ones in particular can be carried out with relative independence from rpa-base… I’d be really happy if you dropped a message to <batsman dot geo at yahoo dot com> (adding RPA to the subject will help get it past the spam filtering ;) or contacted me via IRC, batsman @ #ruby-lang on freenode.net.

ruvi, an editor written in Ruby		Mon Jul 19 12:00:00 CEST 2004
		[link]
I owe lypanov a link to ruvi, the übercool modal editor he’s writing in Ruby. He’s been adding the features from vim I use to make me switch… and he’s getting close. Multiple buffers plus sort of tabs, Ruby highlighting and autoindent, and :set ruler is coming soon too. I packaged ruvi using rpa-base some time ago but I fear nobody has installed it that way :-| He’s using the code coverage tool I developed for rpa-base (rcov: available with rpa install rcov), which creates reports like this one.

Defining methods with closures		Sun Jul 18 12:00:00 CEST 2004
		[link]
I had forgotten that there are still Rubyists who don’t know the class << self; self end.send(:define_method, :foo){ } idiom. I wrote a short explanation and some hints on how to use define_method. Module#define_method is my method of choice (har har) to define methods dynamically, since when I am meta-programming I usually need to define them as a closure anyway: this is especially useful when you create a new object, add some singleton methods to it that modify some data in your environment and then perform an instance_eval on a Proc; this is a way to define a restricted language to be used inside that block. Let’s see a stupid synthetic example: require "pp" class Foo def initialize(state = "initial") @state = [state] end def update(&block) magic_obj = Object.new # throw-away obj, only to capture method calls state = @state class << magic_obj; self end.send(:define_method, :do_magic) do |a| puts "This object allows do_magic operations only..." puts "i.e. we have restricted the allowable operations inside the block" state << a end magic_obj.instance_eval(&block) end end f = Foo.new f.update do # this block uses a different "vocabulary", i.e. it can behave as a new, # restricted language. This is more general than the <tt>yield self</tt> # technique used normally. do_magic "foo" end pp f I think I’ll probably write a bit more on this (and meta-programming in general) in my Ruby section. BTW, I use this technique (with a couple additional meta-programming layers though) in rpa-base, which allows me to define a IMHO very good language for the rpafied install.rb files: they are as descriptive and short as it gets; I designed the language first and then implemented it with heavy meta-programming in Ruby…

rpa-base 0.1.0 ‘kitanai’ released		Thu Jun 17 12:00:00 CEST 2004
		[link]
… so now you can rpa install instiki ruvi rake ri-rpa officially :) rpa-base.rubyforge.org The first public release of rpa-base 0.1.0 (‘kitanai’) is out, after around 4 months of hacking, over 500 revisions and tens of snapshot releases on #ruby-lang. The current ‘bogorepository’ (which is not the real thing, so this is not a proper RPA release) contains several interesting ports, including Instiki 0.9.1 (IIRC not yet available as a Gem ;), amrita, bluecloth, redcloth, ruvi, rake and rcov. Well, the latter cannot really be compared to the others but it’s pretty neat and will generate things like www.lypanov.net/+generated/coverage/ . I hope somebody will realize RPA’s importance and offer some help in the most important phase, which is just about to begin: Policy definition and mass packaging. Work on the infrastructure (repository, version control system…) will take place in parallel.

Keine leichte Geburt (no easy birth)		Wed Jun 02 12:00:00 CEST 2004
		[link]
Some people on #ruby-lang are aware of my involvement in RPA (see the manifesto), and I have been giving away snapshots of rpa-base on IRC, but it’s still mostly a secret; time to uncover it. Introducing rpa-base Short version of the README (the most important piece of documentation as everybody knows :) rpa-base is a port/package manager created to be the base for RPA’s client-side package management. You can think of it as RPA’s apt-get + dpkg. Some sexy features (see the link above for more) working now: sane dependency management: rpa-base installs dependencies as needed, keeps track of reverse dependencies on uninstall, and will remove no longer needed dependencies atomic (de)installs: operations on the local RPA installation are atomic transactions; the system has been designed to survive ruby crashes (OS crashes too on POSIX systems) handling C extensions: if you have the required C toolchain, rpa-base can compile extensions as needed rdoc integration: RDoc documentation for libraries is generated at install time (currently put in $prefix/share/doc/rpa0.0/portname) unit testing: when a library is installed, its unit tests are run; the installation is canceled if they don’t pass ri integration: ri data files are generated for all the libraries managed by RPA; you can access this information with ri-rpa Releasing I’ve been feeling it was ready for a preliminary release for a long time now (since around mid-March), but I’m notoriously bad at that and just kept hacking. Now that RPA is reported to work on win32, it’s definitely time for a tech-preview release. I’m soon applying for Rubyforge’s services and working on a FAQ and some minimalistic documentation.

Some tricks…		Tue Jun 01 12:00:00 CEST 2004
		[link]
I’ve found some ~150 .rb files in /tmp which illustrate a number of tricks (mostly solutions to problems ppl ask about in #ruby-lang); I’ll put some of them in the Ruby section. First take, capturing instance method definitions for fun and profit (clean AOP using closures & stuff).