Modularity, Ruby and Doing the Right Thing
14. May, 2007 
15 Comments
It seems like we stirred up some controversy with these lines of code in our demo:
JS = require ‘foo.js’
…
JS.initialize
Let’s be clear about what this code does: it creates a ScriptModule (a DLR type) that represents the execution context of the foo.js code, and assigns it to the JS constant. The JS.initialize line calls the initialize method in foo.js. This is a user-defined method, and not some kind of DLR initialization thing.
We debated the idea of using require to import foreign modules in my office for a very short period of time before checking in the change. It seemed like a natural thing to do – monkey-patch the require method so that we can import foreign modules into IronRuby.
Nobody has argued that importing foreign modules into IronRuby is a bad idea.[1] Rather, I think that folks have taken issue with using require as the mechanism for doing so.
Point taken. We will have a different mechanism for doing so in the future.
But this brings up an important meta-level issue: our commitment to doing the right thing. It seems that our intentions in this area have been called into question. We have a fairly straightforward task here: to make a CLR implementation of the Ruby language that is as compatible as we can make it. So when we have a design decision to make, it’s driven by “how does the native Ruby implementation handle it”?
We here on the IronRuby team are under no illusion that we will be 100% compatible with the existing Ruby 1.8.x branch. But we will get as close as we can and bend over backwards to ensure that it is so. And I expect that you guys will keep us honest in this regard when it seems like we made a bad design decision.[2]
Pure Ruby code should just work, and code that depends on native platform libraries will have to be ported (either by us porting the native platform libraries or by folks modifying the code to work with our native platform libraries aka the Framework Class Libraries).
Please do speak up when we post code frags, demos (and eventually source). We’re really interested in what you have to say !
[1] Keep in mind that we wanted to demo something interesting at MIX (cross-language interop) and that using Ruby (the least mature of our language implementations) as the glue to bind other code together seemed to be the right thing to do. However, at that time we couldn’t solicit input from the community since we were in stealth mode.
[2] While I don’t think that overloading require was a bad design decision (nobody has given us a concrete example that shows where this breaks their app / library), it’s controversial enough and the fix simple enough to do that I have no problem doing it.
I think the ability to maintain the way the native Ruby implementation is done is very important. Ideally, only when you need to introduce features that are platform dependent should you deviate.
I’m not a Rubyist so I don’t know all of its design guidelines and code peculiarities, but I think overloading ‘require’ is Just The Right Thing To Do. Perhaps because I’m used to C# and Python’s way of doing it which is similar to the way you’ve done it in IronRuby. I think it makes sense.
What are the objections people have to overloading ‘require’? Is it that it doesn’t work in other Ruby implementations? If so, can’t this be proposed to the Ruby guys to be specified and implemented in a future Ruby version (perhaps 1.9 or even 1.8)? Would that be so bad? I’d love to hear the different opinions on this.
Here’s my 2c. Microsoft earned my trust by trying very hard to keep IronPython compatible with CPython. I hope this extends to IronRuby as well.
require returns a boolean result. We played a bit fast and loose by returning a ScriptModule reference to indicate true (which won’t break code that depends on type: (puts ‘true’ if require ‘foo.js’) will just work).
Dependencies on the negative result are what worry me. One time initialization code would be a good example – we would still need to return the ScriptModule reference but then there would be no good way of detecting whether the foreign module had been included already. It’s this latter case (which BTW nobody brought up in private mail) that worried me enought to change the behavior.
I would imagine that we would be using the IronPython clr.Use() mechanism to require foreign modules – this lets us precisely define the semantics without worrying about breaking existing code.
However, there is a trade-off of requiring folks to learn yet another concept. In RubyCLR, most of the questions on the mailing list were related to not understanding when to use reference vs. require (or even knowing that reference existed at all).
John, I’m glad you explained the incompatibilities you introduced for SilverLight/DLR integration of IronRuby. But I would strongly caution you against making ANY such changes or enhancements to the language or libraries until you are well down the road to compatibility. There are two big reasons for this:
- Without a spec and without being able to look at other implementations’ code, you have a massive challenge in creating a compatible implementation of Ruby. Making selective changes, no matter how well-thought-out, will only make that task harder…and you may unknowingly break compatibility in unfixable ways.
- Such changes, even accompanied by reasonable explanations, are going to be skewered by a mostly Microsoft-unfriendly crowd. Regardless of your good intentions, you need to be extra careful to color inside the lines, or you risk alienating the users you seek to eventually please.
So then, specific reasons why these changes are already dangerous (and why it’s a good thing if you’re planning to remove them):
- RubyGems and other libraries install their own require hooks. If you are enhancing the capabilities of require, you are very likely breaking those libraries. And with RubyGems such a key, core framework, that seems really dangerous to do.
- initialize has a very well-understood meaning in the Ruby world, and having it be public and callable, much less having it set up a DLR component in a Ruby-incompatible way, goes against that meaning. Perhaps you have a DLR-level or SilverLight-level limitation here, but you’re really asking for trouble making this kind of change.
In general, if you want to be compatible with Ruby, you’re going to have to consider Ruby’s needs first, and DLR/SilverLight’s needs second. And yes, that means considering the community you want to support over your employer, if their goals do not follow the same paths. I hope you will be able to do that, and I hope you’ll let me know how I can help.
John, you know my opinions on this, and I agree very much with Charles. The Ruby community wants you to succeed in building a totally compatible Ruby implementation and we other implementors would like to help you as much as possible.
Thanks for the kind words of advice, Charlie.
In the case of initialize, how would you suggest handling it? This is calling a JS function, not a Ruby function. Since it’s perfectly legal to define a method called initialize in JS, how should this behave – especially since we’re invoking it through an explicit ScriptModule qualifier.
Re: initialize.
Here’s a related case in Jython. “print” is a Python keyword, but Java’s System.out has “print” method. Jython appends an underscore when Java method names conflict with Python keywords.
Thanks for explaining it, John. I hope that your approach and outreach to the Ruby community can result in a uniform and compatible solution across all Ruby implementations.
Hi John
I think striving for (at least partial, or even with lower performance) compatibility with native libraries (including gems) is strategic.
I for sure realize it’s a challenging task , but it would change the possibilities by an order of magnitude – and the market share too.
kind regards – Thibaut
What does it mean for an open source project to be in stealth mode?
Perhaps I am mistaken and ironruby is not open source.
Hi John,
I think that it would be a better idea to put the “new features” in a separate module. Otherwise, it will be hard to port code written for IronRuby to another Ruby implementation.
@Malcontent,
Wow, you just like to travel around the blogosphere and attempt to stir up trouble, don’t ya.
Are you of the belief that a project must be broadcast from the mountain tops well ahead of the time that its ready to be broadcast to qualify as open? Is a birthday party not really a birthday party if it’s a surprise birthday party?
Open or closed source, strategy matters.
Well, in Facets there’s
JS = Script.new(‘foo.js’)
or
module JS
module_require ‘foo.js’
end
I’ve been looking for a better name for Script though. ScriptModule looks good.
T.