John Lam on Software

Dynamic languages on .NET are picking up momentum at this year's TechEd. Your typical TechEd attendee is a mainstream .NET developer, since this conference focuses on technologies that are shipping today (as opposed to futures conferences like the PDC). To speak more to this crowd, I focused on how they can make their existing .NET apps better by mixing in some end-user scripting. My talk was DTL332 if you're a TechEd attendee (not sure when/if the videos will open up to the general public). I showed how you can add Ruby and Python scripting to an existing app, and spent some time building some simple REPLs. Toward the end of the talk, I showed a more realistic scenario where I embed a REPL in an existing Open Source .NET application: Witty. During the demo, I showed off a twist on the traditional REPL - one where the editor and the REPL are one and the same. To run code in this REPL, you select the code that you want to run, and the output appears immediately below the selection. Since the REPL is just an editor, you can just delete any output that you don't want. History behaves just fine as well. Here's a screenshot of the REPL in action: There are a couple of other notable things in this REPL. First, it syntax colors the text that ran correctly. Second, it supports more than one .NET language - you can switch between Python and Ruby using %python and %ruby (although whether that's a good idea is left to the reader). Third, all of its configuration is self-contained inside of a single DLL. This means that all you need to do to add scripting support for your app is add a reference to a single assembly. The assembly will write all user configuration files to well-known locations on disk (within your HOME directory) the first time it is run. This idea of a REPL within an editor buffer isn't new by any means. I've been spending some time hanging out with Mark Hamburg recently, and this is his favorite way of implementing a REPL (he's done quite a few, including the embedded IDE that the Adobe Lightroom folks use to build Lightroom). He won me over on the idea, so I thought that I'd hack up an implementation for folks to experiment with. I pushed the code up to Github as the repl-lib project, so feel free to fork, play with the code, and send me some feedback on the idea. Note that there's a nice long tail of bugs in this REPL, so feel free to fix bugs and submit changes! Here's a screenshot of the Witty main window; notice the console button: Here's a sample console session: The last line of code causes the Witty UI to display: After poking around inside Witty, I think there is some work that needs to be done to make scripting truly a first-class citizen in the project. While some might look at this as bad news or an indictment of Witty, it really isn't. Any project that isn't designed for such an invasive feature will likely make lots of assumptions that make it hard to make that feature successful. But the good news here is that if you use scripting as a end-user feature to drive your refactoring efforts, you will wind up with an app that is easy to script, but more importantly, is better designed. I argue that your design is better because it's forces a clean separation of concerns in your code: especially your models and your views. This separation of concerns will make it easier to test your end-to-end scenarios and script your integration tests. This all adds up to better software for everyone. I've made it pretty easy to add scripting support to your app. Grab the repl-lib project and spend a few minutes (that should be how much time it takes) to integrate scripting into your project. If you're interested in how much work it took to integrate these changes into Witty, check out this diff; most of the stuff is VS-related, and ReplResources.xaml is no longer required in current builds of repl-lib:

As usual, PDC 2008 was a great event. I had a chance to meet a lot of old friends (it’s been 7 years since I’ve been to a PDC- it attracts a certain crowd that only shows up there) and I’m happy to see that everyone’s doing well. My IronRuby talk was the first day of the conference. It was a crazy mixture of many, many demos (9 in all) which showed: - when and why you should create a new type in IronRuby using C# - how to write a Visual Studio plug-in using Ruby - how to build a unit test and mock object framework - how to integrate Ruby scripting into your existing C# application - how to build simple web services using Sinatra Only one blew up (the last one) when I forgot to change the type name in my ironruby_mischief.rb file to the type that I created in the IronRuby libraries. You can watch it on TV now (Silverlight required).

Brief Rant:

Session evaluations are a great way to give direct feedback to the teams. Sure, you can send email to the speakers directly, but session evaluations get circulated fairly broadly internally and are often used by management for future product plans. They are a great way for you to influence the way we plan and prioritize the work that we need to do. But there’s really not a lot in it for you, the attendee, outside of maybe some personal satisfaction that you helped to influence the products that you use (that’s if you knew how influential they can be in the first place). Here’s my proposal:give direct incentives for folks to fill out their evaluations. It can be something as simple as 1 eval == 1 ticket for a drawing for prizes at the end of the conference. The more evals you submit, the greater your chances for winning prizes that are donated by the sponsors. You could do things like what Stack Overflow does: give merit badges to folks that provide quality feedback. Written comments are always preferable to just clicking on radio buttons, so you could earn more merit points by doing just that. The whole system is online anyway, and it wouldn’t be that hard to implement. You could even do something simple like write 5 evaluations to get your conference T-shirt. That should significantly improve the response rate, considering what the awesome power of T-shirts are in the geek world :) I don’t believe that this would skew the feedback in a significant way. Hopefully we’ll hear more from the ‘silent majority’ this way. For example, I had 245 people in my IronRuby talk and as of this writing, I only have 15(!) evaluations. If you read this far (and you attended my talk at PDC) please click here to submit your evaluation. Apparently that link is some kind of one-time link. You'll (unfortunately) have to go to the PDC site and navigate to my session (TL44).

What I liked:

I loved the Big Room. It was a place where folks could gather to meet and mingle. Since I want to talk to customers at the show, I was happy to spend virtually all of my time in the Big Room talking to customers. But there was one talk that I did attend: Miguel de Icaza’s awesome Mono and .NET talk. Horrible title – it really should be called “Awesome Mono hacks by Miguel and his band of merry hackers”. Here’s the coolest thing that he showed: Mono applications running on a non-jailbroken iPhone. Yes, you heard that right. For those of you who aren’t in the know about these things, the iPhone SDK prevents you from writing a JIT compiler through some kernel restrictions (you can’t mark a writable page as executable). It also prohibits you from writing an interpreter so that they can maintain their lock on application distribution via the App Store. So how did they do it? They made it possible to compile an entire Mono application into a single binary executable which gets signed via xCode and downloaded to the phone. This opens up some pretty awesome opportunities for using .NET as a platform for building apps that run on the iPhone. He also showed a very cool C# REPL in action. There are some cool UI ideas from his REPL that are going to find a nice home in the IronRuby REPL :) Lots of fun banter during the talk as well. Click here to watch it on TV (video not there right now at the time of this writing, but should be there soon). I had a chance to meet a lot of folks, and once my talk was done, I decided to try and shoot photos of as many of my friends as I could. I’ll continue to upload them to my PDC 2008 flickr photoset as I process them on my laptop. More later as I continue to collect my thoughts.

I had the honor of meeting with a delegation from Fukuoka, Japan this morning at Microsoft's Executive Briefing Center. Part of the delegation represents the Ruby Business Commons, one of the largest Ruby SIGs in Japan (> 500 people as of August 2008). They were here to talk to us about Ruby, its impact in Fukuoka, and how we can work together to help promote Ruby and IronRuby in the IT industry in Fukuoka. It was a challenging meeting since everything had to be done through a translator (who did a remarkable job). They had some great questions and suggestions for how we can work together to help promote Ruby in the Enterprise in Japan. I'm looking forward to seeing what we can do together to help spread the love.

OSCON is a happy place for me. It’s my third OSCON, and I created good memories at each one. At my first OSCON, I was showing off RubyCLR to the world, and I met some folks from Microsoft who convinced me that it was the right time in my life to go ship something big. At my second OSCON, I was working for The Man, and we announced that were accepting contributions into IronRuby – a big change at the time. Today, we’re raising the bar even higher in IronRuby. There are three changes happening today: 1. We’re shipping our first binary release. In this package, we’re taking a “batteries included” approach and shipping the Ruby standard libraries in it. This takes us a big step closer towards “IronRuby runs real Ruby programs”. 2. We’re announcing a new project: ironruby-contrib. It’s hosted at GitHub, an awesome place to do collaborative development. The goal of ironruby-contrib is to provide a place where folks from the community can participate in projects that enhance IronRuby or its underlying platforms. The first project in ironruby-contrib is the Rails plugin that we demonstrated at RailsConf this year. It makes it easier for Rails developers to add Silverlight to their Rails applications, and contains some of our ideas about migrating controller code to the client. Jimmy Schementi is the brains behind the project; see his introductory blog post for more details. 3. We’re announcing that we’ve submitted our first set of changes to the awesome RubySpec project. We’ve been talking about contributing to RubySpec for a long time now, but various things kept us from doing so. We’ve fixed those things now, which takes us a big step closer towards being an active participant in all things Open Source, and not just the things that we create.

At Tech Ed last week, I showed a prototype of integration between IronRuby and ASP.NET MVC. I was really happy about how straightforward this turned out to be; it’s a strong testament to the robustness of our hosting API that Levi on the ASP.NET MVC team was able to add IronRuby integration with only a simple example and the existing docs. Let’s walk through a simple example. Keep in mind that this is a prototype, and that many things will likely change before we’re done with it (particularly some of things we do today with globals and instance variables). First, let’s look at how you can define some default routes via routes.rb: Next, let’s look at the Controller code for the HomeController: The index action renders a view using a trivial Model which lives in models\HomeModel.rb: This model is used by the index action View, which lives in views\home\index.rhtml: Next, let's look at a slightly more sophisticated example using a Products Controller that renders a list of categories that looks like: It also renders a list products for a category when you click on it: This is what the controller looks like: There are two actions in this controller: index and list. The index action returns a list of the categories, and the list action returns a list of products for that category. Note that we're using instance variables on the controller to communicate with the view - this is likely going to change in the future, but it was something we hacked in to see what it would look like. If we look at ProductsModel, you'll see that it contains a simple adapter around a Visual Studio generated wrapper for the Northwind database: We had to write our own adapter to work around a few bugs that we have in our .NET interop story as of right now, as well as adding a few model-esque methods. Here's what the NorthwindDatabase adapter looks like: Notice how we can use LINQ to SQL to write our queries to the database, and the funky C# 3.0 anonymous delegate syntax that we use for GetCategory(). The interesting code in the view looks like: Today we're using the MVC template engine, but there isn't any reason why we couldn't enable erb / haml etc. in the future.

How can I get it?

You can download a ZIP of the entire project from here if you want to take a closer look. Note that you can't run these bits yet, since I demo'd it using a private build of ASP.NET MVC from mid-way through their current milestone. That said, when Preview 4 of ASP.NET MVC ships you'll be able to run IronRuby on top of it (their current release is Preview 3). Look for a blog post here, and from the folks like Phil over in MVC-land when we're ready to ship you some bits to build MVC apps using IronRuby.

I’d like to start out by thanking everyone who came up to talk to me at the conference. We had a Dynamic Languages booth at the conference, and it was great to talk to all of the folks who took the time out to drop by for a visit.

A big thanks goes out to our booth staffers – Dave Fugate, Curt Hagenlocher, Jimmy Schementi, Bill Chiles and Mahesh Prakriya who helped to keep things running smoothly. Dave somehow managed to grab an enormous whiteboard from somewhere and used it to help answer many of the FAQs for folks who were hanging around waiting for a blue-shirted guy to become available.

Tech Ed is (not surprisingly) a very enterprisey conference. We spent at least as much time telling folks what Dynamic Languages were (or correcting their misperceptions) as we did talking about our team’s progress. We also had a whole bunch of folks come up to ask us questions about F# because, apparently, F# is now a dynamic language :)

By far the most frequently asked question was why dynamic languages? The advice that resonated the most with the folks that I talked to was about using it to help them build internal DSLs for their applications. This is the idea that you can use DSLs for part of your app, dynamic languages to help faciliate those DSLs, and statically typed languages for the foundational pieces. You'll find that this idea is often called polyglot programming.

The Talk

The IronRuby talk was the very last talk of the conference. Apparently I was one of the ‘big guns’ that would convince folks to stay all the way to the very end of the conference. While I prefer being at the start of the conference so that my talk can help drive further discussions with folks while they’re still there, it is what it is. Oh well, let’s see if the tubes can help start a conversation :).

My talk focused on three things. First up was an introduction to Ruby since half of the folks in the room hadn’t used Ruby before. I wrote a simple unit-testing framework live on stage. Each new feature in the framework helped to introduce a different feature of Ruby. At the very end of the talk I showed how that same framework could be used to test .NET code. Here it is in its entirety:

And here is some .NET code that we tested using this framework:

It worked rather well, and I think I’ll continue to use it at talks where I need to introduce Ruby to the audience.

Next, I talked about our Silverlight integration. I showed the excellent set of demos that Jimmy created for RailsConf last week, including his client-side Try IronRuby demo:

and the lovely Silverlight watch demo:

I also got a chance to sneak in some of the code I was hacking on in the evenings – an adapter that maps the HTML 5 Canvas API to Silverlight. I ran a few of the examples from the excellent Mozilla Canvas tutorials using a my adapter. I pasted some code (with minor rubification) from the Mozilla tutorial into the text box:

and hit run - the colored boxes with the translucent circles appeared below:

There's no reason why we couldn't do this with managed JavaScript either using the cross-language interop features of the DLR.

Oh yeah, there was one more thing that I demo'd - more on that tomorrow.

IronRuby dispatched some simple requests through an unmodified copy of Rails a few days ago. Today, we’re going to show off our progress live at RailsConf. This is an important milestone for IronRuby; it’s our ‘ticket to entry’ to the world of alternative Ruby implementations.

We started our work on IronRuby back in February 2007. Now, just 15 months later, we’ve reached what others are calling the “Rails Singularity”. A few folks claimed that we would never get here this quickly, or that we wouldn’t be allowed to accomplish this goal. But we did it on our own, in our own way and with help from our community. And we’re just getting started.

I have always maintained that you must judge us based on our actions and not our words. Running Rails shows that we are serious when we say that we are going to create a Ruby that runs real Ruby programs. And there isn’t any a more real Ruby program than Rails. This demonstrates that we’re true to the language, and that we’ve put compatibility above all else on our TODO lists.

But we have a lot more work to do.

Our performance is nowhere near where we expect it to be, particularly in startup of a large application like Rails. We are consuming much more memory than we would like to. But this is the price you pay when you put compatibility ahead of all other work. We’ve shown that we are willing to do what it takes to run Rails. Now we have to do the work to make it run better, and faster.

But there are other things to talk about as well.

IronRuby doesn’t just let you run Rails; it lets you interact with the rich set of libraries provided by .NET. You’ll be able to use IronRuby to build server-based applications that run on top of ASP.NET or ASP.NET MVC. You’ll be able to use IronRuby to build client applications that run on top of WPF or Silverlight. You’ll be able to use IronRuby to test, build and deploy your .NET applications. You’ll be able to run Ruby code in your web browser and have it talk to your Ruby code on your web server. That’s a feature that we feel that many folks will enjoy.

Perhaps even more important than all of this technical stuff is what the IronRuby project represents at Microsoft. IronRuby has pioneered a number of new processes that make it easier for other folks at the company to build and release Open Source products. What we learn from building IronRuby will be applied in other product groups to help us become more open and transparent than we have been in the past. We have a great leadership team that is willing to push the envelope on openness and transparency to create a world where both Microsoft and our customers can benefit.

Come join our project on Rubyforge and help us show everyone how it’s done!

OK. So here’s the scoop. 2pm at the International Meeting Place (see map).

I’m going to be at the Convention Center from around 1:30 onwards. There are a lot of central public meeting places at the convention center. From where I sit at my desk this morning, it looks like the “International Meeting Place” on the second floor will do just fine:

I’ll hang out there and I’ll be happy to demo / talk about IronRuby, OpenSource and whatever else *you* want to talk about.

Follow me on twitter (john_lam) if you want up to the minute updates on where we’ll be just in case this location doesn’t work out.

I’ll be giving talks on Ruby in both the C# and VB tracks. Right now it looks like 10:30AM and 12:30pm – check your schedules to make sure – the second talk’s time slot looks fishy to me.

BTW, for those of you who are reading this who don’t know what an MVP Summit is, it’s an event where we fly put up in hotels and feed some of our closest supporters to Redmond for a week-long tech love-fest. This is an awesome event since we get a chance to give back to the folks who help us do our jobs here at Microsoft.

I have a simple request for you if you are:

• an MVP (MS Most Valuable Professional)
• are attending the MVP Summit in Redmond in April
• want to talk about IronRuby *or* dynamic languages in Silverlight in an ‘Open Spaces’ format

Please go ahead and nominate us for one or both of these sessions and I’ll show up along with Jimmy Schementi to discuss at the MVP summit. Take advantage of the Open Spaces format – you get to control the agenda!

BTW- please leave a comment here if you want to nominate so that we’ll be sure to show up then!

Update: Apparently the folks running the “Open Spaces” event at the MVP Summit want to exercise central command and control over the event, quite unlike this definition of a BarCamp from Wikipedia:

“The procedural framework consists of sessions proposed and scheduled each day by attendees, mostly on-site, typically using white boards or paper taped to the wall. This has been dubbed, with another play on words, The Open Grid approach.”

Apparently nominations are now closed, so we won’t be there to participate, sorry.

Update 2: OK. So we’re going to stick it to the man and take matters into our own hands. Follow me on twitter, and we’ll figure out a place to do our own Ruby meetup at the MVP summit.

Yep, that’s right. That’s my other job here at the company. This means that I participate in a set of events that have nothing to do with IronRuby as a technology, but have everything to do with IronRuby as part of a movement toward greater openness within the company.

Last week, I participated in the Microsoft Technical Summit that we held here on campus. Every year we invite a bunch of Microsoft skeptics to campus and subject them to mind conditioning engage in a dialogue with them. I talked about why we were doing Open Source, why we were doing dynamic languages in particular, and showed them a few demos of stuff that works today. It was great to get blunt feedback from folks who took time out of their lives to attend, and hopefully we did move the dial on their perceptions of what we’re up to here at the company.

I had a lot of fun talking to Adam Keys who rocked my world with his RubyConf one-man play (warning – you need to either be a Ruby programmer to really appreciate the crazy humor that this is, or be fascinated by what geeks think is funny):

On Friday, I participated in our inaugural Open Source Day internal conference at Microsoft. I was on a panel with three other folks: Rob Mensching, who did the first Open Source project at Microsoft – WiX, Shawn Burke, who runs the AJAX Control Toolkit project, and made the .NET library source code available among many other cool things, and Tom Hanrahan who runs our Linux Interoperability lab. We talked about experiences – Rob and Shawn have been at the company a long time and had a ton of fun anecdotes about what it was like to try and do Open Source at the company back in the dark ages. I contributed some stories about how we do IronRuby development and some pointers about how other product groups can think about why and how they should participate in Open Source. Tom was our elder statesman, and talked a lot about why interop is important to our customers (bottom line is that virtually all of our medium to large customers live in a heterogeneous aka non-100% Windows environment).

One thing that came out in the discussions is how we need to be better at transparency, even while developing our non-Open Source products. One of the powerful ideas of Open Source is the ability for outsiders to actively participate in the creation of products even if they never crack open the sources themselves. That’s a powerful idea, and one that I think that (at least in Developer Division – where I work) we’re in a great position to deliver on.

*

I’m a huge fan of vimperator after discovering it via Zed Shaw. If you’ve internalized the vim keybindings, you’ll be surprised at how you can leverage your muscle memory while surfing the web.