IronRuby at OSCON 2009: Mono, Moonlight, and scripting open source apps

Today I’ll be giving a talk at OSCON 2009 titled “IronRuby 0.9”. For any new readers, IronRuby is an Open Source implementation of the Ruby programming language for the Microsoft .NET Framework; it brings the expressiveness of Ruby to .NET, and the capabilities of .NET to Ruby. I’ll walk you through the talk in this post:

• Road to IronRuby 1.0
• Running the same application on the desktop and in the browser, on most major platforms (Windows, Mac OS, Linux) and most major browsers (Firefox, Safari, Internet Explorer)
• Detour – Gestalt: Running Ruby and Python in the browser, again.
• Using code written in any C# and Python from Ruby without it feeling foreign.
• How IronRuby has improved in the past 6 months, and how you can help IronRuby become a 1.0 quality implementation.

Road to IronRuby 1.0

It’s been a long ride since IronRuby was announced alongside Silverlight at MIX ‘07, but in a few days the 0.9 version will be available, and all eyes will be focused on getting it to 1.0. The past couple of versions have made vast compatibility and performance improvements, and 0.9 marks the last set of major features that will be added to IronRuby. From now on, all IronRuby languages will be primarily focused on bug fixing and anything else the community deems necessary to call IronRuby a 1.0 release. These improvements be delivered as point-point releases (0.9.1, 0.9.2, etc) until you decide it is ready to call 1.0.

But we’ll dive into the numbers and metrics at the end; let’s talk about why IronRuby is useful in your toolkit.

Cross-platform, cross-browser Ruby implementation

Whenever people hear .NET [they first think about a top-level domain, but when in the context of “Microsoft”] they immediately think “Windows”. But Miguel and the Mono team have shown that isn’t reality. They’ve built a open-source implementation of .NET which runs on Linux, Mac OS, and also Windows. They are also building a open-source Silverlight 2 implementation, called Moonlight, allowing Silverlight applications to run on Linux browsers supporting Mozilla’s plug-in architecture; Silverlight 2 runs on Intel Macs and Windows. Since IronRuby runs on .NET and Silverlight, it also runs on Mono and Moonlight. Let’s explore that a bit:

With Microsoft.NET and Mono, IronRuby can run on Windows and Linux (anywhere Mono runs, including Mac OS).

You’ll notice that real Ruby code runs, it’s running on a Unix environment on Mono, and .NET types can be accessed in a Ruby-esk way, even generic types.

Because IronRuby integrates directly with .NET, you can use any existing .NET code from IronRuby. This means that any frameworks, be them Windows Forms, Windows Presentation Foundation (WPF), or GTK(#), can be used from IronRuby because .NET and Mono [respectively] have CLI bindings for these frameworks. Mono even has a Windows Forms implementation that translates to GTK, so the same application can run unmodified on both implementations. Mono only supports the “Silverlight” subset of WPF, which I’ll elaborate on in a bit.

There’s also a tiny subset of .NET which is part of a browser plug-in called Silverlight; Microsoft’s implementation runs on Windows and Mac, in Firefox, Safari, and Internet Explorer (though any browser which supports ActiveX, Mozilla’s, or Safari’s plug-in architecture should work fine, like Opera or Chrome).

The IronRuby tutorial application runs as a WPF desktop application (on the left), as well as Silverlight application (on the right).

Also, Silverlight 3 supports running applications out of the browser, very simply on Windows and Mac OS by right-clicking on the Silverlight application and selecting “Install”.

Through Moonlight the IronRuby tutorial app also runs on Linux:

Mono also supports running Moonlight applications through GTK on the desktop, just as a WPF application would work. While direct support isn’t in current daily builds of Moonlight, Miguel shows how easy it is to make the same code target the browser with Moonlight and the Linux desktop with Mono, just using C# code:

Note: the latest Moonlight stable release will not work for IronRuby, you must install the latest Moonlight build from their build bot: http://sublimeintervention.com:8123/GetFile.aspx?id=11487919

Detour: Gestalt –- Ruby and Python in the browser

My last blog-post talks about a new prototype from the MIX Online Lab called Gestalt, which lets you write Ruby and Python in place of JavaScript – literally in <script> tags. Here’s a very simple example:

It uses Silverlight to do this, so this will work in Moonlight as well.

Dynamic and Static language interop

IronRuby has great integration with the .NET framework APIs, so C#/VB code can be called from IronRuby without it feeling like “interop”. C# can also call IronRuby through the Dynamic Language Runtime (DLR) Hosting APIs. However, in .NET 4.0 dynamic method dispatch is part of C#, so calling IronRuby from C# is just as easy as a C# method call. Being that IronRuby is built on the DLR, it can interoperate with other DLR languages just as easily, so Python and Ruby play along nicely today, and future DLR languages as well.

A simple way to show that capability is by writing a unit test for a class in .NET; let’s pick “Stack”:

As you can see, creating .NET types, calling methods on .NET classes, and all the other language features Ruby has work against .NET types (even re-opening .NET classes).

To show the reverse, lets look at a very simple REPL implementation:

In less than 100 lines of code, you have a program that runs arbitrary Python and Ruby code. The DLR hosting API lets you run dynamic language code from a static language just from strings.

Expanding on these ideas, let us give the experience a upgrade to the 21st century. John Lam has shown this before at TechEd 2009: an editor that can double as a REPL.

Downlaod it here

This seems like a great idea, and it is even better once you start using it. It works like a normal editor, but selecting a block of code and pressing ctrl-e will run the code, showing the output directly below selected code as a comment, and ctrl-enter runs the current line. The code that gets run gets syntax highlighted. There is no need to have up-down-cursor history, as it’s all right there in the editor. If you need to modify something you’ve already run just, just modify it in the editor and re-run it! This integrates the editor-repl experience, making it a very valuable tool.

Note: this is not a new idea by any means, as TextMate does something similar, but no one seems to explain this as an upgrade to your traditional REPL.

Using this editor, it can be integrated into any .NET project as a lightweight way to experiment with adding scripting support. Using the hosting ideas I presented before, an application written in C# could use IronRuby to allow users to write applications against their object model, or extend it at runtime. Here’s a small example of hosting this editor in Witty, an open-source Twitter client running on .NET:

You can get this version of Witty from GitHub.

While it is great that you can so easily add scripting into a .NET application, it ends up exposing the design deficiencies in the application being scripted. In Witty’s case, the Twitter class (T) is very good, but the Window class (U) isn’t in good shape, as there are a ton of methods on the type, and it’s unclear how they should be used to drive the UI. Jon Galloway wants to merge this into Witty, so hopefully some of these refactorings can be made to enable scripting better.

I extend this offer to all open-source .NET/Mono projects – if you want to improve your architecture and add scripting support, please let me know and I’ll help out.

It's worth noting that .NET 4.0, the next version of the .NET framework, C# supports a "dynamic" keyword which will let you call Ruby or Python (or any DLR language) just by using method calls.

Project status

Compatibility is really good at this point. We run the RubySpec test suite, passing at (85%) and the test suites from popular Ruby frameworks, including Rails, Rake, RSpec, and RubyGems, on every check-in, and can run pretty substantial Rails applications. Increasing compatibility bugs will definitely be the focus of the point releases approaching 1.0; mainly based on what you tell us needs fixing.

This upcoming version will boast support for the Win32OLE library, making existing Ruby applications with interact with COM work as they do in MRI, like Watir.

Performance is also in a very good place. This past version has pushed on the Ruby benchmark suite, and now we are >= 2x better than MRI on almost all benchmarks. Those numbers will be published on http://ironruby.info around the same time IronRuby 0.9 comes out. The above image shows that IronRuby (in blue) is normally significantly faster than MRI (in red), except for a handful of cases. The is not the end-all-be-all metric to measuring performance, but it’s nice to see IronRuby continuing to improve here.

In prior releases, Rails startup has gotten dramatically better, going from 80+ seconds in March to ~20 seconds now. Read more about Rails performance is the IronRuby 0.6 release.

Interesting IronRuby usages

Now that IronRuby is approaching 1.0, we're seeing very interesting usages of Ruby with .NET:

• AutoCAD scripting -- Kean Walmsley
• magic DSL for Windows UI frameworks -- Thibaut Barrère
• Automated conversion of an app written in an in-house language (lang runs on FORTRAN77) and MOTIF UI “to” IronRuby and WPF -- Robert Brotherus
• Point of Sale application based on WPF, IronRuby and FoxPro -- Nick Rickets

If you know of more interesting usages, please send them my way!

Thanks!

Thanks for reading this far! Keep an eye out on this blog and the ironruby website for the upcoming 0.9 release.

Technorati Tags: IronRuby,OSCON,Mono,Moonlight,Silverlight

Gestalt: Ruby and Python in the browser, again

Today Mix Online Lab released a prototype implementation of Ruby and Python in the browser, named Gestalt. Feeling a bit of de ja vu? If not, you can ignore this post and just see what it’s all about. Regardless, I suggest you read on.

A simple Ruby-based web page using Gestalt would look like this:

Gestalt adds “ruby” and “python” to the languages supported by the <script> tag. You can also include other files rather than writing the code in the HTML page:

My first reactions when hearing about this project was “Holy crap, wow, they got Ruby and Python running in the browser, that’s freakin’ awesome!” But I work on the Silverlight integration with IronRuby and IronPython, already letting you write Ruby and Python in the browser instead of JavaScript. Hmm. Not to my surprise, Gestalt uses IronRuby and IronPython in Silverlight to accomplish this. But damn, wouldn’t it be awesome if this is how the world worked?

For any new readers, IronRuby and IronPython are implementations of Ruby and Python for the .NET Framework, utilizing the Dynamic Language Runtime (DLR).

Gestalt is a effort to show what it would be like if there were other standard languages in the browser, other than just JavaScript. The Mix Online Lab focuses on using Microsoft technology in ways that make non-Microsoft developers interested, so this project is right up their alley. They wanted to showcase the DLR languages in Silverlight as a way of showing how our efforts can make the web a better place.

So what are the difference between IronRuby and IronPython’s Silverlight integration and what Gestalt provides?

Again, you can already make IronRuby and IronPython based applications in Silverlight; Gestalt just puts a new spin on how to do that. IronRuby and IronPython use the XAP file for the application packaging, including script files, and provides a development-tool called Chiron to auto-package the app, providing the edit-refresh experience that is familiar for a web browser app. Chiron is also used to write the package to disk so it can be deployed to any web-server. This model works great for getting new and existing Ruby and Python programs running in Silverlight, as the packaging is transparent to a developer. And it doesn’t fight Silverlight’s packaging conventions, so using Silverlight-specific features will be straight-forward from a DLR language.

Gestalt takes a much different approach using <script> tags, making the integration much more familiar to current JavaScript developers. They also handle all the Silverlight <object> tag stuff in gestalt.js, taking away the need to host Silverlight yourself.

Why Gestalt is interesting to me

The thing that I really like about Gestalt is that Chiron is not needed for development. Just edit the file and refresh the browser. No need to run something in the background generating a XAP file. Gestalt.js detects which languages the page’s <script> tags use, and picks a pre-packaged Gestalt-*.xap file, one for Ruby, one for Python, one for both, and one for none. This means your application could look something like this:

• index.html (can contain Python and/or Ruby code)
• gestalt/
  • gestalt.js
  • gestalt-rb.xap
  • gestalt-py.xap
  • gestalt.xap
  • gestalt-x.xap

Very clean. The gestalt directory is just dropped onto your web-server (to be shared among all your applications), and gestalt.js is referenced from your HTML file, and your good to go. No running Chiron.exe, or installing Mono on the Mac to run Chiron.exe. I’m jealous.

So what’s the catch?

While Gestalt is a much simpler to get started writing Ruby and Python applications in the browser, it does not support a major feature of Ruby and Python – accessing something like a file-system. For example, including other files at runtime uses the language’s underlying File access to read the file in and run it’s contents. Ruby uses “require” or “load”, and python uses “import” to do this.

IronRuby and IronPython in Silverlight normally support this, as all file reads are redirected to the XAP file. Since Gestalt’s XAP files are opaque to the the developer, they do not support File access. This is only an issue when you want to use existing Ruby or Python code in the browser. For example, existing code usually has more dependencies, especially into portion of the language’s standard library which is written the language itself. Usually you’d just put these files in the folder that Chiron is auto-xapping for you, and that’s it. However, Gestalt has no way of doing this today.

Making it better

A possible solution is to just cache the contents of <script src> tags, and use the languages path resolution mechanism to test if those files have already been downloaded. If it finds a match, serve the file’s contents. If not, it could either fail fast and say that your missing a <script src> reference, or it COULD download the file synchronously (and still cache it) and display a warning that you are missing a <script src> reference.

Unfortunately, this only works for script files; what if you just want to download a txt file? Having <script src=”foo.txt”> is not going to fly. Maybe using <link> tags could work?

Why can’t we just download them synchronously? No! Though the current bits do have a synchronous downloading mode, it’s only experimental and should not be used in production. Synchronous downloads are significantly slower since only one download can occur at a time. Downloads need to be synchronous since “opening a file” is expected to block until it is complete. Also, IronRuby/IronPython do not support continuations (for good reasons), so the code cannot be “paused” while a file is downloaded, and then be “resumed” when done. And even if they did support continuations (as there is a way to get it working for the very specific case of file access), the downloads would still all occur one at a time, making the loading much slower. So downloading “on-demand” for file reads is not a good solution, and that’s precisely why the browser doesn’t do that today … so let’s not fight it.

Is Gestalt going to always be separate from IronRuby and IronPython?

No! :) While Gestalt’s first release was best to be separate, I’m very interested in merging the ideas Gestalt has back into Microsoft.Scripting.Silverlight … in-fact discussions have already begun on the IronPython Mailing List.

In short, Ruby and Python Silverlight applications are a bit too complex, and Gestalt shows us that it doesn’t need to be that way. Work has already begun to merge Gestalt and Microsoft.Scripting.Silverlight, so play with Gestalt and let me know what you think!

Technorati Tags: IronRuby,IronPython,DLR,Silverlight,Browser,Ruby,Python,dynamic languages

IronRuby 0.6 Released!

The IronRuby team is pleased to announce a new release of IronRuby: IronRuby 0.6!

Download IronRuby 0.6

You can also get the source code for this release

Staying true to our "Conference-driven development schedule," this release is in conjunction with ... wait, there is no conference this time! Just a good ol' monthly release of IronRuby, chock full of awesomeness. And just so happens to be the 4th of July weekend, so hopefully you have some downtime this weekend and can give the new release a spin.

What's in the Release?

Performance has been a major focus to this release, getting startup time quicker and optimizing built-in types. There has been more progress with Ruby compatibility (Cucumber works!) and .NET interop. Silverlight binaries are back in the release, and there are a couple of samples to help you learn IronRuby.

Most of these descriptions are from Tomas's very detailed code review emails, so thanks Tomas! For more detailed information, please see the CHANGELOG (which includes all commit messages for the release ... not just "syncing to head of tfs")

• Lazy method compilation
• CLR member enumeration
• Generic-Method Parameter-Inference
• Testing C# with Cucumber
• Silverlight building and updated binaries
• Samples

Lazy method compilation

Last release adaptive compilation was enabled for IronRuby, which uses a fast-to-start-up interpreter to start up the program, while code which gets run a lot is compiling on a background thread. This gave IronRuby significant performance improvements for large apps like Rails, which has improved startup by 2.5 times.

This release we've added "lazy method transformation" to the bag of startup performance tricks. In IronRuby 0.5, a method was transformed to the DLR AST as soon as it was defined. IronRuby 0.6 postpones the transformation until the first time the method is called. This significantly improves startup time. For example (not NGEN'd):

This has made Rails startup time 30% faster, coming in approximately 20 seconds on my dual-core laptop running Windows 7.

CLR member enumeration

Now methods like Module#instance_methods, etc, include CLR member names. Though it's a lot cooler than that. The array of methods returns contains strings for Ruby methods, and objects of type IronRuby::Clr::Name to represent CLR methods that can be called by either the actual CLR name or a mangled (Ruby-esk) name. ClrName has methods to_s, to_sym, to_str, <=>, inspect, and dump so that it can be used wherever a string can be used. The display string for the name uses single quotes so that you can easily distinguish CLR (dual) names from regular names (plain mutable strings). CLR strings display themselves as single quoted strings, so this fits nicely.

Even cooler, this works well for meta-programming:

A new set of define_method overloads, strongly typed to ClrName, have been added to enable this. They define the real method using the ruby_name and alias it using the clr_name. So both CompareTo and compare_to calls are intercepted.

Generic-Method Parameters-Inference

Thanks to some work by the IronPython team, IronRuby now has parameter inference for generic methods.

This needs some more TLC before it works perfectly, but this lets you use simple LINQ methods from IronRuby.

Testing C# with Cucumber

This release of IronRuby runs the Cucumber testing framework rather well. Try out Cucumber against some C# code.

Given a feature file (this being the addition.feature)

A step_definition file (calculator_steps.rb)

And a DLL:

Cucumber will test the addition feature:

3 scenarios (3 passed)
12 steps (12 passed)
0m0.753s

Silverlight building and updated binaries

This release re-adds Silverlight binaries to IronRuby, contained in the "silverlight" directory of the release. These bits have been integrated from the AgDLR project, and will be maintained in the IronRuby and IronPython source code repositories from now on. The AgDLR GitHub project will redirect to one of those for binaries for both languages in Silverlight. In addition, the SDLSDK CodePlex site will be redirecting to the IronRuby and IronPython CodePlex sites, for downloads, discussion, and issue tracking. AgDLR was a little side project to add new features to DLR Silverlight applications, and play around with Git. I'm very happy to see it merged back in with the languages.

Note: IronPython Silverlight assemblies are included directly in the release. They may be moved out to the separate package that contains IronPython desktop DLLs in the future.

See the integration commit for more information

Another notable change is that you can now build Silverlight binaries out of IronRuby's GitHub repository very easily. Given that you have Silverlight installed at C:\Program Files\Microsoft Silverlight\2.0.40115.0, this will build IronRuby for Silverlight:

msbuild Merlin/Main/Languages/Ruby/Ruby.sln
  /p:Configuration="Silverlight Release
  /p:SilverlightPath="C:\Program Files\Microsoft Silverlight\2.0.40115.0

You can also build IronPython for Silverlight in a similar manner. Aliases for this will be added soon, but if you need a custom build of the DLR languages for Silverlight, this is the way to do it.

Samples

Six samples are included in this release. The first three are desktop samples in the “/samples” directory. The last three are Silverlight samples in the “/silverlight/samples” directory:

Tutorial - An interactive IronRuby tutorial

DiskUse - A small WPF application which visualizes the disk usage for a particular directory

IRPowerShell - a small library and sample applications showing how to interact with PowerShell from IronRuby

Clock – a simple Silverlight sample

Photoviewer – do AJAX programming with IronRuby (ARAX =P)

REPL – Interactive console in the browser.

Some more interesting changes:

• Improved DLR Interop: adds support for GetMember/SetMember with method_missing, Binary/Unary operators, and indexers.
• Handling of CLR protected and private methods and properties
• Reimplemented File.expand_path such that it does not use System.IO.Path.
• This allows us to get better compatibility with MRI. The motivating reason was
• that RSpec does File.expand_path("filename:linenumber")
• Improves the implementation of singleton method dispatch.
• These changes improve running time of specs significantly (2x)
• Renames Method, UnboundMethod#overloads (plural) to
• Method, UnboundMethod#overload (singular).
• The old name is still available for now and throws an exception.
• Implements adaptively compiled rules
• Improves performance of Array#- from quadratic algorithm to linear.
• Improves implementation of RubyArray

Bugs closed

Here are all 37 bugs closed since the last release (2009-05-19). You can see more information about each one on CodePlex.

1521  Access is allowed to internal fields
1502  alias_method fails for :do
821   File.expand_path does not support a line number after filename
1509  Proc.to_s should include line number where the block was declared
1501  WinForms broken
1400  $? is not always Process::Status	
1345  load_assembly() should work
1344  System.Action.new does not work
1306  Cannot call CLR constructor of builtin type
1184  public(:foo) does not work correctly for mixed-in methods
1085  Cannot call new on subtypes of builtin classes whose "new" method has optional arguments
1060  visibility of send :define_method
917   Passing a Ruby array to a .NET method that expects an IEnumerable derivative fails with GetEnumerator call
783   Assert in SetMethodBasesNoLock when calling #== on Ruby class inheriting from CLR class which overrides Equals
761   Wrong behavior when calling redefined methods on object instances
1470  Can't call the BigIntegerOverload of a method with a DefaultProtocol Attribute on the BigInteger attribute
1426  The located assembly's manifest definition does not match the assembly reference. (ctp dev10 beta1)
1441  Error with cyrillyc text in Sharepoint	
1352  Test Defects	
814   Allocator underfined for  (TypeError)
572   Error when running Cucumber examples with IronRuby
718   IronRuby ignores RUBYLIB environment variable
727   to_proc not working
1351  redist-libs should have rubygems-1.3.1	
466   ''.split(//, -1) returns [""] instead of []	
940   Can't inherit from abstract classes	
1028  Missing conversion from Duration to Fixnum (ActiveSupport)?
374   irails Foo: undefined method for OpenSLL::Random.random_bytes
459   throw FileNotFoundException => rescue Errno.NoEntryError
499   Pathname#cleanpath messes up the pathname
467   "igem install rails" fails
375   "ir script\server" causes a YAML parser error
461   Generic type conversion from Fixnum to Integer
674   autoload does not use File::SEPARATOR
1021  Time class instance
578   yield fails in eval'd code
605   Array#hash should properly handle recursive arrays

Enjoy!

Please test out IronRuby 0.6 and let us know if you have any issues. We hope you enjoy this release!

-- The IronRuby team

Technorati Tags: IronRuby,Performance,Cucumber,Testing,Silverlight,Release,Ruby