“Start spreading the news”: the future of Jimmy and IronRuby

Though Frank Sinatra says it best, “I’m leaving today” isn’t exactly accurate; my last day as a Microsoft employee was July 23rd, 2010. This post is almost two weeks delayed as Felicia and I have been on the road since the 26th, driving cross-country to the east coast; we also decided to leave Seattle in favor of New York, our home state.

Both decisions were extremely difficult to make, as I will miss all the brilliant people I worked with. Just being in their presence made me feel smarter, and we accomplished some amazing things together. Many were also my friends, making this a very heart-wrenching decision too. However, I joined Microsoft to bring Ruby and other open-source programming languages to the .NET framework, as well as to promote open-source practices in general, and I promised myself to ensure the truth of that statement throughout my Microsoft career. So, when my manager asked me, “what else would you want to work on other than Ruby,” I started looking for a new job outside Microsoft.

While Microsoft’s commitment to dynamic languages on .NET has been questioned many times, my tiny team has been excellent at suppressing those fears with quality implementations of Ruby and Python for .NET, compiler services and language embedding API called the Dynamic Language Runtime, and integration with .NET application frameworks like Silverlight and ASP.NET. And most recently the beginnings of IDE support for dynamic languages in Visual Studio. And all this released under an well-known open-source license, the Apache License (Version 2). This was only possible because my team had the freedom to do what we needed to do to counter those fears and run an effective open-source project

However, a year ago the team shrunk by half and our agility was severely limited. I’m omitting the internal reasons for this, as they are the typical big-company middle-management issues every software developer has. In short, the team is now very limited to do anything new, which is why the Visual Studio support for IronPython took so long. IronRuby’s IDE support in Visual Studio hasn’t been released yet for the same reasons. While this is just one example, many other roadblocks have cropped up that made my job not enjoyable anymore.

Overall, I see a serious lack of commitment to IronRuby, and dynamic language on .NET in general. At the time of my leaving Tomas and myself were the only Microsoft employees working on IronRuby. If this direction for dynamic languages on .NET is a path you do not want Microsoft to take, I strongly suggest you provide feedback to the team’s management directly. Also, Jason Zander runs the Visual Studio team, which IronRuby, IronPython, and the DLR happen to be a part of, and is a big proponent of these dynamic languages efforts, so provide him with your thoughts as well.

That being said, I am still interested in implementing dynamic languages on .NET, so I will remain a IronRuby core-team member, ironically making me the first non-Microsoft core contributor. The bad-news is I will no longer be working on IronRuby full-time, but in the near future I’m definitely staying active. Also, Tomas will definitely continue working on IronRuby when he can; we weren’t the last two people left for no reason. :-)

Given that Tomas and I will only be working part-time on IronRuby now, I invite the Ruby and .NET communities to come help us figure out how to continue the IronRuby project, assuming that Microsoft will eventually stop funding it. I’ll start a thread on the IronRuby Mailing List shortly, so keep an eye on that if you’d like to help. [Update: here’s the thread about the next steps for IronRuby. Join the list and discuss.]

While moving to New York is mainly a personal decision, as both my fiancée and I grew up there and our immediate families are still there, it was also for professional reasons; I’ve accepted a new position at Lab49, a financial technology consulting firm in New York City. I chose the financial industry not just because its dominance in New York, but because I see a lot of similarities between financial software and developer tools. Financial software serves a very technical user, much like programmers, but unlike programmers I know nothing these users, making it a challenging new space. It will be familiar as well, as Lab49 has done work with the new Technical Computing team, which many people who once worked on dynamic languages moved to. Lab49 was also very interested in my IronRuby and IronPython background, so it’s a great next step for me.

I’m grateful to have worked on compilers full-time, outside of academia, while still contributing to open-source, especially at a company that hardly showed up on the open-source radar just 4 years ago. And at least one dynamic language at Microsoft is getting love; IE9’s JavaScript engine, and Microsoft has awesome intentions with it. I’m totally a supporter of most things Microsoft is doing, and I look forward to working closely with them on financial problems. I’m just extremely disappointed with their decisions around dynamic languages on .NET. As one former-teammate’s email signature read, “If your ideas are any good, you'll have to ram them down people's throats.”

I’m looking forward to this new chapter in both my life and my career. Not only am I living in the city that never sleeps, but I hope to build upon my dynamic language work and use it in an area completely new to me. While I expect to still be Ruby and .NET oriented, my posts will be about solving new problems, and should make for some good reading. Stay tuned, and thanks for all the support thus far.

ASP.NET dynamic language support is open source

I'm happy to finally announce that the ASP.NET dynamic language support is now open source:

Download IronPython and ASP.NET integration

For a full IronPython release with the Python standard library, download IronPython 2.7 Alpha 1.

This release contains the source code to Microsoft.Scripting.AspNet.dll, located in the src directory, licensed under the Apache License (Version 2). It will be available in the source repository for IronPython in the very near future, but don't hesitate to start sending in patches. This release is compatible with IronPython 2.7 Alpha 1.

Background

This download enables IronPython as an ASP.NET programming language. To create a new IronPython ASP.NET WebForms project, simply copy examples\web.config and examples\bin, and use examples\hello-webforms.aspx as a reference. A redistributed copy of the IronPython 2.7 Alpha 1 binaries can be found in the examples\bin directory; all files except Microsoft.Scripting.AspNet.dll, the IronPython ASP.NET integration, are from the IronPython 2.7 Alpha 1 release.

For more detail on getting started, here’s a simple walk-through of making the “hello-webforms” app.

Package

Here's what's in the zip file:

• /License.html - The Apache License, Version 2.
• /dlr-aspnet.sln - VS2010 solution for examples
• /examples - Examples of using IronPython in ASP.NET.
• /examples/Web*.config - configures ASP.NET to use IronPython
• /examples/bin - Microsoft.Scripting.AspNet.dll, the ASP.NET integration, and a redistribution of IronPython 2.7A1.
• /src - C# source code that builds Microsoft.Scripting.AspNet.dll

Upgrading

This release renames the main DLL from Microsoft.Web.Scripting.dll to Microsoft.Scripting.AspNet.dll. If upgrading, you'll have to replace all occurrences of Microsoft.Web.Scripting with Microsoft.Scripting.AspNet. This will primarily be at the top of all aspx pages, as well as in your application's web.config. Also note the version number is now 1.1.0.1, which matches all the other Microsoft.Scripting assemblies.

Feedback

As always, please report issues on the IronPython issue tracker. You can also try to fix any issues yourself and submit a patch. Lastly, you can actually talk to humans on the IronPython mailing list.

Enjoy!

Summer of “Iron”: LINQ, Visual Studio tooling, and the Apache License

New releases and announcements from the “Iron” projects have come out over the last couple days, so I wanted to give you an overview of what’s happening, point out the really cool parts, and reiterate some of the motivations.

From a release perspective, both IronRuby and IronPython released new versions of the DLR-based .NET programming language implementations: IronPython 2.7 Alpha and IronRuby 1.1. Click on the respective release name for the full release notes and downloads, but I’ll summarize a bit here:

IronPython 2.7 Alpha

Install IronPython 2.7 Alpha (includes Visual Studio tooling)

This Alpha release is the first IronPython release working towards Python 2.7 compatibility, and contains a number of bug fixes and performance improvements. It also now requires .NET 4.0 or Silverlight 4; you will need to build from source for down-level frameworks. The installer now includes Visual Studio support for IronPython, rather than being a separate installer, and the source-code for the tools has been open-sourced! Keep reading for licensing information …

IronRuby 1.1

Install IronRuby 1.1

Aside from a bunch of bug-fixes, the latest release of IronRuby adds support for .NET extension methods, meaning that .NET programming patterns that are dependent on extension methods, like LINQ, the Reactive framework, Parallel.NET, etc, are now all usable from Ruby code. For example, here’s a simple LINQ example:

For more information see the 101 LINQ samples ported to IronRuby.

Also, since I never announced the IronRuby 1.0 release on my blog … consider this the announcement.

Apache License, Version 2

IronRuby, IronPython, and the DLR are now licensed under the Apache License, Version 2. To address all concerns around “why” this is changing, it is solely in reaction to feedback about the licensing terms from users and the .NET/Python/Ruby/dynamic-language communities at-large. The Apache License is a more familiar and popular license to those communities than the Microsoft Public License; using it should lower any license-related barriers people encountered in the past when considering these programming-language implementations.

Enjoy!

MIX10, Part 3 - Using dynamic languages in existing web-applications

This post is part of the “MIX10 – Pumping Iron on the web” series:

• Part 1 - Introduction
• Part 2.1 – Server-side web development with dynamic languages
• Part 2.2 – Client-side web development with dynamic languages
• Part 3 - Using scripting in static applications
• Coming soon: Part 4 - Web-app extensibility with scripts

The original post was mistakenly removed, so it’s been reposted with the original post date, 4/27/2010. Sorry if this confuses your blog readers or causes any other inconvenience.

Up until now I've discussed how to use dynamic languages to power both the server-side as well as the client-side of your web-application, but what if you want to apply these methods to solve certain problems in an existing application?

Testing

A low-risk, high-benefit use of dynamic languages in your existing applications is for testing. This helps make the act of writing tests simpler, and quite possibly more fun, encouraging your team to actually maintain the test suite.

Before looking at how to test web-apps, let's take a brief look at what a test written with RSpec, a popular Ruby testing framework, looks like:

Note: there are Ruby testing frameworks that look a bit more like what you might be used to. The following is an equivalent test written with test/unit, and this will give you a better idea of the structure of the above example:

The RSpec snippet almost reads like english, making it very clear what the intended behavior of Stack is. Also, it shows the power of Ruby for creating internal DSLs; a "language" built out of the constructs of an existing language. describe and it look like language keywords, but in-fact they are really just methods, because Ruby has optional parameters (as we discovered earlier with Sinatra). Using actual strings as the test name, rather than a method name, allows you to describe the test accurately. Each object has a should method which makes any subsequent calls part of an assertion, making it very obvious which value is the "expected" value and which is the "actual".

The crazy thing is how little code is required to make that work; 24 lines of Ruby. The key points are that yield executes the do-end block passed to a method, and the should method is added to every object, turning any subsequent methods calls into an assertion:

However, please don't use this example as your real testing framework, and then get mad at me when it doesn't have a feature you want; RSpec, Bacon, or test/spec are much more mature testing frameworks that support this same syntax.

See my previous post about using IronRuby to test C# Silverlight applications; it’s still relevant though it’s a fairly old post.

Hosting

IronRuby and IronPython are built on-top of the Dynamic Language Runtime, which is comprised of many parts, one of which being a .NET Hosting API, allowing you to embed a scripting language in any .NET app.

Now we come to the "ah-ha!" moment of the talk; everything you've seen in the MIX10 posts is made possible by this API. Keep in mind these languages are built on .NET, so their implementations are accessible from any .NET language. C# and VB today are not built on .NET; they just compile programs to run on .NET, which is why you can't easily host those languages today.

Here's the catch; since these language engines are built on .NET, they need to run in a .NET application. So, all Ruby or Python code runs by hosting the languages inside a .NET application. We do things to make this seamless in specific environments: for example, ir.exe and ipy.exe are both .NET programs which host the language and run the code in a command-line, mimicking the behavior of ruby.exe and python.exe. Here are some other popular hosts:

• ipyw.exe: runs scripts in a console-less program for Windows applications
• Microsoft.Scripting.Silverlight.dll: entry-point for Silverlight applications which run HTML script tags and scripts inside the XAP
• IronRuby.Rack.dll: run rack-based applications on IIS
• Microsoft.Web.Scripting.dll: run Python in ASP.NET
• System.Web.Mvc.IronRuby: run Ruby in ASP.NET MVC

However, we can't provide "runners" for every environment that will spring up, so we allow you to use the same APIs that these runners use in your own apps. These APIs have been kept very simple, as we want any .NET developer to be able to use a DLR scripting language in their applications.

But why embed a scripting language into your application? The main scenario is to scripts as an extensibility mechanism, either internally or as functionality you provide for your end-users. Here are a few concrete examples of what scripts could be used for:

• An advanced search/filter (letting users use LINQ safely)
• High-level business logic to computing prices of items, applying discounts, etc; any type of rules engine
• A system which changes behavior based on external data
• Customizing a single codebase for different clients
• Add-ons for end-users to make your application better (eg. Facebook)
• Making application logic simpler to read than the core of your system (polyglot)

Let's show you how to do the basics, and hopefully that will spark your imagine to think up other cool use-cases:

1. Create a new web application project in Visual Studio 2010, and open the Default.aspx.cs page.
2. Place a label on the page and call it “Message”.
3. Add references to the necessary DLLs to host IronPython (IronPython.dll and Microsoft.Scripting.dll)
4. Write the 5 lines of code to update the label’s text from Python:

There are basically three types you need to know; ScriptRuntime, ScriptEngine, and ScriptScope.

• ScriptRuntime is a level of encapsulation for your scripts; it represents the DLR scripting runtime, and all script operations go through it.

• ScriptEngine is the type that is returned from ScriptRuntime.GetEngine; it represents a DLR-language. In this case, we asked for the language by name, as that's the easiest way to keep it easily configurable, but the downside is you need language configuration info in your App.config. However, if you only want to depend on IronPython, you can use IronPython.Hosting.Python.CreateEngine(), which does all the setup for Python for you.

  The ScriptEngine enabled you to execute code in that language, in a variety of ways, from the basic engine.Execute method (essentally eval), or being more fine-grained engine.CreateScriptSourceFromString(code).Compile().Execute(), which parses the file, compiles it, and then executes it. Code can be executed against a ScriptScope to set initial state and share state between executions.

• ScriptScope defines the state for your script; like what variables/methods are present. It is a dynamic object, so you can do things like scope.page = this, and that will set the page variable for scripts that execute against the scope. In downlevel .NET frameworks, you'd have to use scope.SetVariable(page, this).

Slight aside: since these APIs are .NET based, the dynamic languages themselves can consume them to run other dynamic languages! For example, here's Ruby executing Python code through the DLR Hosting APIs:

What's also interesting is the dynamic languages can communicate between each-other just as easily; here's Ruby calling Python code:

Anyway, hopefully this sparks some creativity! For more web-related information, also posted about this in relation to Silverlight applications: Scripting C# apps with IronPython.

The next post will show some cool applications of this … (coming soon).

MIX10, Part 2.2 – Client-side web-development with dynamic languages

This post is part of the “MIX10 – Pumping Iron on the web” series:

• Part 1 - Introduction
• Part 2.1 – Server-side web development with dynamic languages
• Part 2.2 – Client-side web development with dynamic languages
• Part 3 - Using scripting in static applications
• Coming soon: Part 4 - Web-app extensibility with scripts

The original post was mistakenly removed, so it’s been reposted with the original post date, 4/26/2010. Sorry if this confuses your blog readers or causes any other inconvenience.

IronRuby and IronPython are fully-supported in the browser, thanks to Silverlight. In-fact, they are hands-down the simplest way to develop a Silverlight application. This is not only because of how expressive the programming languages are; the integration with Silverlight doesn't fight how the web works.

For example, here's an entire Silverlight app which just writes a message into the HTML page, written in Python:

DLR-based Silverlight applications let you write HTML script-tags in other languages than JavaScript, but in a way that works cross-browser and cross-platform; the languages work in Moonlight on Linux as well.

Both inline and script-src tags are supported, so your scripts can be separated from the HTML file:

This integration makes writing Silverlight applications just as easy as they were in Silverlight 1, but with the power of .NET and Ruby or Python. DLR-based applications also support a XAP model for anyone familiar with how static Silverlight applications work, so you get to choose which way you prefer to write your applications.

All the specific examples used in this section of the talk were taken from these posts:

• Python in the browser: my blog post from PyCon 2010 about IronPython in Silverlight
• Ruby in the browser: my article in RubyInside about IronRuby in Silverlight
• Silverlight.net / dynamic languages: tons of info on getting started with Python, Ruby, and the DLR in Silverlight.

Next up, using dynamic languages in your existing web-applications.

MIX10, Part 2.1 – Server-side web development with dynamic languages

This post is part of the “MIX10 – Pumping Iron on the web” series:

• Part 1 - Introduction
• Part 2.1 – Server-side web development with dynamic languages
• Part 2.2 – Client-side web development with dynamic languages
• Part 3 - Using scripting in static applications
• Coming soon: Part 4 - Web-app extensibility with scripts

The original post was mistakenly removed, so it’s been reposted with the original post date, 4/25/2010. Sorry if this confuses your blog readers or causes any other inconvenience.

One reason for embracing dynamic languages is to make your entire web-development experience simpler, be it on ASP.NET enabled web-servers, or on the client through Silverlight. Let's first look at the server.

Both IronPython and IronRuby can run on the same infrastructure as your ASP.NET applications, though in their own ways. Due to historic reasons, IronPython is supported as an ASP.NET language through the ASP.NET Dynamic Language integration, and IronRuby is supported through IronRuby.Rack, which enables Rack-based Ruby web-applications to run on ASP.NET. However, both are open-source, so each one could be ported to the other language.

Since these DLR-based languages run on ASP.NET, deploying them is no different than any other ASP.NET application; they can be run on any ASP.NET enabled web-server like IIS. Keeping that in mind, let’s first look at how simple Ruby web apps can be.

Minimal Ruby web-applications

Ruby itself has very expressive syntax, and the Ruby community has built many web-frameworks to make web-development really simple. For example, Sinatra is a web-framework made to minimize the amount of code required to respond to web requests:

The above code does exactly what it says; when a get request happens for '/', show "Hello, World" on the page. This highlights Ruby's domain-specific language abilities; get looks like a keyword, though it's really just a method call with '/' as the first argument (Ruby lets you omit parenthesis from method calls too ... any VB script fans out there?). The do-end block is syntactic sugar for passing a lambda as the last-argument to the get method; all Ruby methods take an arbitrary "block" of code between do-end or {}. Inside that block is what happens on each request, and whatever is returned is written to the response ("Hello, World"); the last statement of any expression (blocks, methods, if-statements, etc) is implicitly the return value of the statement.

For all those C# fans, you can use curly-braces too:

Though these features sound arbitrary by themselves, if I were to write this with only the more-familiar language features found in Ruby, it would lose its character:

This defines a Ruby method verbose, which explicitly returns the string "Hello, World", and then calls the get method directly with the first argument being the URL and the second argument being an explicit pointer to the verbose method. Why does this look so much uglier? While this might be closer to how the programming language actually runs the initial examples, it's not how the programmer thinks.

Not to leave Python out of this love-fest, Python can make this look very pretty as well, but in her own special way. Imagining that a Sinatra-like library exists for Python:

Here the index method is created, which explicitly accepts both the request and the response as arguments; Python's all about being explicit, while Ruby is very implicit. Then the method would be "decorated" with the sinatra.get decorator, which would tell the web-framework that index represents a get-request for "/".

What's a decorator?

A Python decorator is basically a function that accepts a function and returns a function, so this imaginary Sinatra-like framework would define get something like this:

Another way of looking at it is without the decorator syntax:

It’s a bit more readable than Ruby, and almost equivalent to the decorator way, except for the order of get in the code. You'll also see that getting a method pointer is much cleaner than Ruby (index vs. method(:index)); in Ruby index would call the method, since Ruby allows method calls with or without parenthesis, where Python uses parenthesis to indicate a method call.

Point being is that both Ruby and Python are very expressive in their own ways, and makes it really easy to make simple websites on Windows. The IronRuby team takes of advantage of this to power http://ironruby.info, a Ruby-compatibility reporting website. This is the code from the main page:

This renders the index.haml template with the data returned from Stats.get_latest, which is pretty much what the code says. The HTML is generated from the haml templating engine, which makes generating HTML and calling Ruby code extremely easy:

To play around with using HAML, you can use aspnet-haml to support .haml files though ASP.NET.

Ruby on Rails - Databases with Ruby

One of the most popular (or most buzzed) web-frameworks is Ruby on Rails, which is just a collection of Ruby libraries for structuring your web-application. Rails uses the Model-View-Controller pattern, so any ASP.NET MVC people will find it a very-familiar framework. However, Rails really shines when it comes to interacting with the database through it's ActiveRecord library (the “Model” layer). ActiveRecord maps Ruby classes to database tables, and provides an Ruby abstraction for interacting with the database:

This is all the code that is required to map your Ruby classes to the database, as well as create the structure of the database. ActiveRecord dynamically provides getters/setters for the table, as well as sets up foreign-keys and relationships based on conventions (belongs_to :posts assumes that the table has a 'post_id' field).

Interacting with the database is just as easy as calling methods; Post.all translates to the SELECT * from posts SQL query, since the Post object is mapped to the posts database table. Post.find(<id>) does a SELECT * from posts where id=<id>, etc.

Because Rails uses the Rack web-server interface, it will also run on IIS using IronRuby.Rack. See the IronRuby Rails documentation for more info about using Rails on IronRuby, and the Ruby on Rails documentation for general Rails usage.

ASP.NET MVC and IronRuby

Those were all Ruby-based web-frameworks, but what about using ASP.NET directly? The IronRuby community has developed an integration with ASP.NET MVC, so you can write your controllers and views in Ruby. Special thanks to Ivan Porto Carrero for single-handedly maintaining it.

ASP.NET and IronPython

Again, to give Python some love, IronPython directly integrates with ASP.NET, letting you write your ASPX code-behind files in Python.

Because of ASP.NET's events-based API (rather than a response-based API like Sinatra/Rails), Python methods can automatically hook events by using the <object>_<event-name> convention, and all server-side controls with "ID"s ends up being a variable available to the Python module. And application-level event hooks can go in global.py. But it's really nice to write Language="IronPython" at the top. =)

Python code can can also interact with the controls:

The <%# %> syntax lets run Python code in the context of the ASP.NET control's data source. The repeater's data-source was set to a list of IMAGETAGS (a python class), which has all those fields on it.

In conclusion, on the server you have many options for using IronRuby and IronPython to simplify your solutions.

Next up, using dynamic languages on the client through Silverlight.

MIX10, Part 1 – IronRuby, IronPython, and the web

This post is part of the “MIX10 – Pumping Iron on the web” series:

• Part 1 - Introduction
• Part 2.1 – Server-side web development with dynamic languages
• Part 2.2 – Client-side web development with dynamic languages
• Part 3 - Using scripting in static applications
• Coming soon: Part 4 - Web-app extensibility with scripts

The original post was mistakenly removed, so it’s been reposted with the original post date, 4/24/2010. Sorry if this confuses your blog readers or causes any other inconvenience.

This past week I had the pleasure of attending and speaking at MIX10 in Las Vegas about using IronRuby and IronPython on the web. If you’re a TV-person instead of a reading-person, here’s a video of the talk:

As I usually do, this series of posts will be a write-up of my talk … but first …

iron? - Jim Hugunin and John Lam have both been quoted as saying "Iron" stands for different acronyms; "Implementation running on .NET" and "It runs on .NET", respectively. I’m going to put my foot down and officially side with Jim, though really they are bacronyms; neither is actually the original meaning. A StackOverflow post explains more, and I hope that puts the wondering to rest.

This talk is all about the why and how of embrace dynamic languages on Microsoft's web platform - be it on the web-server (IIS) or in the web-browser (Silverlight), and even in existing applications. To start out with, here’s my rational for why we as a developer community should care:

It’s no secret that developers like things to be simple, and the web is pretty simple as far as application models go. While the web’s feature-set is pretty attractive itself (server-client oriented, instant client deployment, and cross-platform clients to name a few), the equally attractive development experience (simple UI mark-up system and a scripting language) still make it easy for people to build amazing websites.

Though the application model is simple, developers continue to evolve it; server and client frameworks are vital tools that make the web-development experience even more productive – it’s very rare that a website has no server side or client side dependencies.

These frameworks provide such innovative features because they stand on the shoulders of powerful and expressive dynamic/scripting languages, giving the frameworks the unique ability to model the "web" as they see fit.

While each web-framework is powerful in its own right, the power really comes from the choice to use whatever tools help get things done. Developers and designers for .NET have the same need to get things done, and if getting things done is essentially the result of programming language choice, what choices are there? Really only C# and VB, which are static programming languages, requiring a compile step before execution and relying on debugging to see code in action. Take a look at the other languages mainly used on the web again -- they're all dynamic languages, running from source code, and providing interactive environments for running code. Why is .NET static while the rest of the web is dynamic? Why can't they exist together? If .NET provided some language choice for developers, all the languages be used together, and the .NET community could benefit from the amazing work being done by dynamic language community, and visa versa.

We in-fact live in this world, and embracing dynamic languages is possible on .NET, but why would you want to do it? I'll discuss this in the following posts (links to posts coming soon):

• Server-side web development with dynamic languages
• Client-side web development with dynamic languages
• Using dynamic languages in your existing web-applications
• Coming soon: Opening up your applications to extensibility with scripts