MIX10, Part 3 - Using dynamic languages in existing web-applications

This post is part of the “MIX10 – Pumping Iron on the web” series:

• Part 1 - Introduction
• Part 2.1 – Server-side web development with dynamic languages
• Part 2.2 – Client-side web development with dynamic languages
• Part 3 - Using scripting in static applications
• Coming soon: Part 4 - Web-app extensibility with scripts

The original post was mistakenly removed, so it’s been reposted with the original post date, 4/27/2010. Sorry if this confuses your blog readers or causes any other inconvenience.

Up until now I've discussed how to use dynamic languages to power both the server-side as well as the client-side of your web-application, but what if you want to apply these methods to solve certain problems in an existing application?

Testing

A low-risk, high-benefit use of dynamic languages in your existing applications is for testing. This helps make the act of writing tests simpler, and quite possibly more fun, encouraging your team to actually maintain the test suite.

Before looking at how to test web-apps, let's take a brief look at what a test written with RSpec, a popular Ruby testing framework, looks like:

Note: there are Ruby testing frameworks that look a bit more like what you might be used to. The following is an equivalent test written with test/unit, and this will give you a better idea of the structure of the above example:

The RSpec snippet almost reads like english, making it very clear what the intended behavior of Stack is. Also, it shows the power of Ruby for creating internal DSLs; a "language" built out of the constructs of an existing language. describe and it look like language keywords, but in-fact they are really just methods, because Ruby has optional parameters (as we discovered earlier with Sinatra). Using actual strings as the test name, rather than a method name, allows you to describe the test accurately. Each object has a should method which makes any subsequent calls part of an assertion, making it very obvious which value is the "expected" value and which is the "actual".

The crazy thing is how little code is required to make that work; 24 lines of Ruby. The key points are that yield executes the do-end block passed to a method, and the should method is added to every object, turning any subsequent methods calls into an assertion:

However, please don't use this example as your real testing framework, and then get mad at me when it doesn't have a feature you want; RSpec, Bacon, or test/spec are much more mature testing frameworks that support this same syntax.

See my previous post about using IronRuby to test C# Silverlight applications; it’s still relevant though it’s a fairly old post.

Hosting

IronRuby and IronPython are built on-top of the Dynamic Language Runtime, which is comprised of many parts, one of which being a .NET Hosting API, allowing you to embed a scripting language in any .NET app.

Now we come to the "ah-ha!" moment of the talk; everything you've seen in the MIX10 posts is made possible by this API. Keep in mind these languages are built on .NET, so their implementations are accessible from any .NET language. C# and VB today are not built on .NET; they just compile programs to run on .NET, which is why you can't easily host those languages today.

Here's the catch; since these language engines are built on .NET, they need to run in a .NET application. So, all Ruby or Python code runs by hosting the languages inside a .NET application. We do things to make this seamless in specific environments: for example, ir.exe and ipy.exe are both .NET programs which host the language and run the code in a command-line, mimicking the behavior of ruby.exe and python.exe. Here are some other popular hosts:

• ipyw.exe: runs scripts in a console-less program for Windows applications
• Microsoft.Scripting.Silverlight.dll: entry-point for Silverlight applications which run HTML script tags and scripts inside the XAP
• IronRuby.Rack.dll: run rack-based applications on IIS
• Microsoft.Web.Scripting.dll: run Python in ASP.NET
• System.Web.Mvc.IronRuby: run Ruby in ASP.NET MVC

However, we can't provide "runners" for every environment that will spring up, so we allow you to use the same APIs that these runners use in your own apps. These APIs have been kept very simple, as we want any .NET developer to be able to use a DLR scripting language in their applications.

But why embed a scripting language into your application? The main scenario is to scripts as an extensibility mechanism, either internally or as functionality you provide for your end-users. Here are a few concrete examples of what scripts could be used for:

• An advanced search/filter (letting users use LINQ safely)
• High-level business logic to computing prices of items, applying discounts, etc; any type of rules engine
• A system which changes behavior based on external data
• Customizing a single codebase for different clients
• Add-ons for end-users to make your application better (eg. Facebook)
• Making application logic simpler to read than the core of your system (polyglot)

Let's show you how to do the basics, and hopefully that will spark your imagine to think up other cool use-cases:

1. Create a new web application project in Visual Studio 2010, and open the Default.aspx.cs page.
2. Place a label on the page and call it “Message”.
3. Add references to the necessary DLLs to host IronPython (IronPython.dll and Microsoft.Scripting.dll)
4. Write the 5 lines of code to update the label’s text from Python:

There are basically three types you need to know; ScriptRuntime, ScriptEngine, and ScriptScope.

• ScriptRuntime is a level of encapsulation for your scripts; it represents the DLR scripting runtime, and all script operations go through it.

• ScriptEngine is the type that is returned from ScriptRuntime.GetEngine; it represents a DLR-language. In this case, we asked for the language by name, as that's the easiest way to keep it easily configurable, but the downside is you need language configuration info in your App.config. However, if you only want to depend on IronPython, you can use IronPython.Hosting.Python.CreateEngine(), which does all the setup for Python for you.

  The ScriptEngine enabled you to execute code in that language, in a variety of ways, from the basic engine.Execute method (essentally eval), or being more fine-grained engine.CreateScriptSourceFromString(code).Compile().Execute(), which parses the file, compiles it, and then executes it. Code can be executed against a ScriptScope to set initial state and share state between executions.

• ScriptScope defines the state for your script; like what variables/methods are present. It is a dynamic object, so you can do things like scope.page = this, and that will set the page variable for scripts that execute against the scope. In downlevel .NET frameworks, you'd have to use scope.SetVariable(page, this).

Slight aside: since these APIs are .NET based, the dynamic languages themselves can consume them to run other dynamic languages! For example, here's Ruby executing Python code through the DLR Hosting APIs:

What's also interesting is the dynamic languages can communicate between each-other just as easily; here's Ruby calling Python code:

Anyway, hopefully this sparks some creativity! For more web-related information, also posted about this in relation to Silverlight applications: Scripting C# apps with IronPython.

The next post will show some cool applications of this … (coming soon).

MIX10, Part 2.2 – Client-side web-development with dynamic languages

This post is part of the “MIX10 – Pumping Iron on the web” series:

• Part 1 - Introduction
• Part 2.1 – Server-side web development with dynamic languages
• Part 2.2 – Client-side web development with dynamic languages
• Part 3 - Using scripting in static applications
• Coming soon: Part 4 - Web-app extensibility with scripts

The original post was mistakenly removed, so it’s been reposted with the original post date, 4/26/2010. Sorry if this confuses your blog readers or causes any other inconvenience.

IronRuby and IronPython are fully-supported in the browser, thanks to Silverlight. In-fact, they are hands-down the simplest way to develop a Silverlight application. This is not only because of how expressive the programming languages are; the integration with Silverlight doesn't fight how the web works.

For example, here's an entire Silverlight app which just writes a message into the HTML page, written in Python:

DLR-based Silverlight applications let you write HTML script-tags in other languages than JavaScript, but in a way that works cross-browser and cross-platform; the languages work in Moonlight on Linux as well.

Both inline and script-src tags are supported, so your scripts can be separated from the HTML file:

This integration makes writing Silverlight applications just as easy as they were in Silverlight 1, but with the power of .NET and Ruby or Python. DLR-based applications also support a XAP model for anyone familiar with how static Silverlight applications work, so you get to choose which way you prefer to write your applications.

All the specific examples used in this section of the talk were taken from these posts:

• Python in the browser: my blog post from PyCon 2010 about IronPython in Silverlight
• Ruby in the browser: my article in RubyInside about IronRuby in Silverlight
• Silverlight.net / dynamic languages: tons of info on getting started with Python, Ruby, and the DLR in Silverlight.

Next up, using dynamic languages in your existing web-applications.

MIX10, Part 2.1 – Server-side web development with dynamic languages

This post is part of the “MIX10 – Pumping Iron on the web” series:

• Part 1 - Introduction
• Part 2.1 – Server-side web development with dynamic languages
• Part 2.2 – Client-side web development with dynamic languages
• Part 3 - Using scripting in static applications
• Coming soon: Part 4 - Web-app extensibility with scripts

The original post was mistakenly removed, so it’s been reposted with the original post date, 4/25/2010. Sorry if this confuses your blog readers or causes any other inconvenience.

One reason for embracing dynamic languages is to make your entire web-development experience simpler, be it on ASP.NET enabled web-servers, or on the client through Silverlight. Let's first look at the server.

Both IronPython and IronRuby can run on the same infrastructure as your ASP.NET applications, though in their own ways. Due to historic reasons, IronPython is supported as an ASP.NET language through the ASP.NET Dynamic Language integration, and IronRuby is supported through IronRuby.Rack, which enables Rack-based Ruby web-applications to run on ASP.NET. However, both are open-source, so each one could be ported to the other language.

Since these DLR-based languages run on ASP.NET, deploying them is no different than any other ASP.NET application; they can be run on any ASP.NET enabled web-server like IIS. Keeping that in mind, let’s first look at how simple Ruby web apps can be.

Minimal Ruby web-applications

Ruby itself has very expressive syntax, and the Ruby community has built many web-frameworks to make web-development really simple. For example, Sinatra is a web-framework made to minimize the amount of code required to respond to web requests:

The above code does exactly what it says; when a get request happens for '/', show "Hello, World" on the page. This highlights Ruby's domain-specific language abilities; get looks like a keyword, though it's really just a method call with '/' as the first argument (Ruby lets you omit parenthesis from method calls too ... any VB script fans out there?). The do-end block is syntactic sugar for passing a lambda as the last-argument to the get method; all Ruby methods take an arbitrary "block" of code between do-end or {}. Inside that block is what happens on each request, and whatever is returned is written to the response ("Hello, World"); the last statement of any expression (blocks, methods, if-statements, etc) is implicitly the return value of the statement.

For all those C# fans, you can use curly-braces too:

Though these features sound arbitrary by themselves, if I were to write this with only the more-familiar language features found in Ruby, it would lose its character:

This defines a Ruby method verbose, which explicitly returns the string "Hello, World", and then calls the get method directly with the first argument being the URL and the second argument being an explicit pointer to the verbose method. Why does this look so much uglier? While this might be closer to how the programming language actually runs the initial examples, it's not how the programmer thinks.

Not to leave Python out of this love-fest, Python can make this look very pretty as well, but in her own special way. Imagining that a Sinatra-like library exists for Python:

Here the index method is created, which explicitly accepts both the request and the response as arguments; Python's all about being explicit, while Ruby is very implicit. Then the method would be "decorated" with the sinatra.get decorator, which would tell the web-framework that index represents a get-request for "/".

What's a decorator?

A Python decorator is basically a function that accepts a function and returns a function, so this imaginary Sinatra-like framework would define get something like this:

Another way of looking at it is without the decorator syntax:

It’s a bit more readable than Ruby, and almost equivalent to the decorator way, except for the order of get in the code. You'll also see that getting a method pointer is much cleaner than Ruby (index vs. method(:index)); in Ruby index would call the method, since Ruby allows method calls with or without parenthesis, where Python uses parenthesis to indicate a method call.

Point being is that both Ruby and Python are very expressive in their own ways, and makes it really easy to make simple websites on Windows. The IronRuby team takes of advantage of this to power http://ironruby.info, a Ruby-compatibility reporting website. This is the code from the main page:

This renders the index.haml template with the data returned from Stats.get_latest, which is pretty much what the code says. The HTML is generated from the haml templating engine, which makes generating HTML and calling Ruby code extremely easy:

To play around with using HAML, you can use aspnet-haml to support .haml files though ASP.NET.

Ruby on Rails - Databases with Ruby

One of the most popular (or most buzzed) web-frameworks is Ruby on Rails, which is just a collection of Ruby libraries for structuring your web-application. Rails uses the Model-View-Controller pattern, so any ASP.NET MVC people will find it a very-familiar framework. However, Rails really shines when it comes to interacting with the database through it's ActiveRecord library (the “Model” layer). ActiveRecord maps Ruby classes to database tables, and provides an Ruby abstraction for interacting with the database:

This is all the code that is required to map your Ruby classes to the database, as well as create the structure of the database. ActiveRecord dynamically provides getters/setters for the table, as well as sets up foreign-keys and relationships based on conventions (belongs_to :posts assumes that the table has a 'post_id' field).

Interacting with the database is just as easy as calling methods; Post.all translates to the SELECT * from posts SQL query, since the Post object is mapped to the posts database table. Post.find(<id>) does a SELECT * from posts where id=<id>, etc.

Because Rails uses the Rack web-server interface, it will also run on IIS using IronRuby.Rack. See the IronRuby Rails documentation for more info about using Rails on IronRuby, and the Ruby on Rails documentation for general Rails usage.

ASP.NET MVC and IronRuby

Those were all Ruby-based web-frameworks, but what about using ASP.NET directly? The IronRuby community has developed an integration with ASP.NET MVC, so you can write your controllers and views in Ruby. Special thanks to Ivan Porto Carrero for single-handedly maintaining it.

ASP.NET and IronPython

Again, to give Python some love, IronPython directly integrates with ASP.NET, letting you write your ASPX code-behind files in Python.

Because of ASP.NET's events-based API (rather than a response-based API like Sinatra/Rails), Python methods can automatically hook events by using the <object>_<event-name> convention, and all server-side controls with "ID"s ends up being a variable available to the Python module. And application-level event hooks can go in global.py. But it's really nice to write Language="IronPython" at the top. =)

Python code can can also interact with the controls:

The <%# %> syntax lets run Python code in the context of the ASP.NET control's data source. The repeater's data-source was set to a list of IMAGETAGS (a python class), which has all those fields on it.

In conclusion, on the server you have many options for using IronRuby and IronPython to simplify your solutions.

Next up, using dynamic languages on the client through Silverlight.

MIX10, Part 1 – IronRuby, IronPython, and the web

This post is part of the “MIX10 – Pumping Iron on the web” series:

• Part 1 - Introduction
• Part 2.1 – Server-side web development with dynamic languages
• Part 2.2 – Client-side web development with dynamic languages
• Part 3 - Using scripting in static applications
• Coming soon: Part 4 - Web-app extensibility with scripts

The original post was mistakenly removed, so it’s been reposted with the original post date, 4/24/2010. Sorry if this confuses your blog readers or causes any other inconvenience.

This past week I had the pleasure of attending and speaking at MIX10 in Las Vegas about using IronRuby and IronPython on the web. If you’re a TV-person instead of a reading-person, here’s a video of the talk:

As I usually do, this series of posts will be a write-up of my talk … but first …

iron? - Jim Hugunin and John Lam have both been quoted as saying "Iron" stands for different acronyms; "Implementation running on .NET" and "It runs on .NET", respectively. I’m going to put my foot down and officially side with Jim, though really they are bacronyms; neither is actually the original meaning. A StackOverflow post explains more, and I hope that puts the wondering to rest.

This talk is all about the why and how of embrace dynamic languages on Microsoft's web platform - be it on the web-server (IIS) or in the web-browser (Silverlight), and even in existing applications. To start out with, here’s my rational for why we as a developer community should care:

It’s no secret that developers like things to be simple, and the web is pretty simple as far as application models go. While the web’s feature-set is pretty attractive itself (server-client oriented, instant client deployment, and cross-platform clients to name a few), the equally attractive development experience (simple UI mark-up system and a scripting language) still make it easy for people to build amazing websites.

Though the application model is simple, developers continue to evolve it; server and client frameworks are vital tools that make the web-development experience even more productive – it’s very rare that a website has no server side or client side dependencies.

These frameworks provide such innovative features because they stand on the shoulders of powerful and expressive dynamic/scripting languages, giving the frameworks the unique ability to model the "web" as they see fit.

While each web-framework is powerful in its own right, the power really comes from the choice to use whatever tools help get things done. Developers and designers for .NET have the same need to get things done, and if getting things done is essentially the result of programming language choice, what choices are there? Really only C# and VB, which are static programming languages, requiring a compile step before execution and relying on debugging to see code in action. Take a look at the other languages mainly used on the web again -- they're all dynamic languages, running from source code, and providing interactive environments for running code. Why is .NET static while the rest of the web is dynamic? Why can't they exist together? If .NET provided some language choice for developers, all the languages be used together, and the .NET community could benefit from the amazing work being done by dynamic language community, and visa versa.

We in-fact live in this world, and embracing dynamic languages is possible on .NET, but why would you want to do it? I'll discuss this in the following posts (links to posts coming soon):

• Server-side web development with dynamic languages
• Client-side web development with dynamic languages
• Using dynamic languages in your existing web-applications
• Coming soon: Opening up your applications to extensibility with scripts