Friday, March 27, 2009

Testing C# Silverlight apps with IronRuby

http://blog-well.com/2008/03/04/100-resources-for-web-developers/

« This post is part of the combined
IronRuby and AgDLR release at MIX '09

Testing is a great use for dynamic languages, especially if the application being tested is written in a compiled language like C# or VB. Ruby is an especially interesting language for writing tests in due to the popularity of RSpec, a behavior-driven-development testing framework. IronRuby makes it really easy to test .NET applications because of it's .NET integration and ability to run real Ruby programs, like RSpec. However, this becomes much more difficult to do in Silverlight for a variety of reasons. And Silverlight doesn't have good options for testing to begin with, except for the Silverlight Unit Test Framework, but I yearn for something simpler. This post will show you how to test a C# Silverlight application with IronRuby, using a testing framework called Bacon, a lightweight RSpec clone.

Note: A similar exercise could be performed with IronPython and a Python testing framework like unittest or PyUnit, but I have much less experience with writing test code in Python. If anyone else does, please feel free to put together a similar package for IronPython).

Demo: testing a calculator on steroids

calculator-tests

Download MIX '09 demos

This example is in the /Calculator directory. Special thanks to Boyan Nikolov for implementing the Calculator control.

This is a example of testing a calculator (I'll explain the right-side with the Python code in the next post); the tests run in their own text-based REPL console, and even let you poke and prod the application after it's finished running the tests. The tests are very succinct and read easily, showing a big advantage to using IronRuby.

The tests do not actually click the buttons, as Silverlight doesn't have extensive UI automation support like WPF does, but the tests call the methods that are hooked to the events, and also test that the controls hook the correct method, making it an equivalent test, and a lot less magic-y.

For another testing example, see the spreadsheet example John Lam put together for his MIX '09 talk.

Test any Silverlight app

Testing any Silverlight application is pretty straight-forward, so let's walk through how to set it up. You can start from either your own Silverlight application, or a new Silverlight application, however you'll need a website/web-application project to be serving the actual Silverlight application; if you're creating a new application simply use the default options.

First, download eggs.xap (I'll explain what it is later) and place it at the same location in your webserver as your application's XAP file is being served; most likely the ClientBin directory. If you put it anywhere else, make sure to account for that in the code below. Note: Eggs.xap will be part of agdlr in 0.5.1, which will come out in a week from this post's date.

Next, download this Web Application project, rename it to whatever you want, and add it to the same solution as your Silverlight application; this will hold our tests.

There is a Sample.Tests/tests directory in that project, with a single sample_test.rb file; that where the actual tests will go. You’ll also see eggs_config.rb; this file defines the list of tests we want to run, since Silverlight can't enumerate the files in a XAP file easily.

The clientaccesspolicy.xml file makes it possible to download the generated XAP file, since our Silverlight application is being served from a different web-server. You can move these tests to the same domain as the application, but for the sake of this demonstration they are different.

To get your application running the tests, add the following C# code to your application at the point which you want tests to run; this might be on Application_Startup, or at some later time ... it's up to you. If you're not sure, just add this code to the method which handles Application_Startup in App.xaml.cs to download eggs.xap and run the tests, only when the query string contains "test".

When it is all said and done, run your application with "?test" at the end of the URL, and you'll see something like this:

eggs-firstrun

The tests run inside a HTML-based IronRuby REPL control, so you can inspect your application after the tests run, maybe to track down a failure or prototype your next test.

Note: if you don't see the REPL in your application, or just see the REPL and not the red menu bar below, the Silverlight control is blocking its view. If you make your Silverlight control "windowless", by adding <param name='windowless' value='true' />" to the Silverlight object tag, or Windowless='True' to the ASP.NET control which creates the object tag, it will allow the REPL to show up. If for some reason you can't make your control windowless, you'll need to set the width and height of the Silverlight control so it doesn't overlap the console.

How it works

Everything above is pretty transparent on what it's doing, except for two parts: (1) what is this "eggs.xap" thing, and (2) how did my tests magically become a XAP file?

Eggs

"eggs.xap" is a small library to run Bacon in Silverlight. It containing bacon.rb, the Bacon framework, eggs.rb, a Bacon test runner for Silverlight, Eggs.dll, a simple .NET API for running the tests, and DLLs required to run IronRuby.

When you added the code to App.xaml.cs, it:

  • Downloaded eggs.xap
  • Pulled out Eggs.dll and loaded it
  • Runs the tests in the supplied XAP via "Eggs.Start"

Eggs.Start loads IronRuby's DLLs and and uses the DLR Hosting APIs to call the Ruby Eggs.run method. This is abstracted by Eggs.dll so the application being tested doesn't need a compile-time dependency on IronRuby.

Chiron.XapHttpHandler

All you did was put tests in a web application project, so how are they getting packaged up into a XAP file? In that project's web.config, there's one very important line:

Yes, Chiron's XAPing functionality is being used, but it's packaged in an ASP.NET HttpHandler, giving any ASP.NET application the ability to auto-XAP a directory when requested, without having to use Chiron's web-server functionality. So when you give Eggs.start the Uri to the tests (http://localhost:35863/Calculator.Tests.xap), it resolves to that project, sees that the Calculator.Tests folder exists but no XAP file exists, so it creates one in memory and responds to the request with it. Here is the implementation if you are curious.

This same strategy can be used to develop Python or Ruby Silverlight applications in Visual Studio.

Now, get to testing!

Now you have no excuses for not testing your Silverlight application! Hit me up on twitter (@jschementi) or the comments if you have any questions.

Monday, March 23, 2009

Silverlight Dynamic Languages SDK 0.5

AgDLR

In conjunction with MIX ‘09 and the announcement of IronRuby 0.3, I’d like to announce a new release of AgDLR 0.5 – the Silverlight Dynamic Languages SDK.

Download AgDLR 0.5

You can also check out the source code for this release

This is the MIX ‘09 release for Silverlight 2 and Silverlight 3 Beta. It updates the languages to IronRuby 0.3 and IronPython 2.6 pre-alpha, and adds a bunch of new features to Microsoft.Scripting.Silverlight and Chiron. Please read the release notes for details, but I’ll highlight the awesomeness below.

Silverlight 3 Beta

There is a separate package (agdlr-0.5.0-sl3b.zip) which contains binaries specifically for Silverlight 3 Beta. Though the Silverlight 2 binaries will work just fine in SL3, they are slightly different. The Silverlight 3 binaries use the Silverlight Transparent Platform Extension feature to download the DLR assemblies on demand, significantly reducing the size of a IronRuby or IronPython Silverlight application. This feature warrants a post just for itself, so that’ll be coming shortly.

REPL

image

Adding “console=true” to the initParams value of a Silverlight control will add a dynamic language REPL to any Silverlight app for a explorative developer experience. See samples/python/repl and samples/ruby/repl as well for how to use REPL programmatically ... in three lines of code.

Testing

image

Having a REPL in the browser is extremely liberating, because all your console applications can run as well. A killer application of this in the browser is testing, so this release has made it really easy to write tests for your Silverlight application, be it written in Ruby, Python, C# or VB, using IronRuby and a small Ruby testing framework, Bacon. This little subproject is called Eggs, and will be elaborated on in a following post.

AgDLR uses Bacon directly to test itself in the browser, making up a test-suite contained in agdlr-0.5.0-test.zip. This suite contains a lot of new tests written in Ruby, as well as ported Python tests from an internal Microsoft DLR Silverlight test suite.

Contributions

Thanks to Dan Eloff for contributing two fixes to Microsoft.Scripting.Silverlight, and Harry Pierson for Chiron.XapHttpHandler, a way to have IIS and the ASP.NET Development WebServer auto-xap dynamic language applications (which is freakin’ awesome, and I’m going to write about it shortly as well).

So, download agdlr-0.5.0 and ...

image From http://www.zune.net

Friday, March 20, 2009

IronRuby 0.3 Released

ironruby-400

I’m pleased to announce a new release of IronRuby: IronRuby 0.3!

Download IronRuby 0.3

You can also check out the source code for this release

Staying true to our “Conference-driven development schedule,” this release is in conjunction with MIX ‘09, where Silverlight 3 Beta and ASP.NET MVC 1.0 were announced, to name a few. John Lam spoke at MIX today at 12:30pm in Las Vegas about using dynamic languages in Silverlight for testing, scripting, and application building. And some ASP.NET MVC with IronRuby is thrown in there too.

The demos are also available for download here. Check out http://github.com/jschementi/mix09 for information on what the demos are, and how to use them. Also, throughout the day I will be posting walk-through-style posts about each major usage of dynamic languages in Silverlight.

Now that that’s out of the way, what did we do in this release?

RubySpec

IronRuby passes approximately 80% of RubySpec, the best test suite Ruby has today. IronRuby is best at language compatibility, passing 95% of those tests, but worse on the standard libraries with a pass-rate of 77%. Overall the pass-rate as gone up approximately 10% since the last release, and there hasn’t been a major library push since getting Rails running. For IronRuby 1.0, that number is going to be as high as possible, and we’ll ensure that by continuing to work with the RubySpec project to improve the test suite.

IronRuby.info shows up-to-date RubySpec statistics about IronRuby.

.NET inter-operation

Since this release is around MIX, and John’s talk is going to be focused around Silverlight and ASP.NET MVC, it seems like a great time to polish IronRuby’s .NET interop. For example, the release enables calling generic methods, implementing interfaces with IronRuby, and better conversions between .NET and Ruby types. We’ve also started to build a .NET interop test suite, using the same infrastructure RubySpec uses (MSpec), to provide a executable specification of how IronRuby and .NET play together. There will also be a written specification showing what these .NET interop features are good for, but not for this release.

For more specifics of what this release contains, checkout out the CHANGELOG.txt in the binary download.

What’s with the version number?

IronRuby is going to be a bit unconventional-Microsoft (as if that’s something new) and simply release 0.x releases (and maybe 0.x.x if needed) until it gets to 1.0, rather than release something with “Alpha”, “Beta”, or “RTW” attached to the name. That being said, 0.3 is a complete underestimation of where IronRuby is; it’s definitely more than 30% completed. We’ll move through the point releases rather fast, compared to the 7 months it took to get 0.1 (Alpha 1) releases, and another 3-4 months for (Alpha 2). Any maybe even skip some numbers along the way.

What’s next?

While the .NET interop testing/specing work will continue, the next releases will be much more focused on getting real Ruby applications running well; whether they be already existing apps/frameworks like Rails, RubyGems, Rake, etc, or new applications that people are building with IronRuby. We’re starting to run the gems, rake, and rails tests with IronRuby, and this type of discovery with more Ruby libraries will drive what we work on next.

In short, expect to see more binary releases, more often. Until then, Download IronRuby 0.3!

Thursday, March 05, 2009

State of the DLR for Silverlight

If you’re a frequent reader of this blog, you definitely know what’s going on with the DLR in Silverlight. For those who are new, or just don’t know, hopefully this post will clarify some things.

First off: my previous entries about Silverlight since sdlsdk-0.4.0 dropped:

It would seem the next release will be focused around REPLs, testing, scripting support, and maybe some server-side integration pieces. And you’d be right. Proof is in the project’s public source code repository.

But what about the dynamic language runtime and the languages themselves?

There has not been a new release of sdlsdk for many months, and that’s simply because the above features are not finished, NOT because the DLR and languages have not releases anything for Silverlight since then! The DLR released dlr-0.9 and IronPython released ironpython-2.0.1, both including Silverlight binaries. Also, Harry Pierson put together a nightly build site for the DLR, which includes IronRuby and IronPython, and had Silverlight-specific builds.

Then why the sdlsdk project?

sdlsdk is the main project and release vehicle for Chiron and Microsoft.Scripting.Silverlight, which makes writing applications in DLR-languages possible. It’s also the place for collaborating and experimenting with crazy usages of Silverlight+DLR. The DLR and languages are parts of other projects, so I can not justify bumping the version number of sdlsdk simply because a new version of IronRuby, IronPython, and the DLR are in there. The easiest way to update sdlsdk’s DLR+languages is to grab a daily build of the DLR and drop the binaries into sdlsdk’s /bin folder. The next sdlsdk release might include a script to automate that, as long as new builds don’t break it. =)

This is possible because the DLR, IronRuby, and IronPython teams use a shared continuous-integration server that runs test suites in Silverlight on each checkin, ensuring that they all continue to work really well.

Common misconceptions

During development the state of these projects could be completely different within a few weeks, so misconceptions form very fast. Here are ones which beg for correction …

Chiron, a special HTTP server, performs language compilation on the server and generates a .xap file.

FALSE. While Chiron is a special HTTP server, which generates a XAP file for any given directory, it does not perform language compilation on the server! Chiron is a development-time tool which only sticks the script files in the XAP, and they are run in the browser. When putting your app into production, you tell Chiron to write the XAP file to disk, and then you can put your app on any web-server you want.

Chiron’s existence shows the limitation Silverlight imposes by requiring a XAP file. DLLs can be downloaded by the AppManifest.xaml outside the XAP, so the XAP just serves as a AppManifest.xaml container. I’m hoping to get this fixed by Silverlight 4. =\ It also shows a limitation in not supporting continuations, which would let the user script pause while asynchronously downloading a script file, and then resume the initial script. I’ve been prototyping ways to get around this, but nothing working yet.

Also, if you’re using another web-server in your app (like you have a ASP.NET app with a Silverlight control), you can just set up a file-system-watcher and tell Chiron to regenerate the XAP to disk everytime you edit a file, but it will be served with the other webserver. This is used by silverline, a Rails plugin I wrote to integration Silverlight and IronRuby.

This is why you can’t include JavaScript files from inside other JavaScript files, and all your dependencies need to be pre-defined in the HTML file, because JavaScript can’t download the script files while the JavaScript engine is running.

Common interface implementations were inconsistent between languages, so I can’t create a solution where advanced users can pick and choose a language.

FALSE, in theory. The DLR hosting API (contained in Microsoft.Scripting.dll) provides a single set of APIs that can be used to host many languages. For example, Microsoft.Scripting.Silverlight uses the DLR hosting APIs to run any DLR language in Silverlight, based on what extension the entry-point script has (.py, .rb, etc ...). There are simple hosting examples here.

It is the language’s responsibility to implement certain interfaces so they are host-able by the DLR. IronRuby and IronPython are existing examples of that, and there is documentation in the hosting spec on how this all works, so new languages can follow suit.

However, there are always going to be language-specific scenarios which have different ways of using the hosting API, or the hosting API doesn’t support the feature and you have to use the language’s hosting API. In those cases, it’s best to re-think your scripting design and make the hosting code as simple as possible.

Also, a couple months ago, IronRuby’s hosting support was no where near where it is today. Again, things change fast. If you have any feedback on DLR hosting, please ask around on the DLR Discussions List.

And the state is ...

I hope that made some things clear; the DLR, IronRuby, and IronPython are committed to supporting Silverlight developers. Unfortunately, the reverse is less strong, so if this is important to you please talk about it on the silverlight.net forums. Things might be a bit nicer when the inner ring of the DLR (Microsoft.Scripting.Core.dll) is part of the System.Core in Silverlight, but we’ll see.