Wednesday, November 11, 2009

Embedding IronPython in Silverlight - Importing

I’ve heard plenty of times on the IronPython Mailing List that embedding IronPython in Silverlight is easy at first, but then you fall off a cliff when trying to import, be it from a .NET namespace, the Python built-ins, or even other Python files. Let me clear some things up, and fix some code in the process.

Slight detour – Embedding in Silverlight 101

First off, Michael Foord has an article on his website about Embedding IronPython in Silverlight, which still works. However, that’s a ton of complicated code, all having to do with creating a ScriptRuntime instance correctly configured for Silverlight. To make this much less verbose, DynamicEngine.CreateRuntime (in the Microsoft.Scripting.Silverlight namespace) will create a ScriptRuntime all prepped for use in Silverlight; this significantly reduces the boilerplate hosting code:

In case you need to customize your ScriptRuntime, the DynamicEngine.CreateRuntimeSetup method is there to create a ScriptRuntimeSetup object configured for Silverlight, which then you can tweak things as necessary and create your own ScriptRuntime.

Note: The IronPython sources currently are broken for embedding, but in the next few days the sources should reflect what this post shows.

Testing that "import" works

Ok, great, the boilerplate hosting code doesn’t need to look hideous, but let’s get back on track.

Embedding IronPython in Silverlight - Importing test

Embedding IronPython in Silverlight – importing test (sources)

Here’s a quick test of IronPython’s ability to import various modules in Silverlight, testing this exact code works while being embedded:

None of these worked when I first wrote the tests, since there was an exception being thrown in the XAP-virtual-filesystem, reported yesterday on the IronPython mailing list. But with that fixed, the import bar.baz case still failed:

[FAIL]
Traceback (most recent call last):
  at test_import in foo.py, line 5
ImportError: No module named bar.baz

Here's the layout of the Python files in the XAP

foo.py
bar/
bar/__init__.py
bar/baz.py

Application.GetResourceStream("bar/__init__.py") was returning null, though bar/__init__.py was definitely inside the XAP. However, bar/__init__.py was empty, and it turns out that Application.GetResourceStream returns null if the file is empty as well as if it doesn’t exist! Putting a single space in bar/__init__.py (or anything that will make the file not empty … I suggest a comment) causes it to be found, and then bar/baz.py can be imported properly.

So, moral of the story:
“an empty file in the XAP is a non-existent file!”

Anyway, I hope that clears up some of the issues most people initially face with embedding IronPython in Silverlight. Let me know if there are any more issues.

Note: As I said before, the current IronPython sources are broken for embedding, but in the next few days the sources should reflect what this post shows. If you’re impatient, grab these binaries.

4 comments:

kayschluehr said...

This quite naturally raises the question of modules/scripts not living in the xap file containing also the DLR?

It might be possible ( I haven't checked this ) to write import hooks used to load scripts from remote URLs but this would be out of touch with the DLR and its API which standardizes access beyond individual languages. Moreover each application developer had to create their own import infrastructure supplement - something I'd like to avoid if I can.

Jimmy Schementi said...

@kayschluehr we already support this with the new script-tags feature. The decision to use script-tags was because you need a list of files to download when the page starts --- you cannot pause the execution of the script to asynchronously download something, and then resume execution (the CLR does not support continuations). Async downloading is required because all downloading in the browser is done on the UI thread.

For more documentation about this, read: http://ironruby.com/browser/sl-back-to-just-text.pdf

kayschluehr said...

Isn't this an interpreter problem? I don't see why the interpreter can't pause on import. The UI might be frozen or creates a wait indicator - something I do consider for my own app. This should be easier than doing general interrupts of the interpreter which doesn't work either right now.

Not sure what to make with script tags? I suppose neither interactive nor Prism style applications are feasible right now using the DLR. So what is precisely the selling point? Wasn't interactivity one of yours on your own blog?

Jimmy Schementi said...

>>> Isn't this an interpreter problem?

Well, it's not something IronPython can address; it's a VM problem. The CLR does not support "continuations", which are what you're explaining ... pause the execution of a thread (lifting the current state), change that thread's instruction pointer (bring in it's state), and then at some point resuming execution at the original fork.

Without this there is no way to write a procedular (synchronous) import hook in Silverlight which runs on the UI thread (just like JavaScript), without hanging the UI thread. Not hanging the UI thread is importnant, since Silverlight runs in the same process as Firefox and IE, so the entire browser, including all tabs running in the browser, will hang. In Chrome all plugins run in a different process, but Chrome is extremely sensitive to processes not responding, and it will tear down a blocked plugin process very agressively.

However, you could write this asynchronously, but it doesn't look very Pythonic.

downloading_import 'foo', on_complete
def on_complete(module)
module.bar() # equivalent to foo.bar()
# rest of your program

You can also imagine running the Python code on a background thread to start, and when the import begins immediately dispatching the download to the UI thread, stopping the user-code background thread, and when the download is completed restarting the background thread. However, changing the thread user-code runs on would make writing Python code more painful, since it would require an explicit dispatch to the UI thread for all UI manipulations.

>>> Not sure what to make
>>> with script tags?
It's just a simpler application model for Python browser apps.

>>> I suppose neither interactive nor Prism style applications
>>> are feasible right now using the DLR. So what is precisely
>>> the selling point? Wasn't interactivity one of yours on your
>>> own blog?

What made you come to that conclusion? The DLR lets you treat everything as a string, and you can download things as much as you want, but no asynchronous activities can happen while running user-code; user-code blocks until it finishes, as there is no way to interrupt (unless you run on another thread like I suggested above).

What exactly do you want to do that you don't think is possible with the DLR?