I used Reflexil to change the assembly references, then tried loading in Silverlight.
Kaboom, hit missing type exceptions.
Tried to modify the assemblies to not include the 'missing' types, but still hitting errors.
My conclusion is that it is definitely possible, but require some work to get it right.
I also have an issue what to do with library files as they are not compiled untill used.