Code Excavations and Domain Specific Test Frameworks

[00:07] (external link, opens in new tab) Hi everyone. Today's presentation is going to be a very interesting for me, not necessarily for you. And it's because there's a whole new technique that that I've I've stumbled upon which I'm going to introduce and I'm going to introduce some patterns for u breaking dependencies (external link, opens in new tab) in frameworks where framework is actually a host is an envelope for the code that you're trying to write. A little bit about me. If you want to follow me on Twitter, that's my Twitter address. I'm I started a new blog called 5whys.com which is actually not about coding.

[00:42] (external link, opens in new tab) It's about team leading and it's a whole subject which I think is very missing. So we in in everything that we do as programmers there's the technical side but then there's the people side and there are a lot of problems that come from the people side. So even if we if we all believe in unit testing (external link, opens in new tab), we can't a lot of times we're the only ones on the team and we can't we don't know how to convince other people or we cannot lead the team that that's under our control to do the stuff we believe in and team leadership is something that's missing and that's that's why I started a course about it and a blog about it and I think it's going to be the next wave in the agile development space not just technical.

[01:23] (external link, opens in new tab) I work for a company called Typemock and we just recently released a free tool called Test Lint. So if you write unit tests in .NET and in Visual Studio 2010 it's a free plugin that will actually parse your code as you type it and will detect common errors. For example, when you have logic inside a unit test, it will actually flag it, give you a nice popup inside Visual Studio, tell you about it. It's extensible. you can write your own rules and again it's it's free and extensible.

[01:54] (external link, opens in new tab) So this is my own little pet project. I think that's that if we are trying to teach other people to do the stuff we believe in that we need to give them tools to help them to do it. And I think that's that could be a step in the right direction. I'm also the author of a book called the art of unit testing (external link, opens in new tab). And today I will be giving one free copy of that book. I will explain how you can get it later. I will be coming back here in July 19th to do a 5-day TDD master class with Iimea.

[02:28] (external link, opens in new tab) So if you are interested there they actually have a booth right here. Feel free to talk to them about details. You can go and read about my book at artofunittesting.com (external link, opens in new tab). So today I'm going to talk a little bit about what cogs are. I think we all know the the problems that that we're facing. And I'm going to show several solutions to the untestability problem when we try to write code that's hosted somewhere.

[02:59] (external link, opens in new tab) I'm going to introduce two new terms excavation (external link, opens in new tab) which is a new technique or it's not new but it's new for me and it introduces a new a new a new way of testing something. And it's a way to create what's called a domain specific test framework. And I'm going to show demos of these things as well. And I'm going to show how how we do that, how we try to solve the same problem today. If we have time, I'll do a little song about unit testing.

[03:33] (external link, opens in new tab) But first of all, I want to show an example of what a cog in a machine feels like. And the example is taken directly from my own life experience of trying to write unit tests for something for a control in Silverlight. Is anyone here has anyone here ever looked at Silverlight related stuff? Okay, so about half the people here. Has anyone here ever tried to do unit testing on anything? Not just Silverlight, just unit testing. Okay, so that's the kind of crowd that I like.

[04:04] (external link, opens in new tab) You like to raise your hands? That's always good. An example of a cog is something that to that you can't really just create a simple unit test for a complete box is a control built by Microsoft as an open source control that runs only inside the Silverlight runtime that is inside the browser. If we try to instantiate it in a simple unit test, we try to run this test,

[04:36] (external link, opens in new tab) We will actually get plenty of errors that have to do with a lot of dependencies down the line. Just as a short example, this is the whole stack trace that we get. We try to instantiate this and then there's a constructor that's being called and a framework element as a base class and another constructor on the base class another constructor and at some point this whole thing is calling some other type by that's created as part of the Silverlight runtime called XCP imports jolt helper

[05:15] (external link, opens in new tab) The point is that even though the control looks harmless it's actually like like the root of a tree. You can only see the top of it, but the roots go way deep because it inherits from the system. So just trying to to to create an instance of that object invokes multiple dependencies inside the system. So it's basically untestable because it inherits from the system. So there are different ways to test it, but unit testing is not one of them right now.

[05:46] (external link, opens in new tab) There are integration ways of testing the same thing. And I'll show a way for integration testing (external link, opens in new tab) that problem. But cog structures are are the ways code can be tied to the system that's enveloping it. For example, a cog might inherit from an object from the envelope framework. What is an envelope framework? Silverlight. SharePoint is something that hosts your code.

[06:19] (external link, opens in new tab) Windows workflow innet the CLR and Java runtimes are actually envelopes as well and you can actually replace both of them you can actually create your own envelope of the CLR you can actually host a CLR instance and actually make the system work differently so think about it a piece of compilable code is also a cog in a machine which is the CLR or in the Java runtime Visual Studio or anything that loads plugins is a host is an envelope that treats your code as part of it.

[06:53] (external link, opens in new tab) But for plugins, there is a testability (external link, opens in new tab) story because they have contracts which are well defined. So inheriting from an envelope object is very very common to be part of a system. Sometimes you don't have to inherit from anything but to get things done in the system you have to call a lot of static methods. You have to create a lot of instances of system types as part of your code. And sometimes you just have to implement interfaces from the system. And again, interfaces are okay because they are the equivalent of a contract which is fully replaceable.

[07:29] (external link, opens in new tab) So you can replace the system when you have interfaces and they are the only means to contact the envelope. So I showed the example of just creating a simple control and the truth is that there is really no way that Microsoft could have found to create a real unit test for it. So what they did is they created a whole new test framework specifically designed for Silverlight.

[07:59] (external link, opens in new tab) Now it's not a unit testing framework although they call it that way. It's an integration testing framework. It invokes the browser. It actually loads the browser instance. It runs the code inside the browser. Now, because everything runs inside the browser, everything is asynchronous. So, what happens is that your tests that kind of look like unit tests now have to know about asynchronicity. And that means that you have to use a lot of interesting attributes and function calls that are built into the framework that Microsoft created.

[08:30] (external link, opens in new tab) For example, you have to put the asynchronous attribute on the test. You can put priorities on the specific test. Whenever you want to do something that has that is touching the browser UI or visuals or something like that, you can't just do it because the browser is running on a different process than the test. So, you have to incue it. you have to actually create a queue of actions which are incued inside the browser and are executed as part of the Silverlight runtime.

[09:03] (external link, opens in new tab) So every action you would have done normally without threads such as creating adding a child to some test panel asserting something setting text on the control asserting that something is not null or whatever now have to be enclosed in a what's called an inq function which is part of the test framework and what it does the the syntax that you're seeing the the arrow actually means that the whole thing here is an anonymous method.

[09:35] (external link, opens in new tab) It's basically a delegate inside the braces. So, it actually means run whatever is inside these braces later on in the browser in Q these things. Now, because these are all incued and I and this is asynchronous, I have no idea when the test is really complete and the browser never knows. So I have to also remember to do inq test complete so that the browser can actually shut down or the browser can stop waiting because my test will have to wait until the actual test in the other process finishes.

[10:07] (external link, opens in new tab) So if I don't call this method now my test becomes stuck because the browser doesn't know I don't know when the test actually finished in the browser. So this is a horrible way of doing things. I think we can all agree that just learning the syntax and trying to understand whatever is written in this in this test it's practically impossible. It took me two days to find out that I forgot to do in Q test complete at some point. Ju just imagine >> get informed whether >> sorry >> does this tester even get informed whether the assertions

[10:43] (external link, opens in new tab) >> does the test get informed whether the what >> whether the assertions >> whether the assertion worked. Yes, it gets informed. The framework actually will get back to you all the exceptions and it will inform you but all the exceptions are written in the browser. So you have to look at the browser. You have to be there as a person. You cannot run this as part of an automated build (external link, opens in new tab). You actually have to be there sitting there watching the tests. Okay? And if if you see something red then means it's problem and your test will fail but all the results will be in the browser.

[11:17] (external link, opens in new tab) There's a special test runner (external link, opens in new tab) in the browser. So there are other problems with asynchronicity. For example, because it runs in the browser, sometimes there are timing issues. So the tests, like Net likes to say, flicker. Sometimes they work, sometimes they don't work. Thankfully it's only like 5% of the time that they fail for timing issues. So that's great. That's perfect. That's exactly what you want, a test that only 5% of the time fails because that's a measurable

[11:48] (external link, opens in new tab) Good thing. So when we try to to do unit testing against such a system and against such a cog or piece of code, we need to ask ourselves, does the system allow itself to be replaceable in some way? For example, the CLR is actually allowing me to replace the whole envelope. you can actually host your own CLR and run compiled code with a different behavior. For example, you can create a CLR that traces all the method calls.

[12:20] (external link, opens in new tab) It's very simple to do once you know how to do it and you only do it once. You create that that envelope and you do it. So imagine the folks that created the CLR. That's how they tested it. That's how they tested that you can actually compile something that you can actually take the output and put it somewhere that you can check the output from and so on. If the system is replaceable, it means that you can either implement interfaces or you can easily handcode your own adapter code against the the new code.

[12:52] (external link, opens in new tab) For example, if SharePoint was really easy to program against, I could say, you know what, it's a really good idea to take whatever code is trying to use SharePoint and wrap it with a role-based SharePoint model of your own which only has two, three, five methods or so and so on. But when you have a chatty interface into a system like you have to call the system continuously to get everything that you need with multiple methods that becomes a bit problematic because hand combing that stuff becomes a very big chore and

[13:28] (external link, opens in new tab) And it becomes unmaintainable at some point. But for sure for simple interact interaction systems this works great. And the third option is to use isol is isolation frameworks (external link, opens in new tab) to fake (external link, opens in new tab) parts of the system just in an ad hoc manner. So that you can actually for this test just say that the system for this interaction point for that method will return a fake object. You don't fake out the whole system. You just fake out two three points in the system. So for systems that are replaceable,

[14:00] (external link, opens in new tab) If we take the star here and the star represents the new code that we're trying to write and the the envelope system is in blue here. Our code needs to run in the system. Usually our code will not run alone. It runs inside other code that we already wrote for the system. So our code will interact with the code that we wrote which also interacts with the system. But any contract between us and the system because the system is replaceable is not really hardly defined.

[14:33] (external link, opens in new tab) That is it's very easy for me as as the code. I don't really care if I'm working with the real system or not. I work as if everything is just fine. So the boundary or the perimeter I call this a perimeter because boundary is a bit of an overloaded word. Now, the perimeter of testing or the perimeter of understanding if I'm against the real system is right here in the interaction points. And I don't care if I'm in the real system or not. As far as I'm concerned, it's always a real system. As far as the code is concerned, that's only true for replaceable systems, systems that allow replacing.

[15:10] (external link, opens in new tab) But we're not dealing with those kinds of systems because if we dealt with those kinds of systems, if we were to use a framework like JMock or Rhino Mocks (external link, opens in new tab) or Typemock, you would be able to easily just fake out parts of the system where you're trying to interact with. For example, your code only calls methods in this part of the system. So I don't need to wrap the whole thing. I just wrap whatever method that this test is currently calling.

[15:42] (external link, opens in new tab) When the system is not replaceable. What we used to do and what we a lot of people are still doing and definitely what's recommended in Steve's book is to actually detect roles in the system and surround your code under test with these role abstractions so that you don't actually talk to the real system, right? You just talk to those specific things that the system represents to you. So what does the parameter look like when you create your own wrapper or adapter system

[16:16] (external link, opens in new tab) Against the actual code? So again imagine that this is our code. This is how we interact with the system and the white line is the line that represents the code that's wrapping the code. That is this is our rolebased interaction system. So to my code, I have no knowledge that Silverlight or SharePoint actually exists. The only knowledge that my code has is that the wide thing around here, whatever roles I need to talk to, they exist.

[16:51] (external link, opens in new tab) So it's just a piece of pure logic. If it needs something from the system, it uses a specific role that gives it that information. So the code is purely independent. it doesn't actually have to run inside SharePoints tomorrow if we decide we move to a different framework. Now again this is great when you want to also look at the design of your system and it's great when you can do it in a how would I say when you do it in a in a in a time efficient manner.

[17:26] (external link, opens in new tab) If it takes you a year to write that it's a bit of a problem. And the third and the the the third way to do it and the way that I'm going to show is called excavation or basically I I'm taking the term from ar ar ar ar ar ar ar ar ar ar ar ar ar ar ar ar ar ar ar ar archaeological type of work because as you're going to see what I'm going to do is I'm going to try to invoke a piece of the system see what the problem is just like you saw look at the stack trace

[17:57] (external link, opens in new tab) Find the dependencies inside the system that have a problem which might not be the direct interaction that I need and and try to fabricate or fake out those specific dependencies which might be very deep inside the system and then I would continue to do another wishful invocation. What's wishful? Wishful means I wish if that if I just invoke this method nothing bad would happen. I can just invoke it in unit test and nothing bad would happen. It's wishful because I don't really know because it's part of the technique of excavating and finding the interactions of my code in the system.

[18:33] (external link, opens in new tab) So if imagine if I get started testing something in Silverlight and I have no idea what the dependencies are, the simplest thing for me to do is to try to invoke the constructor of an object and see what happens. That's wishful invocation. Now you do this repeatedly and every time you fabricate an interaction, you actually close one loophole in the system. So that eventually you can get an interaction against an object that inherits from the system or anything but

[19:04] (external link, opens in new tab) The system just works. So any invocation that we need actually just works. Just a what would a perime a parimeter like that look like? It looks very different than what I just showed where you would create a wrapper or an adapter because what happens is that if I try to create an instance of the the code, the code might get an exception way deep inside the system, maybe five levels down in an inheritance chain where some static constructor or something tries to load an XML file which doesn't exist.

[19:42] (external link, opens in new tab) So I actually fabricate that specific piece of code which has nothing to do with the interactions against the real system. It could be five levels down. That means that I'm actually going down into the system and I'm creating a deeprooted u perimeter which is actually might have nothing to do with a specific test. It might work for a lot of other tests later. So I'm going to show a specific way of how I'm going to do it.

[20:14] (external link, opens in new tab) So again this is taken from my specific experience of creating a Silverlight unit test framework. And what I wanted to do is I wanted to I tried to run the tests that I showed you that are based on the Microsoft test framework. And I tried to write these tests. It was horrible. And I I was I was not going to do that anymore. So I try to find a way using AOP aspectoriented programming to intercept the calls into the system and fake out any inputs any outputs from the system and so on.

[20:50] (external link, opens in new tab) Now the way I'm going to do it is I'm going to use there are several tools that that that are able to do what I'm going to show. I'm going to use the tool that's the simplest for me and it's called it's a framework called CThru (external link, opens in new tab) and it's an open source framework that's based on a commercial tool. So it's an open source framework that's based on the Typemock open AOP API which is the one of the tools that my company produces and what it allows it allows runtime interception into any system even if it's pre-ompiled.

[21:24] (external link, opens in new tab) Now, of course, any aspectoriented program will will be able to do that given the ability to intercept compiled code and there are some frameworks that allow you to do it. the way I use the CThru to intercept parts of the system and in this case I'm just going to do it using a setup method (external link, opens in new tab) because this is going to be true for all the tests is that I add a reference to specific DLL and to the type API and I and I add a specific aspect to something called a CThru engine.

[22:01] (external link, opens in new tab) So the CThru engine is actually the engine that collects any entry points and allows me to decide what to do on any method call during the system runtime. That is any method call. I can intercept even calls to plugins in Visual Studio or anything else. Now the way you configure the CThru engine is by adding aspects to it which is just something which inherits from an aspect and I tell the CThru engine to start listening.

[22:33] (external link, opens in new tab) Inheriting from an aspect is basically inheriting from the aspect class and overriding once one important method called should intercept. Now, should intercept takes an an info such as the method that's about to be called the name, the type name, and all that stuff. And for now, I just return true for everything. I tell the engine to intercept everything. And of course, that means it's going to be much slower. Those tests are not as fast as unit tests, but it's good enough for me to get started because this is the excavation part.

[23:09] (external link, opens in new tab) I'm just trying to see what's going to happen. And now I can override specific methods. For example, I can override constructor behavior throughout the system. And I can say for any constructor in the system, if the type name of of the constructor in contains the dependency object type, then the method behavior will be skip the actual method. Don't really don't really call that method. Okay. And for any method behavior in the system, if the type name of the method is dependency object,

[23:41] (external link, opens in new tab) If if it's not dependency object return, so I only care about something that inherits from dependency object. What is dependency object? It's basically the thing that my control inherits from. If we look at the stack trace that I showed earlier, you will see that here in the stack trace, there is a constructor for a base class in dependency object. And this constructor is called for any control that's created in Silverlight. And eventually this constructor calls a whole bunch of things inside the system.

[24:12] (external link, opens in new tab) XCP import scrap whatever anything here. So I'm just going to disable that specific constructor. So I'm looking for a specific type and I'm disabling that constructor. I'm just going to show that for a second and I'm going to show what happens. So I'm going to run again this same line of code with the CThru engine running. And

[24:45] (external link, opens in new tab) What's going to happen is that I'm going to get a different exception. Now, how do I know it was running? Because inside the aspect, I actually wrote to the console. I said skipped a constructor. You will see that the stack trace looks very different. Now skip ctor is basically this piece of code here if type name is dependency object write line the method name and skip. So now this is the first step in my excavation and what I find out is that I disabled the constructor but there's still a problem.

[25:17] (external link, opens in new tab) There is still another dependency. So that's the second dependencies that I'm finding and what I'm finding is that by default someone is calling control set default style key. Where is this being called? It's being called from the actual code under test. So I so given that this is code that I want to test. What happens is that whenever you try to set values inside dependency objects they actually call another dependency in the systems because values in Silverlight and WPF in.net always use static dependencies and save the values somewhere else not especially inside the object.

[26:00] (external link, opens in new tab) This is what's called dependency properties. And dependency properties are nasty because they're all static. So any call to a regular object actually instantiates some static property that has a lot of instantiations of bad stuff. Now I just want to disable the internal stuff. Now set value internal is about three levels deep from the original interaction. But I'm going to disable anything that is set value because I don't care about really setting values internally.

[26:30] (external link, opens in new tab) I'm just faking out the system so that I can instantiate an object. We're going to talk later about what it means to actually mimic the system behavior, but for now I'm just stubbing everything. So I'm going to use this piece of code to say only work on dependency objects. And I can say for now or and that's another dependency which I'm going to show in a second. If the method name is set value internal then skip that method and continue and then I try to run the same test again.

[27:09] (external link, opens in new tab) So now skip ctor skip set value internal and I get a third dependency. Now this third dependency is from something called add loaded. What is add loaded? Basically what happens is that someone is calling dependency object addeventlister which gets native object which basically could not find filer assembly XML which is a different Silverlight XML assembly and

[27:40] (external link, opens in new tab) At this point I don't even care. I just want to mute this problem. So again, I'm going to add a third part to my excavation series and I'm going to add the method name for add event listener and I'm going to say don't do anything when someone tries to add an event. So I run this test again and hopefully now everything should be okay. So you can see I skipped all this stuff and now I can actually create an instance of a Silverlight object. Now, of course, this is all a big lie because I create the instance, but the instance is actually might set some values internally, might add some events.

[28:18] (external link, opens in new tab) It really has no behavior right now. So, what's left for me is to actually not just override and mute stuff. I now need to add some system behavior that looks like the real system. So I could actually override set value and say when someone tries to set value on the problematic system just save it in memory with the value name so that when someone asks for the value back you save it and you give it back. So I act like the real system when someone adds an event listener you take the delegate and you return it later

[28:49] (external link, opens in new tab) So that I act like the real system but I don't depend on the real system. So that was a simple excavation and to find the dependencies of an object and again this is why the perimeter looks like this because some of the stuff that I am actually replacing is not very close to the object. It could be three five levels down. I could maybe find very low-level static constructor that calls the system and sometimes I just disable a specific interface directly against the object.

[29:24] (external link, opens in new tab) So the parameter looks very different and of course it works because I have a full framework that's called SilverUnit (external link, opens in new tab) that's built on the same the same idea and this is what test look like for SilverUnit after imagine about a 3-day iteration where I at some point I also create a simple API to test verifications of autocomplete box. So what does the framework look like? There is an you you basically write a simple unit test.

[29:58] (external link, opens in new tab) You put an attribute on it. You create instances of objects. You can set their values and you can check you can assert on their values back. For example, you can check that in Silverlight you you have custom logic that says if someone gives you an an invalid minimum prefix length, it automatically resets to the next valid one. That's that simple logic behavior. the piece of code that that exercises this piece of that's exercised by this test

[30:28] (external link, opens in new tab) Exists here basically let's see this this is the piece of code that I'm actually testing now this piece of code is invoked in an event that's driven by a static property setter and when I have enough AOP capabilities I can definitely mimic that behavior and invoke any code by the user as if I'm the real system. So I'm able to test simple code

[31:01] (external link, opens in new tab) And this is an example of how I can use this the framework to override the idea of XAML. What is XAML? Who for those who don't know Silverlight and WPF all controls and visual stuff is also laid out in a XAML file which actually is kind of like an HTML that says what is the visual looking like and to whenever you try to create something it loads tries to loads the XAML for that object as well but I didn't want that dependency so instead of letting the XAML tell me all the layout and internal controls.

[31:40] (external link, opens in new tab) I'm actually using my own framework to set and inject controls into the parent control that's under test. So I can check all the logic that I need. No XAML included, nothing. This is a pure unit test. Everything runs in memory. No file dependency whatsoever. I can also check interactions against the system. When you call on apply template, you assert that visual state was changed to normal. This is a piece of logic that says when the Silverlight control is just created, it moves to a normal state.

[32:15] (external link, opens in new tab) Now, if you don't know the Silverlight object model, don't worry. It's just an example of something that is actually this is an assert that is specialized for the system. And this assert what it does, it actually collects values that were sent into the system. So whenever I need to, I can actually ask if something in the system was called. So the whole system can be used as one big mock objects (external link, opens in new tab) if I really need it. By default everything is fake and if I need to I can always ask the system something. So every value is saved.

[32:46] (external link, opens in new tab) Now there is no setup code that does all this stuff. The setup code is actually exists in this attribute. Okay. So imagine that I'm doing the same setup of overriding using aspects and all that stuff for all the the tests that you see here. Here's the third example of I showed of what a Microsoft test would look like.

[33:17] (external link, opens in new tab) Let's see. This is the Microsoft test framework. This is a test that checks text completion via text property. And it does all this stuff, this horrible stuff. I'm going to show you what this test looked like. as a unit test in SilverUnit. This is the same test. When you set the text box, the autocomplete box is updated. I call is enabled. I set the selection start. I set the text.

[33:48] (external link, opens in new tab) I actually let u make the control fire an event. So using SilverUnit I'm actually faking event invocations so that I can trigger this is actually the act part if you think about it the whole thing. So I'm triggering an event with with a piece of fake event arguments. I'm actually making a fake event argument type because part of the system actually the constructor for text changed events is actually internal.

[34:20] (external link, opens in new tab) So I can't really create it publicly. So again, SilverUnit and interception comes to my aid in this case, but I can actually fake event arguments into the system. And these are the asserts and this all runs in memory. And of course, you can add some more stuff into the framework. For example, you can add timing and reporting and all that stuff. So these are the three tests that you just saw and they all run inmemory, no problem. And again this is specifically for writing unit tests inside that system.

[34:53] (external link, opens in new tab) So some more important questions that we have are first of all what what are the three types of isolation for a specific cog in a machine. The the simplest one on a replaceable system is interaction based where you use frameworks like JMock and so on to create a parameter that's only partial to the code under test. So the code only interacts with these functions. So I only fake them out.

[35:24] (external link, opens in new tab) I might have found out very easily that I just need to fake these interactions. So I fake only them. There's the role-based contract which is much cleaner. The design is much easier to understand. It might take a long time to create. I'm going to talk about the differences very soon. You can definitely create this with TDD and you would get this automatically. There's the dependency based or the one that I just showed the excavated one which looks different.

[35:56] (external link, opens in new tab) Now the the interesting thing is or the the big surprise for me is is that I was really afraid that if you try to create a perimeter that that looks like this because you're actually faking out something that you don't own, you're actually faking out part of that system is that it's going to be very fragile because you have no control of that system. And because you're faking out stuff that's private or internal, then it's more likely to change. But the surprise was that that wasn't true at all for that system.

[36:31] (external link, opens in new tab) In Silverlight, I started SilverUnit when Silverlight was the version one. Right now, Silverlight version 4 is coming out and I didn't need to change one piece of code in the framework. And I think the reason is is for big systems that host stuff like that your your own cogs if you will is that the system is so heavy is that it's not likely to change at all. And this is just just a conjecture. It's I I don't I don't know if it's true, but it feels true because that's my own experience.

[37:03] (external link, opens in new tab) The system was very very not changeable in terms of the way Microsoft wanted to change things because it's already a monster and monster systems don't change internals like that. So I think that rolebased is very robust. Interactionbased is not as it can be very brittle because the interactions against these interfaces from your own test code keep changing. This is code you own. So no problem there. You have full control of that code. This is actually very robust as well or so it seems for me because

[37:38] (external link, opens in new tab) The interactions against the system can be much closer to the actual parameter. So even if the interactions with the real system change because what I do here is in such broad strokes I just mute whole types whole functions I just do everything so if I need to mute something I will mute all the static methods in that type I don't care because it's all internal then it's it actually doesn't care what interactions happen against the system at some point so it's very robust and again because the code internally in the system doesn't really change in the in the in the Silverlight framework

[38:15] (external link, opens in new tab) Then I don't need to change the actual SilverUnit framework as well. So it's robust both in terms of interactions with the external system and both in the idea of faking stuff in that external system. So excavated parameters seem pretty robust and I say seem because this could be not true for other systems but it feels like it would be true for systems like SharePoint. It feels like it would be true for systems like ASP.NET where you can someone actually created using CThru again their own ASP.NET net testing utility or API

[38:55] (external link, opens in new tab) Which intercepts HTTP runtime related calls and works the same way and he hasn't needed to change that in a year. So if the host system deep internals don't seem to change at all over versions. So creating deeprooted parameters for that for for testing could also be robust. They could be maintainable because you don't need to maintain them. So some more interesting points one of some one of the things that I need to do is I needed to cancel code that was asynchronous in the system.

[39:30] (external link, opens in new tab) A lot of things in the system actually created timers and expected those timers to fire at some point in time. So I I intercepted those timers and I invoked that function or that delegate immediately. So all timers in the system canvoke on the same thread never on different threads. So the whole test is fully synchronous. I had to mimic a lot of system behavior like event subscription. I had to save a lot of values. And everything in the system is saving values as well so that I can actually test that something was called because a lot of times your code that lives inside other people's system their end result (external link, opens in new tab) is calling that system.

[40:15] (external link, opens in new tab) For example, the logic says when a control is being clicked, it tells the system to change the visuals. That's all it does. And the system will take care of the rest. So that's an interaction boundary and I need to test that. And the idea of faking whole types, this is very different from regular testing where in an ad hoc manner you fake that point and that point and that point. When I do excavation, I mute whole types. I say, "Oh, I see something is calling Xcp imports.static method." I will fake everything in XCP imports.

[40:52] (external link, opens in new tab) I don't care about that type because it's so deep down. I can use these broad strokes to do it. The process was interesting as well because all these wishful invocations didn't just create my tests. My tests actually helped create my own domain specific test framework. And I think that when you have cogs inside machines and you use this the excavation stuff you will always or it looks like you will always get your own domain specific test framework out of it.

[41:23] (external link, opens in new tab) Now you create it basically once and you can reuse it. It's very reusable. And the tests are very robust. They're very readable. The end result is very readable, very simple to do and suddenly you have full control of that system. But the tests have no idea that they're running in a different system because what actually happens is the tests actually do run in the external system. We act we just disable parts of that real system. Some of the tests might actually be a form of integration tests because they're actually re using real parts of that external system up to the point down below where we removed part of that or muted some function.

[42:09] (external link, opens in new tab) Tools that you can use to create these parameters. PowerMock in Java will allow byte code manipulation. Anything that allows byte code manipulation or IL manipulation will work. In .NET there is a thing called Moles (external link, opens in new tab) which is free and relatively new and it might be a good idea to try to do that. So Typemock is not the only way to do it in .NET as well. And that's basically what I have. Mostly I'm raising this issue because I want to hear about your experiences if anyone here has ever done something close to this or they try to intercept something in a real system.

[42:48] (external link, opens in new tab) There is I know of maybe just one other example and again it's someone who uses the CThru engine as well. So, I'm not sure if this is a real pattern or not, but, someone told me that it only takes three times to look at that in the wild and say it's a pattern. So, I'm just I just need one more. Just one more. Are there any questions? Anything?

[43:18] (external link, opens in new tab) >> Yeah. >> So, you got to framework. What about do they help are they part of this kind of >> so the question is do aspectoriented frameworks in .NET help this well aspect or aspectoriented frameworks in .NET most of them don't support manipulation of byte code on external systems just in code that you own for example I also use PostSharp inside our products but the problem with post is that it only works when you compile the code that you own And here I'm actually intercepting code that I don't own in Silverlight which is already compiled.

[44:01] (external link, opens in new tab) So PostSharp will not work there. It only works for runtime AOP not compile time AOP. >> Yeah. Any other questions? Yeah. >> I've actually been trying to test SharePoint workflows with a lack of success using Typemock and I started to look down the CThru routes but sort of peted out on it. And the point test because it's a workflow. I can set up all of the properties for it, but then I've got to force it to iterate to move through the workflow.

[44:33] (external link, opens in new tab) >> Okay. >> An awful lot of work to simulate the effect of the workflow engine. >> so the question was that you were trying to do kind of the same thing with CThru in SharePoint workflows. But at some point you stopped because it was a bit too complicated. >> you ran out of time. That's that's definitely possible. It did it wasn't a trivial attempt to do SilverUnit. It actually was an interesting exercise to see what h what would happen. So I think first of all that doing the same thing for SharePoint workflows is totally possible where you mimic not the whole workflow just the basic stuff that you need to test.

[45:12] (external link, opens in new tab) For example, if you need to test a specific thing that runs inside a workflow, you just want to test that the logic of where it should sit is true. I'm not sure. I haven't tested any kind of workflows and I'm not an expert in SharePoint workflows. >> What I've tried to do in the past is if I've writt a custom activity, I'll test that independently by just injecting values. But if the idea was to try and test a workflow that the business logic was flowing so integration aimed towards fitness or something in the end. >> Okay.

[45:42] (external link, opens in new tab) So you wanted to run your own workflow engine which was fully estable. I think that's possible. >> You think that is possible? Yeah, I think that's fully possible given the fact that the workflow engine will take configuration of what to run from an external file. You don't need to intercept the whole engine. You just need to incept the engine in the correct points to inject the values of what to run. The the problem is finding those points. That means a lot of mcking around with Reflector maybe or just doing a lot of wishful invocation to see what would happen. So I think it's it's it's it's a powerful technique.

[46:15] (external link, opens in new tab) It takes dedication to do. Once you do it you've actually created another domain specific test framework for for SharePoint workflows which you can reuse and you can be so if you wanted to be an open- source wizard or something and you know be cool to your friends you say I just created my own domain specific test framework okay buy me a beer or something there should be like a shirt or that so I think it's possible not trivial yeah it wasn't trivial to create SilverUnit it's just trivial ble to show it in a demo which is always true.

[46:50] (external link, opens in new tab) Any other questions? Yeah. >> This reduces your confidence in tests because you're playing about with internals. You're not always 100% sure that the system is going to behave the same way that it does in the real system. >> So do I find that I'm less confident in the tests because I'm actually faking parts of the system. So in real life it would behave differently. Well, I don't have any problem with that because for unit tests I'm only testing the logic. So I can do it in TDD style. It doesn't mean that I won't do integration testing actually run it as part of the browser.

[47:22] (external link, opens in new tab) Okay, it doesn't mean that it's in instead it's just as an added way. If I wanted to do TDD on a Silverlight control now I can do that. But I will always check it on the real system. No such thing as just doing unit tests on a system and that is what gives me the confidence in the integration part of the system. Unit versus integration will always come together. I don't think that you can always any any time replace one with another unless you're willing to give up either confidence if you're willing to give up integration testing or you're willing to give up early early implementation failures in TDD when you give up unit testing

[48:02] (external link, opens in new tab) Okay yeah any other questions anything at all yeah >> so from what I understand this is kind of a work to make test >> this is a workaround to make Silverlight testable. This is the only workaround that I could find because you inherit from the system and that's one of the only cases where I think and Steve correct me if I'm wrong. There is no way around that even if you create a subsystem if the object that you want to test inherits from the system you cannot wrap around that because it's part of it.

[48:36] (external link, opens in new tab) So I think that was real problem I couldn't work around Silverlight runtime because the object doesn't call methods on the system it inherits from the system. Is there anything if you kind of I know it's a bit of another question but if you design now do you think they could have made it more testable opened it up more >> if they were designing Silverlight now could they have made it better in terms of testability? I think that well that the only way to do that was to remove inheritance so that whatever you try to create only implements interfaces in instead of real inheritance and that would have worked so it's possible and but the the thing is that every framework like Silverlight or visual framework eventually you will want to have a base class in the framework you can have 10 interfaces but at some point one of the de developers will say you know But we have all these interfaces and everyone always wants to implement that interface with the base class.

[49:34] (external link, opens in new tab) Everyone. So let's create a base class for those people. Let's not be jerks, right? Because they want it to be more usable. So the usability aspect is actually hindering the testability aspect. I think now again this is just a conjecture. Because I I didn't design Silverlight. I don't know what other stuff they were thinking about when they were designing it. But definitely there are a lot of problematic testability stuff and they needed to make a lot of that work because of XAML and dependency properties and the way Silverlight and WPF work which is pretty amazing and they made a lot of hacks or workarounds and created a whole new pieces of not notions in programming in the Silverlight object model to make all that stuff happen.

[50:21] (external link, opens in new tab) So, I don't know if they could have done it any better for testability. Maybe just in terms of design, but I think testability would still suffer. Any other last questions? Okay. So, there is time and we're going to do a little song. And for doing the song, I'm going to I'm going to need one volunteer to help me with the slides. >> Yes, sir. What's your name? >> Sorry.

[50:52] (external link, opens in new tab) What? >> More and near. >> More near. >> Yes. >> More near. Okay. Please go up. Can you give give your hand for more? >> you know what to do, right? First of all, don't touch anything. >> Okay. I I want to explain what I'm going to do here. I like to to to write songs about my experiences as a developer. And before we do the song,

[51:22] (external link, opens in new tab) Let's test out your skills. Okay? So the the point is every time I finish to sing the line, I finish the line, you press space. That's it. Okay? Are you sure? Okay. This is a test line. Looks like you're doing just fine. It's time for a song of mine. Okay, that's very good. So, now that we know that, we have a passing test, I want to sing a song based on a true story.

[51:56] (external link, opens in new tab) Now this, was a company that I used to work with a long time ago. And, it was a chaotic, problematic company. And nothing worked. And it got to the point where customers had a real problem with the stuff we were shipping. And this is a song about that company. Okay. Go. Our client likes to change his mind.

[52:50] (external link, opens in new tab) It all comes crashing down in flames. But we are knocking on the client's door. I think we got it. Knock knock knock knock knock knock knock knock knock knock knocking on the client's door cuz it worked in my machine knocking on the client's door

[53:43] (external link, opens in new tab) And our culture looks like one big mess. But never mind, we're still knocking on the client's door. Yeah, I think this time we got it. Knock knock knocking on the client's door cuz it worked in my machine. Knock knock knocking on the client's door. But he doesn't trust us anymore. Thank you very much, guys.