Lesson 6 of 17
Version by Snapshot
In this lesson
Roy Osherove shows a problem with artifact dependencies between components built by different teams: artifacts come from the last successful build, so a component can be built against an old version of another whose build has been failing for weeks. A snapshot dependency makes all components build from the same source control version, so if a dependency, or a dependency of a dependency, fails, the dependent build does not run, while components that already passed on the latest version are not rebuilt. Snapshot dependencies from production back to staging, test and CI also give a one-click deployment that runs every step in order, a feature he says does not exist in TFS.
Transcript
[00:04] (external link, opens in new tab) Hi everybody, Roy again. In previous videos we talked about relating to builds, we talked about artifact dependencies. Today we're going to talk about snapshot dependencies and and how that's going to help you win life at work. Because one of the biggest problems, especially when working with other people, first of all, other people are a problem. I realize that. but also they keep creating their own versions of their own components. And we're going to talk about how we can solve that problem with snapshot dependencies.
[00:38] (external link, opens in new tab) So, what is this problem? Let's say that you're working on awesome component number two, but your component depends on component one and component four and component three and also component four depends on component three. Awesome. What do you do? Well, in the build, you could of course do what we talked about before and just get the artifacts compile those components and get the artifacts from those components into component two, right? That's what we would do. Get the artifacts from component three into component four so that it can build, then get component four, artifacts into component two so that can build, etc., etc.
[01:18] (external link, opens in new tab) All great really. Some of my best friends do that. However, there could be some problems. Let's imagine what would happen. Where do we get the artifacts from? Artifacts are only gotten when a build is successful. And by default, when we get artifacts, especially in products like TeamCity, we get them from the last successful build. Unfortunately, last successful build could mean that we might have versioning issues between the components.
[01:49] (external link, opens in new tab) Here's a simple example. In this case, component two is already at version 103. We want to compile it. We want to get the artifacts from component one which we depend on. Unfortunately, component one's team has been kind of disrespectful towards code quality or you know what it's just something horrible has happened. Product owner whatever someone spilled some coffee on the source control. I don't know. Anyway, their build has not been working for the last two weeks.
[02:19] (external link, opens in new tab) So their product still their component is still at version 101. So when we get the artifacts we get an older version of component one to build with component two. Now of course that could create a big problem is that we might only recognize this problem during deployment. So what we would like to do instead is to realize that there is a versioning issue or maybe to avoid a versioning issue in the first place.
[02:50] (external link, opens in new tab) Here's another simple example. Component 4. Maybe that one has only been working for the last week. But the newest build 103 has failed. So we're going to get version 102 into our component. And component 4 has its own dependency. So ju just look at this. Component 4 depends on component 3 and component 3 actually has a newer version than 103. It's actually working on new APIs which are on version 104.
[03:21] (external link, opens in new tab) But we're getting 103 which is the the last successful one. So we won't know that we're we're working with new APIs until that build finally succeeds. And then it will be a lot more time until we get feedback. We might be working against an older version not knowing that we should be getting a a newer one. So we might end up with at deployment time with component one version 102 and the other components version 1, two, and three.
[03:56] (external link, opens in new tab) And that's just hell. Just recognizing all that and testing all the components with those specific versions. That's really really complicated. And this is where snapshot dependencies come to our rescue. You see, instead of just having artifact dependencies, which is one kind, we can actually do two kinds of dependencies. We can have artifact dependencies, but also set a snapshot dependency, which is kind of like tagging something and saying, I always want to have the latest version. So if I don't recompile that something and if it fails
[04:29] (external link, opens in new tab) Don't let me run. In this case what this means is that snapshot dependency means that we will always have the exact same version between all the components. It will force the build on components that are in an older version will actually fail our build if they don't succeed. In this case, component one, if we had a snapshot dependency on it, our build would of component two would actually fail because component one fails our build and the CI server (external link, opens in new tab) in this case TeamCity recognizes that a snapshot dependency has failed to build and then component 2 will actually stop from building.
[05:12] (external link, opens in new tab) So we cannot integrate these two components until their versions are exactly the same. And when I say versions, I mean that the source control tag, the source control version, the specific check-in version is the same. If the source is different, then those builds will fail and component two can never succeed until component one succeeds. Same with component four. If that fails, component two fails. What if component three fails? It actually will affect component two as well.
[05:45] (external link, opens in new tab) You see in this case component three failed to build 104 which consequently failed component 4. Even if component 4 to the right here would have succeeded its snapshot dependency failed has failed and then its own build will never actually run. meaning that component two, the the full component on the left will never actually compile and will never actually build because component three, they're both waiting for component three in this case.
[06:17] (external link, opens in new tab) So even if you have a small subd dependency of a dependency, it can break the it can break the build. So when you have a component that depends on other components, snapshot dependencies are a good way to make sure that every one of those dependencies is in check. When they're all in check, we're going to get a successful version. They don't all have to be they all just have to be in the latest version of the source code. They do not have to be the exact same version of the binary.
[06:51] (external link, opens in new tab) We could have more check-ins in some component than others. But as long as the latest specific version of that component is what we have, then the build should s and the build succeeds then the component 2 build will actually succeed. And of course if if the build has succeeded, there's no reason to rerun them. So when if component 4 has succeeded and there were no check-ins when we move to a new version, component 4 will not be built. We'll just reuse the existing artifacts and that saves a lot of time.
[07:25] (external link, opens in new tab) Working snapshots are skipped. If the if the code is in the latest version and the the previous build has succeeded, then there will be no extra build. So we don't really have to build all the components, just the ones that have changed. So that's about the team. We can also use snapshot dependencies based on the structure that I showed you in previous videos. In this case, if you remember, we had CI and deploy to test, deploy to staging. I can use a snapshot dependency.
[07:56] (external link, opens in new tab) I can set a snapshot dependency from prod to staging and between staging to test and between test to CI. So that when I run deploy to prod, it will actually trigger CI and then test and then staging in the right order. And if CI is already run, it will deploy to test and staging before that. So I can have a one-click full deployment operation that just based on the snapshot dependencies will trigger everything in the right order. And because I get the artifacts, because I have both dependencies, then not only do I make sure they're the same version, I also get the artifacts and I reuse them so that I have a cumulative build (external link, opens in new tab).
[08:37] (external link, opens in new tab) So that's it about snapshot dependencies. This is one of the coolest things that almost nobody knows about except those with that have all that have had those problems for a long time and finally found out about this. So this is something for example doesn't exist in TFS. TFS I think is like 3 four years ahead of back where it should be. Team CD does support this. I'm not sure what other tools support this but I would love to hear your comments about this.
[09:10] (external link, opens in new tab) So till next time, cheers.