Tipping Point

[00:03] (external link, opens in new tab) Hi everyone, Roy Osherove again and this time we're going to talk about continuous delivery (external link, opens in new tab) with multiple teams or components working in parallel and what do you do when you have an application that you want to deliver that is made up of all of these components. Now in the previous videos I talked about versioning problems between components and their dependencies. For example, a component might depend on other components and then we would have snapshot dependencies between components to check to check their versions to make sure that the versions are all the same.

[00:39] (external link, opens in new tab) We're going to use kind of the same thing, but this time we're going to take a bigger view. For example, in this case, let's imagine that because we have all these components and each component is actually developed by a separate team. In this case, what we can do and we can also do this if if it's all done by a single team, but if all these components are developed separately in parallel and they have their own schedules of when they get released and when the and the team fixes bugs on a different in different times or they work in different time zones, whatever.

[01:17] (external link, opens in new tab) When we work in these kind of separate areas, it's very important that each component actually has its own little pipeline. So instead of just having a single CI unit test and integration steps in our build, we can actually have a small parallel universe for each component or team that is working and for each component there is actually a set of steps that it's going through. So each component will have its own CI bill and its own unit testing and its own integration test

[01:50] (external link, opens in new tab) So that each component can be developed and maybe actually released as a separate small product and then later we might have a bigger build that does integration etc with all these components. But then there is a different problem when we have all of these components. The question is where do you get the artifacts? If I wanted to get the artifacts from the components and into my build process where I want to deploy the application, where do I get the artifacts from?

[02:24] (external link, opens in new tab) Should I get them? I can get them directly from the CI. I can get them after they have passed integration testing. And in here based on this diagram it kind of makes sense that we actually get artifacts and publish artifacts from the integration test configuration for each component. But that doesn't necessarily ring true because from my experience different components can have actually different stages and different processes.

[02:55] (external link, opens in new tab) For example, some components might have integration and they might even even have acceptance tests or they might have various other stages like manual not manual but automated installer tests or various configuration management generation, code generation, things like that. Publication to various locations. So actually the the pipelines might have for some components more than three steps and for some of them less than three steps.

[03:28] (external link, opens in new tab) Some components might not have unit tests. Some of them might not have integration tests. So then when we want to have when we want to know what what to publish from each component there is no one single way of knowing which which location in the flow should we get the binaries from. It kind of makes sense that it's the last one but sometimes it's not the last one. So it's important to create something that

[03:58] (external link, opens in new tab) A convention where we always know where to get the latest version of the binaries after having been tested and gone through the entire important flow so we can deploy them as part of the application. And I call this pattern the tipping point because we're going to actually create an a tip for each component that contains the binary. So instead of a component having just CI unit test integration now each component is going to have a convention it's going to have one more

[04:30] (external link, opens in new tab) Configuration and we call this tip or this just one name you can name it whatever you want. But the idea is that all components have a tip and then we know that tip is the thing that contains the binaries after they've gone through the entire process and we know that they're deployable. they're good to go. So a tip h in terms of dependencies you would have probably would take the artifacts from directly from the CI after compilation.

[05:04] (external link, opens in new tab) But a tip configuration would also make sure that the the binaries that it publishes are actually the ones that have gone through the entire process. So we might have multiple snapshot dependencies inside a component. A tip will have a snapshot dependency on integration test. So that if I trigger the tip build, it will actually trigger integration test first if the code has changed since the last build. And then there's a snapshot between integration and unit test.

[05:35] (external link, opens in new tab) So that integration test will actually not run before a unit test has run if the code has changed. And the same for unit test versus CI. That means that if I click and I try to run the tip configuration it might trigger CI unit test and integration tests before tip can actually publish anything. And that's exactly what we want. We want tip to actually represent the best version the best deployable version of those binaries. So now that we have that tip location,

[06:08] (external link, opens in new tab) Imagine that we have these four components and we and each one of them is developed in parallel and they have their own little pipelines. Now we can see flow and now let's imagine that we have this deployment chain configurations right we have system integration deploying all the components to test and staging and production. Now we can actually have the that convention work for us. So we can have those snapshot dependencies taken directly from tip. So I have a a single convention where if I want to do a system integration test with all the components in their best versions from the latest versions, I just take the things from tip and I actually have a snapshot dependency.

[06:53] (external link, opens in new tab) Now snapshot dependency is is kind of a choice here. I have a snapshot dependency because I want to make sure that when I test system integration I actually am testing it with the latest deployable version of all components and that means that if one of the components actually fails like component one I won't actually be able to run the system integration now that is either good or bad for you when I was working on one of those systems we had we had taken artifacts but we didn't have snapshot dependency between components.

[07:29] (external link, opens in new tab) And so what we ended up is with is that we had components in different versions deployed into system integration and then things seems to be okay at the component level but then when they came into the actual integration system integration things just started to fail because of versioning issues where things not were not in the latest version sometimes for and we I explained this in the previous videos. So snapshot dependencies from the component tips is actually a very good way to make sure that we have the latest version and because each one of them does not have a dependency on the others in this case.

[08:07] (external link, opens in new tab) This can actually run four par parallel build agents that will trigger and run in parallel if we wanted to so that we don't have to wait linearly for each component to build. So only we will only have to wait for the slowest component but not for all of them one after the other. And this way when we get to system integration we have the full versioning support and we get the latest deployable binaries so that when we finally get to 40 in the deploy chain

[08:44] (external link, opens in new tab) We know that what we have is something that is the latest version of all the components. we can't actually get there before all components have a verifiable runnable tip because we have snapshot dependencies. If we wanted to, we could just pin to a specific version of a component to make sure that we always deploy to system integration with version 101 of the tip of that component. But we can also choose to have the latest version via snapshot dependency on the absolute latest version.

[09:16] (external link, opens in new tab) But at least we have a choice now. And we can also again see the flow and make sure that we have fully integrated systems. The system has integrity. And with that, I would like to thank you and I'll see you in the next