Lesson 5 of 18
Iterating over Files
In this lesson
Roy Osherove gathers the content of several input files into one file with a loop: he defines a file set of the files that start with important and end with .txt, and a file set iterator puts each file name in a variable. The actions he moves under the iterator read each file and append its content to the new file, and he notes that a naming convention for the variables would help. After he replaces the placeholder with an empty string, the result still starts with an empty line.
Transcript
[00:02] (external link, opens in new tab) Okay, let's take now let's make our example a bit more complicated. Let's say that we need to go through all of these files, all of the important files and each one of them has a slightly bit different content. Just put numbers at the end of each one of those. Those numbers don't necessarily correspond to the name of the file. Just so you know, we don't want to we do want to see different types of content. I want to put all of them inside the new file. So I want to have multiple lines here gathered from all the stuff in all the files here and for that we're going to introduce the concept of loops
[00:39] (external link, opens in new tab) And loops appear inside if I'm going to say loop we have a for loop we have loop and we have file iterators which is what we want. So the first thing we want to do is we want to say we're gonna if we want to iterate over all the files that in a specific directory. Let's create a file set. A file set is basically a list of the files and then you can iterate over them.
[01:11] (external link, opens in new tab) So we can we can for example say file set define. So let's call that file set all input files. Probably you need to give it like a variable name all input files I don't think it can have spaces in it and the base directory of course is can you guess we already have a variable for it so full path generation directory
[01:41] (external link, opens in new tab) Now what are the include patterns so we can say everything that starts with important and ends with txt If we go through it and we'll see. We should be able to see that. Let me hold on. Let's see. Right click on this one. FB demo in new files. It starts with important and then it ends with .txt.
[02:11] (external link, opens in new tab) Now, if we can't see the the extension of the file here, we can go to organize folder and search options view and say yes. Yes, I want to display protected files. I want to I don't want to hide protected files. I don't want to hide extensions. I don't want to hide empty drives. Show me everything.
[02:41] (external link, opens in new tab) Damn it. Basically what I'm trying to say. Show hidden files, everything. So now I should see the extensions like I should have the whole time. So if I want to iterate over all these files, I want to define the files I want to search and that's me here. Okay. So do I want it recursively? I could say recursively. Doesn't really matter right now. So I can add multiple patterns here but for now this should be okay.
[03:14] (external link, opens in new tab) So I define the file set and then I want to say file set iterator. Now the file set iterator basically says hey which file set do you want? I can define the file set right here and just use the file set here. So if I keep reusing those file sets, I can just define them and then reuse them. Or I could do the whole thing in one step. But I wanted to show you the two-step solution for now. Refresh the file set contents before the first iteration.
[03:45] (external link, opens in new tab) Perfect. And next, I want to have an iterator variable to set. Now again, this variable is something that I'm going to set right here. I can't I have to set it from the outside, unfortunately. So I'm going to create a new variable here. I'm going to say temp input file name. I don't have to do anything to it. But I want to use that to iterate.
[04:16] (external link, opens in new tab) So go through each of the input files and put that stuff in the temp input file name. So far so good. Now, what do I have to do for each of these files? I want to to append the content. I want to read the file. So, I'm going to say let's read the file. Let's read the text file.
[04:48] (external link, opens in new tab) Now, what's the file name? Easy. Temp input file name. And I want to put the contents of that file name into a variable now. So that's one way to do it. I can say that and you see now we're starting to say maybe we should have a naming convention for all the variables that we have here because then it's otherwise it's a bit harder to find, right?
[05:19] (external link, opens in new tab) But we can say for now what was the name of my variable here? Let's choose no variable and look we used important contents. So let's use that here. Read a text file of important contents and put that in the contents that we want. And then we want to say hey let's look for an append. There's a write text to a file that also just appends it.
[05:49] (external link, opens in new tab) So the file location we know that stuff. That's the full path of generation deer slash newfile.txt append to the end of the file and the new content would just be in this case important content. Now if we want these to actually happen we have to put them under the iterator. So I'm going to basically push them to the right.
[06:23] (external link, opens in new tab) And there the way we're going to do that is using this arrow. So now the iterator for each iteration it will run all the sub actions under it. So if we just run this here now what should happen? Let's take a look. We can see it running multiple times. That seems to have gone interestingly well. Let's take a look at that. So, let's look at new file now.
[06:56] (external link, opens in new tab) Awesome. Blossom. Looks like it worked. So, you can see it's quite easy to start looking at these things and have iterators etc. Now, since we have that, do we really need to do the initial thing like reading this text file? And the answer is no, we don't. We can just actually get rid of the first action here. So I'm going to just press the delete button here. Yes. And the delete key. Sorry. And I don't have to do a text.
[07:27] (external link, opens in new tab) Well, I could do a text replace. And then I can for example say do we want to replace the text or we want to just you know what? Let's just replace the text with a piece of empty string. So I'm just going to say that here because that way we're going to append all those lines. So we don't need to do that anymore because if you'll notice because we did that twice in new file we have that Frank stuff written here at the end and also at the beginning.
[08:03] (external link, opens in new tab) So by just replacing it with an empty string we should be able to just run it again. one two three and you can see it says how many times this was completed which is again very very nice. If I can now go into new filet txt, I see that the first line is empty.