In the past I used a vbscript to compare two folder locations for added/Deleted and modified files. We do use Merge here and I switched to using a HTML output and triggeing of the HTML file. I have tried to use the XML version and I see that it is taking a long time to:
(varies on size of folder compare)
1. Perform the Merge compare.
2. The iterate through the project and find files delete or added.
In short almost 30 exta minutes compared to 5 minutes with the HTML version and less with vbscript.
What is the best way to compare two folder locations and their subfolders!
Thanks for posting, but I’m a little confused. I think your post might be missing some context.
Unless I’m mistaken, the crux of your problem is that Araxis Merge is slow when comparing XML files. This is probably because it builds a fairly in-depth tree showing the XML node changes, which is probably fairly memory and CPU intensive. Useful when diffing XML, but possibly not so useful for your purposes.
To be honest, I’m not exactly sure what you need to do. Can you explain the problem again, please?
No this is on the results from araxis. merge. What I am trying to do is get a listing from the two directories files added, removed and their locations. Using finalbuilder with the XML result of takes about 20 minutes to parse through. keep in mind that it might be the way we are doing it but this is using node iterate.
Oh OK, I see. Using the node iterator with a giant XML file is probably pretty slow. The node iterator uses XML DOM, and it’s fairly notorious for using large amounts of memory and processing overhead when dealing with giant files.
Are you using more than one XML action? (ie are there XML Actions which are children of the XML Node Iterator?) If you have multiple XML Actions, and aren’t already doing this, then use the XML Document Define action to load the XML Document once, and then pass that XML Document name to all of the actions which use the document. This will make things much, much faster.
If you could send or post a sample then this would be very helpful.
I will send you a project in with both items that I am using. I am trying to simplify it down for you
In this project, I will have one method grayed out and the other method activated.
One will be with the Node iterate we are doing with the XML.
The other will be with finalbuilder fileset.
Along with that I will send a snippet where I have vbscript to do the same thing.
My goal is as follows.
1. Give the differences between two directories and sub-directories.
a. If Differences log the difference and their location. - If added state the add and deletes(this is for the MSI build tool to signal a full MSI build due to new files or deleted files.
b. If files have changed to compare version between the two and log if an older version is found and if it is data files to make sure that the date time are equal to or higher then the original source!
This is for Patching issues.
Once I finish up getting our install packages I hope to have the finalbuilder projects in place and then take my old vbscript and pull out the pieces for you to see.
What would be nice is a Folder Set where we can get a list of Folders and iterate through them to perform a compare at a folder level. looking at file counts and subfolder counts!
Here is a project that has both the Araxis and FInalbuilder ways for compare. I have to pull my old script together and modify for it to run via commandline.
Running XML is about 30 minutes the Fileset compare I stopped after 45 minutes! It was 1/2 done.
Thanks for this. I've reattached your test project with the XML actions optimised using the method that I described above. This should be substantially faster, because it only loads and parses the XML file one time as opposed to 1 + (N * 3) times.
If you could post up a .fbl5 log file with a history that show both methods running separately in this project, then I'd be keen to take a look at it and see what other optimizations we may be able to make.
The problem of "comparing two folders" comes up frequently, so I'll put it on the to-do list for further investigation. We may be able to come up a single "Directory Compare" action for a future release.
Angus, Thanks I update and try your project when I get in the office. The LOG file is 31MB and when trying to compress, it will not allow me to upload it to the Forum!
Thanks that trimmed down the XML iterate by 5 minutes, but what I hope to do is get my vbscript to work.
Again what I am looking for is this
Folder iterate and if tehre is folder count or file count greater then zero to perform some action. If file count compare to second locations. if Folder count, go into that subfolder and perform the action(s) again.
Rignt now I am concerned on if files are added or deleted. But I am also going to be concerned on if the files changed that if it is not a versioned file to compare the file date and time to the orginal location to make sure that it is newer. And flag it if it is a DST time change! These can cause issues with files not seeming to the same in a patch. In Perforce, which we use, the content did not change!
Also I will look to make sure older files did not go down over newer fileset.
If Versioned file then I want to check the file version in the compare to make sure someone did not accidently integrate in an older file during our maintenance releases.
On top of all this I am looking to make sure I capture empty folders that were either added or deleted between the folder compares. This is to ensure that for the release that the patch will be successful and work correctly maintaining what is currently on the clients machine!
Very involved and I know my vbscript compares performed this before. Since moving all my projects over to Finalbuilder, I am looking to add this in.
Also side note that if something did change I want to Log that change to the correct location like a full path to the file being added, deleted or modfied incorrectly. Same with Folders.
Sorry, I am on Vacation this week, I tried to get the script out and working again. But I think I asked if I should email or FTP the stuff to you. As well as the logs!