# FTPMirror error message
**URL:** https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533
**Category:** Discussion
**Created:** [August 15, 2013, 6:13am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533 "2013-08-15T06:13:55Z")
**Posts on this page:** 18
**Page:** 2
### Author:  [@mrrimmer](https://www.finalbuilder.com/forums/u/mrrimmer)
#### Post date: [September 23, 2013, 3:02am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/21 "2013-09-23T03:02:11Z")
FYI, this is a snippet of the log that I am trying to translate, running in Automise 4.0.0.467:
Downloaded /COWATCH02646/A10EXT.ZIP to \\dutch\\Downloads\\Jordans\COWATCH02646\A10EXT.ZIP in 6139 ms
Downloaded /COWATCH02646/A11EXT.ZIP to \\dutch\\Downloads\\Jordans\COWATCH02646\A11EXT.ZIP in 76 ms
Downloaded /COWATCH02646/A12EXT.ZIP to \\dutch\\Downloads\\Jordans\COWATCH02646\A12EXT.ZIP in 60 ms
The idea is to extract the destination full filename
\\dutch\\Downloads\\Jordans\COWATCH02646\A10EXT.ZIP
\\dutch\\Downloads\\Jordans\COWATCH02646\A11EXT.ZIP
\\dutch\\Downloads\\Jordans\COWATCH02646\A12EXT.ZIP
---
### Author:  [@nevar](https://www.finalbuilder.com/forums/u/nevar)
#### Post date: [September 24, 2013, 1:16am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/22 "2013-09-24T01:16:20Z")
Hi Mr Rimmer,
Here are the builds.
[http://downloads.automise.com/downloads/automise/400/ATCMD400\_476.exe](http://downloads.automise.com/downloads/automise/400/ATCMD400_476.exe)
[http://downloads.automise.com/downloads/automise/400/AT400\_476.exe](http://downloads.automise.com/downloads/automise/400/AT400_476.exe)
[http://downloads.automise.com/downloads/automise/400/AutomiseRunner400\_476.exe](http://downloads.automise.com/downloads/automise/400/AutomiseRunner400_476.exe)
---
### Author:  [@mrrimmer](https://www.finalbuilder.com/forums/u/mrrimmer)
#### Post date: [September 26, 2013, 2:32am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/23 "2013-09-26T02:32:07Z")
Still not having any luck with the regex filtering. I found a tool to test Delphi regex’s but I can’t make much sense of it (usually use Expresso for .NET regexe’s which works really well).
Can you try the Regex and see what you get?
Thanks.
---
### Author:  [@nevar](https://www.finalbuilder.com/forums/u/nevar)
#### Post date: [September 27, 2013, 3:15am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/24 "2013-09-27T03:15:55Z")
Hi Mr Rimmer,
I got “^Downloaded [._] to [._] in \d+ ms” to work. In my quick attempt I forgot that you need to escape square braces.
---
### Author:  [@mrrimmer](https://www.finalbuilder.com/forums/u/mrrimmer)
#### Post date: [September 27, 2013, 11:28am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/25 "2013-09-27T11:28:41Z")
Sorry about this Jason, my fault - I had an output monitor filter on the FTPMatch action which looked for
Downloaded '\*
It should have been changed to
Downloaded [\*
---
### Author:  [@mrrimmer](https://www.finalbuilder.com/forums/u/mrrimmer)
#### Post date: [February 19, 2014, 5:20am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/26 "2014-02-19T05:20:11Z")
Things are a lot more stable now with this action, however the evaluation of differences is still very slow when interrogating a large number of files. Admittedly this is an extreme case as there are 732,868 files in 1,184 folders to process locally! I don’t know if it is possible, but making the sorting/evaluation code multi-threaded could give a big boost as the server this is running on has 12 cores.
Keep Alive Timeout: Sending NOOP
Sorted local items in 120298 ms
Keep Alive Timeout: Sending NOOP
Keep Alive Timeout: Sending NOOP
Keep Alive Timeout: Sending NOOP
Keep Alive Timeout: Sending NOOP
Sorted remote items in 8205 ms
Evaluating Differences…
Keep Alive Timeout: Sending NOOP
Keep Alive Timeout: Sending NOOP
9430 Differences Found
Evaluated Differences in 5021087 ms
Sorted Differences in 512 ms
Downloading files…
---
### Author:  [@mrrimmer](https://www.finalbuilder.com/forums/u/mrrimmer)
#### Post date: [February 19, 2014, 5:25am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/27 "2014-02-19T05:25:17Z")
Further to my comment below, one glitch I have noticed is that the FTP connection sometimes drops, causing the mirror action to retry. This leads to another 10 minutes of delay whilst it goes through all comparison steps again.
Would it be possible to have an inner retry loop, so that it would get straight back to resuming whichever step had failed rather than repeating the whole process?
---
### Author:  [@nevar](https://www.finalbuilder.com/forums/u/nevar)
#### Post date: [February 20, 2014, 12:14pm UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/28 "2014-02-20T12:14:04Z")
Hi Mr Rimmer,
That certainly is a great number of files being reviewed, and then mirrored. I have logged both issues you raise here, the speed of the difference check, and an internal retry. I am certain we can get the difference check down to something a lot quicker, the internal retry will require a bit more work as its a structural change to how actions run (keeping state between retries).
I will keep you informed as to how things progress.
---
### Author:  [@mrrimmer](https://www.finalbuilder.com/forums/u/mrrimmer)
#### Post date: [February 20, 2014, 1:06am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/29 "2014-02-20T01:06:53Z")
Thanks for such a prompt reply!
Another thought - the NOOPs seem to happen quite often (10s?). Perhaps the interval could be adjustable, and/or an option to not log the NOOP action. That will make log files a lot smaller.
---
### Author:  [@nevar](https://www.finalbuilder.com/forums/u/nevar)
#### Post date: [February 20, 2014, 10:57am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/30 "2014-02-20T10:57:06Z")
Hi Mr Rimmer,
I have added the option for turning off the NOOPs and changing their interval as another case. It should be very simple to achieve a solution for this. In other news I have found a speed improvement for the differences test. It should show a substantial improvement in speed. I will also look at the sorting methods, I believe these could be quicker.
After these have been done I will make a release. Multi-threading the action will take some more work which will mean we head on into next week before its done.
---
### Author:  [@nevar](https://www.finalbuilder.com/forums/u/nevar)
#### Post date: [February 21, 2014, 7:30am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/31 "2014-02-21T07:30:48Z")
Hi Mr Rimmer,
I am glad to report that I have the FTP Mirror runtime down by a large factor. The releases are located here;
ATCMD: [http://downloads.automise.com/downl…00\_585.exe](http://downloads.automise.com/downloads/automise/400/ATCMD400_585.exe)
Automise: [http://downloads.automise.com/downl…00\_585.exe](http://downloads.automise.com/downloads/automise/400/AT400_585.exe)
Automise Runner: [http://downloads.automise.com/downl…00\_585.exe](http://downloads.automise.com/downloads/automise/400/AutomiseRunner400_585.exe)
Please let me know the timings your script gets with these builds. Currently in testing the timings for 1 million files are looking as such;
Sorting Local Files…
Sorted local items in 46289 ms
Sorting Remote Files…
Sorted remote items in 9850 ms
Evaluating Differences…
773216 Differences Found
Evaluated Differences in 3494 ms
Sorting Differences…
Sorted Differences in 35817 ms
My differences are larger and this does affect the numbers a little, but this is significantly down from the 80+ minutes you were seeing. Please let me know how this goes.
---
### Author:  [@mrrimmer](https://www.finalbuilder.com/forums/u/mrrimmer)
#### Post date: [February 24, 2014, 1:59am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/32 "2014-02-24T01:59:52Z")
Hi Jason,
Looks like a massive improvement in performance. Thank you!
Sorting Local Files…
Sorted local items in 63317 ms
Sorting Remote Files…
Sorted remote items in 1110 ms
Evaluating Differences…
3051 Differences Found
Evaluated Differences in 2306 ms
Sorting Differences…
Sorted Differences in 69 ms
Downloading files…
---
### Author:  [@mrrimmer](https://www.finalbuilder.com/forums/u/mrrimmer)
#### Post date: [March 27, 2014, 1:44am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/33 "2014-03-27T01:44:45Z")
Been using the fixed FTPMirror for a while but I’m now seeing the below error message. Maybe there is a memory leak, or the number of files being requested is too large? I’m thinking the former as stopping the IDE causes it to crash (memory usage is 1.3 **Gb** )
```
CRC is not supported by this FTP Server. Skipped Calculation of Server Time Skew
Generating File List…
FTP Mirror failed. Out of memory
```
---
### Author:  [@nevar](https://www.finalbuilder.com/forums/u/nevar)
#### Post date: [March 28, 2014, 8:23am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/34 "2014-03-28T08:23:27Z")
Hi Mr Rimmer,
We have found the issue which is causing the out of memory error. I have a solution which will keep the speed and reduce the memory overhead we currently have when listing the files. It will take a little time to implement.
In the meantime I suggest splitting your FTP mirror at a certain folder level. This isn’t ideal I know, however it should mean you won’t hit the memory issue. As a ballpark to aim for in this interim solution would be to split the mirror out into approximately 300-400k file blocks to be safe. If you can manage less that would be even better.
I will post a fix as soon as I have one. Working through implementing and testing now.
---
### Author:  [@nevar](https://www.finalbuilder.com/forums/u/nevar)
#### Post date: [April 7, 2014, 10:38am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/35 "2014-04-07T10:38:16Z")
Hi Mr Rimmer,
I am glad to report that I have the FTP Mirror runtime down by some more and the memory usage is significantly down. The releases are located here;
ATCMD: [http://downloads.automise.com/downl…00\_615.exe](http://downloads.automise.com/downloads/automise/400/ATCMD400_615.exe)
Automise: [http://downloads.automise.com/downl…00\_615.exe](http://downloads.automise.com/downloads/automise/400/AT400_615.exe)
Automise Runner: [http://downloads.automise.com/downl…00\_615.exe](http://downloads.automise.com/downloads/automise/400/AutomiseRunner400_615.exe)
Please let me know how these go for you. I have tested them quite extensively here so not expecting anything untoward to occur.NOOPs do send too often at present, I am working on a way to reduce them. It doesn’t impact performance that substantially.
---
### Author:  [@mrrimmer](https://www.finalbuilder.com/forums/u/mrrimmer)
#### Post date: [April 8, 2014, 10:50am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/36 "2014-04-08T10:50:41Z")
I’ve installed 615 but the FTPMirror actions seems to be broken. I managed to get the IDE to send an error report before it crashed. Have reverted to 585 for the moment until this is resolved.
```
The error message is:
Local Directory [%LocalPath%] does not exist.
```
---
### Author:  [@nevar](https://www.finalbuilder.com/forums/u/nevar)
#### Post date: [April 10, 2014, 3:24am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/37 "2014-04-10T03:24:46Z")
Hi Mr Rimmer,
I have found and fixed the errors which were occurring. These were namely a stop build error and mirror download error with dates checked. The releases are located here;
ATCMD: [http://downloads.automise.com/downl…00\_628.exe](http://downloads.automise.com/downloads/automise/400/ATCMD400_628.exe)
Automise: [http://downloads.automise.com/downl…00\_628.exe](http://downloads.automise.com/downloads/automise/400/AT400_628.exe)
Automise Runner: [http://downloads.automise.com/downl…00\_628.exe](http://downloads.automise.com/downloads/automise/400/AutomiseRunner400_628.exe)
Please let me know how these go for you.
---
### Author:  [@mrrimmer](https://www.finalbuilder.com/forums/u/mrrimmer)
#### Post date: [April 24, 2014, 4:23am UTC](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533/38 "2014-04-24T04:23:41Z")
Hi Jason,
Wanted to report that build 634 seems to be working well. Thank you for your efforts in fixing this issue.
[Previous page](https://www.finalbuilder.com/forums/t/ftpmirror-error-message/4533.md?page=1)