SITE UTIME or MDTM or MFMT to set remote file time metadata

markokobal

2007-12-04 18:32:53

Hi.

As we all know there is an big mess with setting remote file time metadata at upload. SITE UTIME or MDTM or MFMT are implemented in various FTP servers - and usualy only one method is implemented in one FTP server. On the other way, most FTP clients can not deal with setting remote file time metadata at upload, and if they do, they use just one of the methods. SmartFTP uses the most non-standard method, that is MDTM wich is not even documeted in any FTP RCFs (as for the use at upload event, not download!).

So, considering MFMT seems to be "the ONE" method for the future (http://www.indyproject.org/sockets/blog ... fxx-01.txt) and "SITE UTIME" is also widely used by FTP servers I woudl suggest you would implement all those methods in the SmartFTP client. Because detecting wich mode should be used is quite hard (or even impossible) as FTP servers can not tell wich methods they support you could let user to select the appropriate method for each FTP server by the settings --> Transfers -> Files -> Keep file time -> Upload ---> here you would select "Enable" and by another drop-down menu you would select the method by the options: "SITE UTIME" or "MDTM" or "MFMT". Actually, because there are two sub-methods for the "SITE UTIME", see http://www.indyproject.org/sockets/blog ... 15.en.aspx - one uses onyl one time parameter, the other three of them, so we would have four options for the new drop-down "SITE UTIME 1" or "SITE UTIME 3" or "MDTM" or "MFMT".

Is there any chance you guys would go for this upgrade? I don't see much work with this upgrade as there is no special algorihms programming, just adding some drop-down and generating appropirate FTP commands ... If you could do this than SmartFTP would most definitly be the best FTP client to handle remote file time metadata at upload event!

Anyways ... SmartFTP is an excellent product so keep up the good work!

Kind regards, Marko.

2007-12-04 19:33:08

Hello ..

MFMT is already supported. I have to look at the SITE UTIME command. Does the server announce it in the FEAT reply?
There was a similar discussion about the SITE UTIME at:

markokobal

2007-12-05 07:30:02

Hi,

MFMT is already supported.

And how do you detect wather to use MFMT or MDTM (as both are supported in SmartFTP)?

I have to look at the SITE UTIME command. Does the server announce it in the FEAT reply?

Unfortunately no:

ftp> quote FEAT
211-Extensions supported:
EPRT
IDLE
MDTM
SIZE
REST STREAM
MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
MLSD
ESTP
PASV
EPSV
SPSV
ESTA
AUTH TLS
PBSZ
PROT
211 End.

But you can detect if command SITE UTIME is available with:

ftp> quote SITE UTIME
501 No file name

as you do not get the 500, like you would get if command would not be supported:

ftp> quote NOTEXISTENTCOMMAND
500 Unknown command

Anyways ... I guess the option to let user select wich method will be used via settings seems to me the most appropriate...

Kind regards, Marko.

2007-12-05 12:35:10

>And how do you detect wather to use MFMT or MDTM (as both are supported in SmartFTP)?
Servers supporting MFMT (should) announce it in the FEAT reply.

>UTIME
We have no plans to support this command.

Regards,
Mat

2007-12-08 11:04:47

>SITE UTIME
I have changed my mind and implemented the NcFTPd/PureFTPd variant in the latest version. However the UTIME command from proftpd:
http://www.proftpd.org/docs/contrib/mod_site_misc.html
is not enabled because it does not support seconds in the time and the time seems to be in local time.

Please give it a try:
https://www.smartftp.com/download
or
https://www.smartftp.com/download

Regards,
Mat

2007-12-09 00:13:21

I just noticed that there is a new update of the MFTM draft:
draft-somers-ftp-mfxx-03.txt

This should be a good argument for FTP server developers (hint: ProFTPD devs) to implement the MFTM command.

Regards,
Mat

markokobal

2007-12-10 19:50:04

Hi,

I have tested the new version with UTIME support and yes yes yes, it works great with pureFTPD, thanks!!!

And I have notified pureFTPD and proFTPD developers about the new MFXX draft.

SmartFTP is now actualy the best FTP client for me! ;-)

2007-12-12 20:30:17

I'm very happy to announce that as of my request TJ Saunders (maintainer of the mod_site_misc module for proftpd) has updated the SITE UTIME implementation for proftpd. It now supports seconds in the time argument and the time is in UTC. You can read more about his patch at:
http://bugs.proftpd.org/show_bug.cgi?id=3012

The latest version of SmartFTP (2.5 Build 1008.25) also supports the new implementation. This version is available from:
https://www.smartftp.com/download

Please let us know if you should run into any problems.

Regards,
Mat
SmartFTP

jeburnett

2008-08-19 00:31:14

I'm very happy to announce that as of my request TJ Saunders (maintainer of the mod_site_misc module for proftpd) has updated the SITE UTIME implementation for proftpd. It now supports seconds in the time argument and the time is in UTC. You can read more about his patch at:
http://bugs.proftpd.org/show_bug.cgi?id=3012

The latest version of SmartFTP (2.5 Build 1008.25) also supports the new implementation. This version is available from:
https://www.smartftp.com/download

Please let us know if you should run into any problems.

Regards,
Mat
SmartFTP

I've just downloaded and installed v3 and am trying out the program. I cannot find any commands or settings to enable preservation of original file time/date stamps. This thread talks about SITE UTIME support but neither the product or the help files describe how to invoke this.

Can you please point me to the "how to do it" document?

Thank you,
Jonathan

2008-08-19 00:57:22

It's done automatically (by default) if your server supports any of the 3 commands.

Many server - mostly the ones running on Unix derivatives - do not support this feature.

Regards,
Mat

jeburnett

2008-08-19 03:39:46

It's done automatically (by default) if your server supports any of the 3 commands.

Many server - mostly the ones running on Unix derivatives - do not support this feature.

Regards,
Mat

Hi Mat,

I'm using a Netgear ReadyNAS NV+ device, which they tell me is running PROftpd. They specifically told me they supported SITE UTIME. When I ftp'd a file to it, it simply got timestamped with the time of the transfer. Below are the messages in the log:

[20:03:08] SmartFTP v3.0.1021.17
[20:03:09] Resolving host name "192.168.1.252"
[20:03:09] Connecting to 192.168.1.252 Port: 21
[20:03:09] Connected to 192.168.1.252.
[20:03:09] 220 ProFTPD 1.3.1 Server (NETGEAR ReadyNAS) [192.168.1.252]
[20:03:09] USER admin
[20:03:09] 331 Password required for admin
[20:03:09] PASS (hidden)
[20:03:09] 230 User admin logged in
[20:03:09] SYST
[20:03:09] 215 UNIX Type: L8
[20:03:09] Detected Server Type: UNIX
[20:03:09] FEAT
[20:03:09] 211-Features:
[20:03:09] LANG en
[20:03:09] MDTM
[20:03:09] UTF8
[20:03:09] AUTH TLS
[20:03:09] PBSZ
[20:03:09] PROT
[20:03:09] REST STREAM
[20:03:09] SIZE
[20:03:10] 211 End
[20:03:10] OPTS UTF8 ON
[20:03:10] 200 UTF8 set to on
[20:03:10] PWD
[20:03:10] 257 "/" is the current directory
[20:03:10] CWD /Telescope-PC
[20:03:10] 250 CWD command successful
[20:03:10] PWD
[20:03:10] 257 "/Telescope-PC" is the current directory
[20:03:10] TYPE A
[20:03:10] 200 Type set to A
[20:03:10] PASV
[20:03:10] 227 Entering Passive Mode (192,168,1,252,106,241).
[20:03:10] Opening data connection to 192.168.1.252 Port: 27377
[20:03:10] LIST -aL
[20:03:10] 150 Opening ASCII mode data connection for file list
[20:03:10] 189 bytes transferred. (2.88 KB/s) (64 ms)
[20:03:10] 226 Transfer complete
[20:03:41] NOOP
[20:03:41] 200 NOOP command successful
[20:04:11] NOOP
[20:04:11] 200 NOOP command successful
[20:04:42] NOOP
[20:04:42] 200 NOOP command successful
[20:05:12] NOOP
[20:05:12] 200 NOOP command successful
[20:05:43] NOOP
[20:05:43] 200 NOOP command successful
[20:06:13] NOOP
[20:06:13] 200 NOOP command successful
[20:06:30] TYPE I
[20:06:30] 200 Type set to I
[20:06:30] SIZE Ha293E_IC434_02786.fit
[20:06:30] 550 Ha293E_IC434_02786.fit: No such file or directory
[20:06:30] The operation has been added to the Transfer Queue. Check the Transfer Queue for the status.
[20:06:35] SIZE Ha293E_IC434_02786.fit
[20:06:35] 213 21427200
[20:06:35] MDTM Ha293E_IC434_02786.fit
[20:06:35] 213 20080819000635
[20:06:35] STAT Ha293E_IC434_02786.fit
[20:06:35] 211-Status of Ha293E_IC434_02786.fit:
[20:06:35] 211--rw-rw-rw- 1 admin admin 21427200 Aug 18 20:06 Ha293E_IC434_02786.fit
[20:06:35] 211 End of status

I don't see the UTIME command being used by SmartFTP but I do see MDTM, which Netgear tells me isn't supported for uploads.

Any suggestions?

Thank you,
Jonathan

2008-08-19 04:03:58

Look at the log of the transfer queue item. To get it double click the file in the transfer queue. Not all proftpd servers support the correct UTIME command.

Regards,
Mat

jeburnett

2008-08-19 10:32:51

Look at the log of the transfer queue item. To get it double click the file in the transfer queue. Not all proftpd servers support the correct UTIME command.

Regards,
Mat

Hmmmmmm.

[06:31:12] Initializing. Transfer Queue v1.0.14.17.
[06:31:12] Resolving host name "192.168.1.252"
[06:31:12] Connecting to 192.168.1.252 Port: 21
[06:31:12] Connected to 192.168.1.252.
[06:31:12] 220 ProFTPD 1.3.1 Server (NETGEAR ReadyNAS) [192.168.1.252]
[06:31:12] USER admin
[06:31:12] 331 Password required for admin
[06:31:12] PASS (hidden)
[06:31:12] 230 User admin logged in
[06:31:12] SYST
[06:31:12] 215 UNIX Type: L8
[06:31:12] Detected Server Type: UNIX
[06:31:12] FEAT
[06:31:12] 211-Features:
[06:31:12] LANG en
[06:31:12] MDTM
[06:31:12] UTF8
[06:31:12] AUTH TLS
[06:31:12] PBSZ
[06:31:12] PROT
[06:31:12] REST STREAM
[06:31:12] SIZE
[06:31:12] 211 End
[06:31:12] OPTS UTF8 ON
[06:31:12] 200 UTF8 set to on
[06:31:12] PWD
[06:31:12] 257 "/" is the current directory
[06:31:12] CWD /Telescope-PC
[06:31:12] 250 CWD command successful
[06:31:12] PWD
[06:31:12] 257 "/Telescope-PC" is the current directory
[06:31:12] Remote file exist check: "Ha293E_IC434_02791.fit".
[06:31:12] TYPE I
[06:31:12] 200 Type set to I
[06:31:12] SIZE Ha293E_IC434_02791.fit
[06:31:12] 550 Ha293E_IC434_02791.fit: No such file or directory
[06:31:12] File exist check failed. File not found or permission denied.
[06:31:12] PASV
[06:31:12] 227 Entering Passive Mode (192,168,1,252,151,251).
[06:31:12] Opening data connection to 192.168.1.252 Port: 38907
[06:31:12] STOR Ha293E_IC434_02791.fit
[06:31:12] 150 Opening BINARY mode data connection for Ha293E_IC434_02791.fit
[06:31:16] 21427200 bytes transferred. (5.60 MB/s) (00:00:03)
[06:31:16] 226 Transfer complete
[06:31:16] MDTM 20071206054206 Ha293E_IC434_02791.fit
[06:31:16] 550 20071206054206 Ha293E_IC434_02791.fit: No such file or directory
[06:31:16] MFMT 20071206054206 Ha293E_IC434_02791.fit
[06:31:16] 500 MFMT not understood
[06:31:16] SITE UTIME Ha293E_IC434_02791.fit 20071206054206 20071206054206 20071206054206 UTC
[06:31:16] 500 'SITE UTIME' not understood
[06:31:16] SITE UTIME 20071206054206 Ha293E_IC434_02791.fit
[06:31:16] 500 'SITE UTIME' not understood
[06:31:16] SIZE Ha293E_IC434_02791.fit
[06:31:16] 213 21427200

2008-08-19 14:22:21

As you can see from the log your FTP server does not support the SITE UTIME command.

jeburnett

2008-09-06 19:54:45

As you can see from the log your FTP server does not support the SITE UTIME command.

Netgear released a new version of firmware that they say will support UTIME. I've transferred a file using it and captured your log to see what went on. The file's time/date stamp was NOT preserved and there is a error message in the transfer log. Your interpretation, please.

Thank you,
Jonathan

[15:17:47] Initializing. Transfer Queue v1.0.14.17.
[15:17:47] Resolving host name "192.168.1.252"
[15:17:47] Connecting to 192.168.1.252 Port: 21
[15:17:47] Connected to 192.168.1.252.
[15:17:47] 220 ProFTPD 1.3.1 Server (NETGEAR ReadyNAS) [192.168.1.252]
[15:17:47] USER admin
[15:17:47] 331 Password required for admin
[15:17:47] PASS (hidden)
[15:17:48] 230 User admin logged in
[15:17:48] SYST
[15:17:48] 215 UNIX Type: L8
[15:17:48] Detected Server Type: UNIX
[15:17:48] FEAT
[15:17:48] 211-Features:
[15:17:48] LANG en
[15:17:48] MDTM
[15:17:48] UTF8
[15:17:48] AUTH TLS
[15:17:48] PBSZ
[15:17:48] PROT
[15:17:48] REST STREAM
[15:17:48] SIZE
[15:17:48] 211 End
[15:17:48] OPTS UTF8 ON
[15:17:48] 200 UTF8 set to on
[15:17:48] PWD
[15:17:48] 257 "/" is the current directory
[15:17:48] CWD /Telescope-PC
[15:17:48] 250 CWD command successful
[15:17:48] PWD
[15:17:48] 257 "/Telescope-PC" is the current directory
[15:17:48] Remote file exist check: "Ha293E_IC434_02789.fit".
[15:17:48] TYPE I
[15:17:48] 200 Type set to I
[15:17:48] SIZE Ha293E_IC434_02789.fit
[15:17:48] 550 Ha293E_IC434_02789.fit: No such file or directory
[15:17:48] File exist check failed. File not found or permission denied.
[15:17:48] PASV
[15:17:48] 227 Entering Passive Mode (192,168,1,252,79,213).
[15:17:48] Opening data connection to 192.168.1.252 Port: 20437
[15:17:48] STOR Ha293E_IC434_02789.fit
[15:17:48] 150 Opening BINARY mode data connection for Ha293E_IC434_02789.fit
[15:17:51] 21427200 bytes transferred. (5.76 MB/s) (00:00:03)
[15:17:51] 226 Transfer complete
[15:17:51] MDTM 20071206044230 Ha293E_IC434_02789.fit
[15:17:51] 550 20071206044230 Ha293E_IC434_02789.fit: No such file or directory
[15:17:51] MFMT 20071206044230 Ha293E_IC434_02789.fit
[15:17:51] 500 MFMT not understood
[15:17:51] SITE UTIME Ha293E_IC434_02789.fit 20071206044230 20071206044230 20071206044230 UTC
[15:17:51] 500 UTIME Ha293E_IC434_02789.fit 20071206044230 20071206044230 20071206044230 UTC: Invalid argument
[15:17:51] SITE UTIME 20071206044230 Ha293E_IC434_02789.fit
[15:17:51] 500 UTIME 20071206044230 Ha293E_IC434_02789.fit: Invalid argument
[15:17:51] SIZE Ha293E_IC434_02789.fit
[15:17:51] 213 21427200

2008-09-06 20:32:32

They have an incomplete UTIME implementation:

[15:17:51] SITE UTIME 20071206044230 Ha293E_IC434_02789.fit
[15:17:51] 500 UTIME 20071206044230 Ha293E_IC434_02789.fit: Invalid argument

Their UTIME command does not support seconds in the argument. The mod_misc from ProFTPd supports seconds in the argument. Also all other UTIME implementations support seconds which makes sense of course.

Regards,
Mat

jeburnett

2008-09-06 21:06:53

They have an incomplete UTIME implementation:

[15:17:51] SITE UTIME 20071206044230 Ha293E_IC434_02789.fit
[15:17:51] 500 UTIME 20071206044230 Ha293E_IC434_02789.fit: Invalid argument

Their UTIME command does not support seconds in the argument. The mod_misc from ProFTPd supports seconds in the argument. Also all other UTIME implementations support seconds which makes sense of course.

Regards,
Mat

Thank you Mat. I've shared this with them. Hopefully they can resolve this.

Jonathan