Problem using UploadFile32 through HTTP Proxy

brenaaro

2005-05-24 14:07:14

When I upload a file using the SmartFTP library through an HTTP proxy (Squid), there is a chunk of file missing on the remote server. (The local file is 20 KB, the remote file is only 18 KB)

However, if I upload using the SmartFTP Client (again through the same HTTP proxy) the file is transfered correctly. Also, I do not have a problem when using DownloadFile32() from the library.

Here is the code I am using:

Dim wCon As FTPConnection
Dim e As enumError

Set wCon = New FTPConnection

wCon.Client = "smartftp"

wCon.ProxyType = ftpProxyTypeHTTP
wCon.ProxyHost = "ttimprx1"
wCon.ProxyPort = 3128

wCon.Host = "10.1.3.207"
wCon.Port = 21
wCon.Username = "user"
wCon.Password = "user1"

wCon.DataTransferType = ftpDataTransferTypeImage

e = wCon.Connect()

If (e <> ftpErrorSuccess) Then
MsgBox "could not connect"
Exit Sub
End If

If wCon.UploadFile32("c:dev2bonjour.doc", "/merx/bonjour.doc", 0) <> ftpErrorSuccess Then
MsgBox "could not upload"
End If

wCon.Disconnect

Is this a bug?

Edit: I am using build 1.0.1.22

2005-05-26 04:10:38

Hello ...

I apologize for not replying earlier. I will test it and see if there is a bug.

It would help if you can post the log of the ftp session. Thank you.

-Mat

brenaaro

2005-05-26 14:43:34

Here is the log:

----------------------

CONNECT 10.1.22.99:21 HTTP/1.0

HTTP/1.0 200 Connection established

220 ftp.gslb.dev.mediagrif.com NcFTPd Server (licensed copy) ready.

USER tim

331 User tim okay, need password.

PASS *******

230 Restricted user logged in.

SYST

215 UNIX Type: L8

FEAT

211-Extensions supported:

CLNT

LANG EN*

MDTM

MLST Type*;Size*;Modify*;Perm;Unique;UNIX.mode*;UNIX.owner;UNIX.uid;UNIX.group;UNIX.gid;

PASV

REST STREAM

SIZE

UTF8

TVFS

Compliance Level: 20040701 (IETF mlst-16)

211 End.

CLNT smartftp

200 Noted.

SITE UTF8

501-The following SITE commands are recognized:

BUFSIZE

CHMOD

DATE

DF

QUOTA

RBUFSIZ

RBUFSZ

RETRBUFSIZE

SBUFSIZ

SBUFSZ

STORBUFSIZE

SYMLINK

UMASK

UTIME

501

PWD

257 "/" is cwd.

TYPE I

200 Type okay.

PASV

227 Entering Passive Mode (10,1,22,99,132,80)

STOR /merx/ZipLib.zip

150 Data connection accepted from 207.139.157.238:22163; transfer starting.

226 Transfer completed.

QUIT

221 Goodbye.

----------------------

I have done some other tests and I notice that with larger files (12 MB, 50MB), only the first 32 KB of the file is stored on the remote server, even though the call to PutFile32() runs long enough to actually be transfering the entire 12 or 50 MB.

Thanks for the reply, and let me know if you need anything else.

brenaaro

2005-05-26 18:02:20

Another thing I noticed:

We took some snoops of the traffic being sent (SmartFTP Client vs SmartFTP Library), and the only major thing we noticed is that the Client's CONNECT request specifies HTTP/1.1, while the Library specifies HTTP/1.0 ...in both cases however, the proxy accepts the connection and inficates HTTP/1.0 ...not sure if that helps.

2005-05-27 13:32:08

Hello ...

Bug fixed in 1.0.1.19. Please get the latest build.

Thank you for reporting.
Regards,
-Mat

brenaaro

2005-05-27 15:18:32

Thanks for the quick response. Unfortunately, we are still encountering the same problem - though it seems to be inconsistent. On some occasions it will transfer the entire file, othertimes not. I have included two logs:

Unsuccessful:

CONNECT 10.1.22.99:21 HTTP/1.0

HTTP/1.0 200 Connection established

220 ftp.gslb.dev.mediagrif.com NcFTPd Server (licensed copy) ready.

USER tim

331 User tim okay, need password.

PASS *******

230 Restricted user logged in.

SYST

215 UNIX Type: L8

FEAT

211-Extensions supported:

CLNT

LANG EN*

MDTM

MLST Type*;Size*;Modify*;Perm;Unique;UNIX.mode*;UNIX.owner;UNIX.uid;UNIX.group;UNIX.gid;

PASV

REST STREAM

SIZE

UTF8

TVFS

Compliance Level: 20040701 (IETF mlst-16)

211 End.

CLNT smartftp

200 Noted.

SITE UTF8

501-The following SITE commands are recognized:

BUFSIZE

CHMOD

DATE

DF

QUOTA

RBUFSIZ

RBUFSZ

RETRBUFSIZE

SBUFSIZ

SBUFSZ

STORBUFSIZE

SYMLINK

UMASK

UTIME

501

PWD

257 "/" is cwd.

CWD merx

250 "/merx" is new cwd.

PWD

257 "/merx" is cwd.

TYPE I

200 Type okay.

PASV

227 Entering Passive Mode (10,1,22,99,139,8)

STOR bonjour2.doc

150 Data connection accepted from 207.139.157.238:4377; transfer starting.

226 Transfer completed.

PASV

227 Entering Passive Mode (10,1,22,99,139,9)

RETR bonjour2.doc

150 Data connection accepted from 207.139.157.238:4405; transfer starting for bonjour2.doc (18432 bytes).

226 Transfer completed.

QUIT

221 Goodbye.

*******************************
and successful:

CONNECT 10.1.22.99:21 HTTP/1.0

HTTP/1.0 200 Connection established

220 ftp.gslb.dev.mediagrif.com NcFTPd Server (licensed copy) ready.

USER tim

331 User tim okay, need password.

PASS *******

230 Restricted user logged in.

SYST

215 UNIX Type: L8

FEAT

211-Extensions supported:

CLNT

LANG EN*

MDTM

MLST Type*;Size*;Modify*;Perm;Unique;UNIX.mode*;UNIX.owner;UNIX.uid;UNIX.group;UNIX.gid;

PASV

REST STREAM

SIZE

UTF8

TVFS

Compliance Level: 20040701 (IETF mlst-16)

211 End.

CLNT smartftp

200 Noted.

SITE UTF8

501-The following SITE commands are recognized:

BUFSIZE

CHMOD

DATE

DF

QUOTA

RBUFSIZ

RBUFSZ

RETRBUFSIZE

SBUFSIZ

SBUFSZ

STORBUFSIZE

SYMLINK

UMASK

UTIME

501

PWD

257 "/" is cwd.

CWD merx

250 "/merx" is new cwd.

PWD

257 "/merx" is cwd.

TYPE I

200 Type okay.

PASV

227 Entering Passive Mode (10,1,22,99,139,26)

STOR bonjour2.doc

150 Data connection accepted from 207.139.157.238:25583; transfer starting.

226 Transfer completed.

PASV

227 Entering Passive Mode (10,1,22,99,139,27)

RETR bonjour2.doc

150 Data connection accepted from 207.139.157.238:25574; transfer starting for bonjour2.doc (20480 bytes).

226 Transfer completed.

QUIT

221 Goodbye.

*************

Note that the RETR is working properly, it is the remote file (after the STOR) that is the incorrect size. Thanks for looking into this.

2005-05-27 15:31:34

Please post the complete log. Your log only contains server replies and commands sent to the server. Use the LogFile property of the IFTPConnection class/interface to get the complete log.

It would be helpful if you send a SIZE command with the GetFileSize method after the upload. This would help figuring out if the problem is with the upload or download.

Thank you.
-Mat

brenaaro

2005-05-27 15:42:56

Sorry, those were Ethereal logs. Here is the SmartFTP Log:

[20050527 15:40:51] SmartFTP FTP Library v1.0.1.29
[20050527 15:40:51] Resolving host name ftp.gslb.dev.mediagrif.com
[20050527 15:40:51] Connecting to 10.1.22.99 Port: 21
[20050527 15:40:51] Proxy: Connecting to HTTP proxy server ttimprx1 Port: 3128
[20050527 15:40:51] Proxy: Connected to proxy server. Sending connection request.
[20050527 15:40:51] Connected to ftp.gslb.dev.mediagrif.com.
[20050527 15:40:51] 220 ftp.gslb.dev.mediagrif.com NcFTPd Server (licensed copy) ready.
[20050527 15:40:51] USER tim
[20050527 15:40:51] 331 User tim okay, need password.
[20050527 15:40:51] PASS *******
[20050527 15:40:51] 230 Restricted user logged in.
[20050527 15:40:51] SYST
[20050527 15:40:51] 215 UNIX Type: L8
[20050527 15:40:51] FEAT
[20050527 15:40:51] 211-Extensions supported:
[20050527 15:40:51] CLNT
[20050527 15:40:51] LANG EN*
[20050527 15:40:51] MDTM
[20050527 15:40:51] MLST Type*;Size*;Modify*;Perm;Unique;UNIX.mode*;UNIX.owner;UNIX.uid;UNIX.group;UNIX.gid;
[20050527 15:40:51] PASV
[20050527 15:40:51] REST STREAM
[20050527 15:40:51] SIZE
[20050527 15:40:51] UTF8
[20050527 15:40:51] TVFS
[20050527 15:40:51] Compliance Level: 20040701 (IETF mlst-16)
[20050527 15:40:51] 211 End.
[20050527 15:40:51] Detected Server Software: WS_FTP Server 3.x
[20050527 15:40:51] CLNT smartftp
[20050527 15:40:51] 200 Noted.
[20050527 15:40:51] SITE UTF8
[20050527 15:40:51] 501-The following SITE commands are recognized:
[20050527 15:40:51] BUFSIZE
[20050527 15:40:51] CHMOD
[20050527 15:40:51] DATE
[20050527 15:40:51] DF
[20050527 15:40:51] QUOTA
[20050527 15:40:51] RBUFSIZ
[20050527 15:40:51] RBUFSZ
[20050527 15:40:51] RETRBUFSIZE
[20050527 15:40:51] SBUFSIZ
[20050527 15:40:51] SBUFSZ
[20050527 15:40:51] STORBUFSIZE
[20050527 15:40:51] SYMLINK
[20050527 15:40:51] UMASK
[20050527 15:40:51] UTIME
[20050527 15:40:51] 501
[20050527 15:40:51] PWD
[20050527 15:40:51] 257 "/" is cwd.
[20050527 15:40:51] CWD merx
[20050527 15:40:51] 250 "/merx" is new cwd.
[20050527 15:40:51] PWD
[20050527 15:40:51] 257 "/merx" is cwd.
[20050527 15:40:51] TYPE I
[20050527 15:40:51] 200 Type okay.
[20050527 15:40:51] PASV
[20050527 15:40:51] 227 Entering Passive Mode (10,1,22,99,139,52)
[20050527 15:40:51] Opening data connection to 10.1.22.99 Port: 35636
[20050527 15:40:51] Proxy: Connecting to HTTP proxy server ttimprx1 Port: 3128
[20050527 15:40:51] STOR bonjour2.doc
[20050527 15:40:51] Proxy: Connected to proxy server. Sending connection request.
[20050527 15:40:51] 150 Data connection accepted from 207.139.157.238:5106; transfer starting.
[20050527 15:40:51] 20480 bytes transferred. (1.22 MB/s) (16 ms)
[20050527 15:40:51] 226 Transfer completed.
[20050527 15:40:51] SIZE bonjour2.doc
[20050527 15:40:51] 213 18432
[20050527 15:40:52] PASV
[20050527 15:40:52] 227 Entering Passive Mode (10,1,22,99,139,53)
[20050527 15:40:52] Opening data connection to 10.1.22.99 Port: 35637
[20050527 15:40:52] Proxy: Connecting to HTTP proxy server ttimprx1 Port: 3128
[20050527 15:40:52] RETR bonjour2.doc
[20050527 15:40:52] Proxy: Connected to proxy server. Sending connection request.
[20050527 15:40:52] 150 Data connection accepted from 207.139.157.238:5079; transfer starting for bonjour2.doc (18432 bytes).
[20050527 15:40:52] 18432 bytes transferred. (1.17 MB/s) (15 ms)
[20050527 15:40:52] 226 Transfer completed.
[20050527 15:40:52] QUIT
[20050527 15:40:52] 221 Goodbye.
[20050527 15:40:52] Client closed the connection.

2005-05-27 16:16:31

Thank you.

Whats the size of the actual file (bonjour2.doc)?

It seems exactly 10 x 2048 bytes are transferred. 2048 is the size of the send buffer we use. The last block didn't arrive at the server 20480 - 18432 = 2048.

Do you none of these problems with SmartFTP v1?

BTW: If you want to communicate through MSN or ICQ please send me a private message with your IM id.

Thanks
-Mat