Curl file download error

[Thogiti, Nagaraju]

Hi,

I wrote a MS SSIS package to download a set of XML files given a URL.
The SSIS package uses CURL to download these XML files as an Execute
Process Task. I am reading the curl argument from a SQL table. I checked
the curl commands manually in a shell and they all work fine.

I am passing the curl argument in a variable to the task by reading from
a SQL table. It downloads some files but fails on some file. But when I
take the exact same curl command and run manually in a shell, it
downloads the file.

When I rerun the package, the execution fails at some other file and
processes previously failed file successfully.

Error: 0xC0029151 at cURL, Execute Process Task: In Executing
"C:\curl\curl.exe" "-o somefilename.xml "URL"" at "", The process exit
code was "56" while the expected was "0".

I was pointed out on the stackoverflow that the curl error 56 was
CURL_RECV_ERROR, a problem related to network. I also tried with -retry
option but the problem doesn't seem to go away.

I was thinking if there is a way to tell curl pause for 2 or 3 seconds
before it pings the next request.

If you have any suggestions, please let me know.

Thank you,

Nagu







Nagaraju Thogiti

Data Mining Analyst - Business Intelligence

PC Mall Inc.

***@pcmall.com <mailto:***@pcmall.com>

Tel: [phone]

www.pcmall.com <http://www.pcmall.com/>





_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

The information contained in this transmission is confidential. It is
intended solely for the use of the individual(s) or organization(s) to
whom it is addressed. Any disclosure, copying or further distribution is
not permitted unless such privilege is explicitly granted in writing by
PC Mall, Inc. Furthermore, PC Mall, Inc. is not responsible for
the proper and complete transmission of the substance of this
communication, nor for any delay in its receipt.

[Daniel Stenberg]

Post by Thogiti, Nagaraju
Error: 0xC0029151 at cURL, Execute Process Task: In Executing
"C:\curl\curl.exe" "-o somefilename.xml "URL"" at "", The process exit
code was "56" while the expected was "0".
I was pointed out on the stackoverflow that the curl error 56 was
CURL_RECV_ERROR, a problem related to network. I also tried with -retry
option but the problem doesn't seem to go away.

Perhaps you should consider using --trace-ascii to see if that reveals
anything, or doing actual network capturing to see what it might show happens
at the time the problem occurs.

Post by Thogiti, Nagaraju
I was thinking if there is a way to tell curl pause for 2 or 3 seconds
before it pings the next request.

"before it sends the next request" would be a correct way to phrase that.

curl has no such option, but you can of course make your own code/script do it
like that.

[Dan Fandrich]

Post by Daniel Stenberg

Post by Thogiti, Nagaraju
I was thinking if there is a way to tell curl pause for 2 or 3 seconds
before it pings the next request.

"before it sends the next request" would be a correct way to phrase that.
curl has no such option, but you can of course make your own
code/script do it like that.

The --retry-delay option can be used to delay before retrying the current
request.

Post by Daniel Stenberg

Post by Thogiti, Nagaraju

Dan

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html

[Daniel Stenberg]

Post by Dan Fandrich
The --retry-delay option can be used to delay before retrying the current
request.

Ouch, how could I forget about that... Thanks Dan!