Multipart/mixed from Config File

[J***@farrow.com]

Hello

Is it possible to create a custom cURL config file, for SSLv3 HTTP Request
with headers with various 'Content-type', 'boundary' between them and
'--data' loaded within places (2 different 'Content-type'-s).

How to encode a 'boundary' into config file?
How to encode a blank line (CRLF) into config file?

Thanks
Jan

This electronic message contains information from Russell A. Farrow
Limited, Russell A. Farrow (U.S.) Inc., canadaplus.com or RAF Express
Inc., that may be confidential or privileged. The information in this
electronic message is only for the use of the intended recipient.
Inappropriate disclosure, copying, distribution or other use of the
contents of this electronic message is strictly prohibited. If you have
received this electronic transmission in error, please notify the sender
immediately by return e-mail. Thank you.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html

[Daniel Stenberg]

Post by J***@farrow.com
Is it possible to create a custom cURL config file, for SSLv3 HTTP Request
with headers with various 'Content-type', 'boundary' between them and
'--data' loaded within places (2 different 'Content-type'-s).

Multipart formposts are made with -F, and no it's not possible to set your own
boundary string.

[J***@farrow.com]

Post by Daniel Stenberg

Post by J***@farrow.com
Is it possible to create a custom cURL config file, for SSLv3 HTTP Request
with headers with various 'Content-type', 'boundary' between them and
'--data' loaded within places (2 different 'Content-type'-s).

Multipart formposts are made with -F, and no it's not possible to set your own
boundary string.

Thanks Daniel for your reply.
I had some success with 'boundary' and multiple use of -F parameters
but...
My header for (SSL) HTTP Request need to looks like this:

Content-type: multipart/mixed; boundary=BOUNDARY<cr/lf>
Content-length: 1036<cr/lf>
<cr/lf>
--BOUNDARY<cr/lf>
Content-type: application/x-www-form-urlencoded<cr/lf>
Content-length: 136<cr/lf>
<cr/lf>
AppVersion=1.0&AcceptUPSLicenseAgreement=Yes&ResponseType=application/x-ups-pld&VersionNumber=V4R1&UserId=user&Password=password<cr/lf>
<cr/lf>
--BOUNDARY<cr/lf>
Content-type: application/x-ups-binary<cr/lf>
Content-length: 719<cr/lf>
<cr/lf>
<data I'm uploading in here><cr/lf>
<cr/lf>
--BOUNDARY--<cr/lf>
.....

and my command like looks like this:

curl -H "User-Agent:" -H "Accept:" -H "Expect:" -H "Content-Type:
multipart/mixed; boundary=BOUNDARY" -F
"Filename=<C:\RAF\Curl\curl-7.20.0\RAF2.txt;type=application/x-www-form-urlencoded"
-F "Filename=<C:\Temp\0066214000.TXT;type=application/x-ups-binary" -k
https://...

I am loading the following lines from the file "RAF2.txt":

Content-length: 136

AppVersion=1.0&AcceptUPSLicenseAgreement=Yes&ResponseType=application/x-ups-pld&VersionNumber=V4R1&UserId=user&Password=password

but I'm getting a response from server 'Missing application version'.
What am I doing wrong?
How can that be done with Multipart formposts using -F parameter(s)?
(I'm relatively new to cURL - command line only, PHP is not an option, OS
= IBM AIX - Application = Progress)

Thanks
Jan

This electronic message contains information from Russell A. Farrow
Limited, Russell A. Farrow (U.S.) Inc., canadaplus.com or RAF Express
Inc., that may be confidential or privileged. The information in this
electronic message is only for the use of the intended recipient.
Inappropriate disclosure, copying, distribution or other use of the
contents of this electronic message is strictly prohibited. If you have
received this electronic transmission in error, please notify the sender
immediately by return e-mail. Thank you.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html

[Daniel Stenberg]

--trace and --trace-ascii are fine to use to get to see exactly what curl
sends.

Post by J***@farrow.com
but I'm getting a response from server 'Missing application version'.
What am I doing wrong?

That "application version" is probably some kind of data that the server
expects you to send to it. We can't really tell.

[J***@farrow.com]

Sorry if I wasn't clear.

Post by Daniel Stenberg
--trace and --trace-ascii are fine to use to get to see exactly what curl
sends.

I am using -trace-ascii, just removed it from (already too long) command
line in my e-mail.

Post by Daniel Stenberg
That "application version" is probably some kind of data that the server
expects you to send to it. We can't really tell.

As I said, I'm loading the following lines from the file mentioned in
first reference to -F, and AppVersion is the first string in the 3-rd
line:


Content-length: 136

AppVersion=1.0&AcceptUPSLicenseAgreement=Yes&ResponseType=application/x-ups-pld&VersionNumber=V4R1&UserId=user&Password=password


Jan

This electronic message contains information from Russell A. Farrow
Limited, Russell A. Farrow (U.S.) Inc., canadaplus.com or RAF Express
Inc., that may be confidential or privileged. The information in this
electronic message is only for the use of the intended recipient.
Inappropriate disclosure, copying, distribution or other use of the
contents of this electronic message is strictly prohibited. If you have
received this electronic transmission in error, please notify the sender
immediately by return e-mail. Thank you.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html

[J***@farrow.com]

Let me try to explain it again... (sorry for the long message)

This is how the HTTP Request Message suppose to look like:

Content-type: multipart/mixed; boundary=BOUNDARY<cr/lf>
Content-length: 1036<cr/lf>
<cr/lf>
--BOUNDARY<cr/lf>
Content-type: application/x-www-form-urlencoded<cr/lf>
Content-length: 136<cr/lf>
<cr/lf>
AppVersion=1.0&AcceptUPSLicenseAgreement=Yes&ResponseType=application/x-ups-pld&VersionNumber=V4R1&UserId=password&Password=password<cr/lf>
<cr/lf>
--BOUNDARY<cr/lf>
Content-type: application/x-ups-binary<cr/lf>
Content-length: 719<cr/lf>
<cr/lf>
<MAIN PORTION OF THE DATA HERE><cr/lf>
<cr/lf>
--BOUNDARY--<cr/lf>


This is my cURL command line...

curl -H "User-Agent:" -H "Accept:" -H "Expect:" -H "Content-Type:
multipart/mixed; boundary=BOUNDARY" -F
"Filename=<C:\DATA1.TXT;type=application/x-www-form-urlencoded" -F
"Filename=<C:\DATA2.TXT;type=application/x-ups-binary" -o
"C:\curl-resp.txt" --trace-ascii "C:\curl-trace.txt" -k https://www.......


This is what is in the file DATA1.TXT:

Content-length: 136<cr/lf>
<cr/lf>
AppVersion=1.0&AcceptUPSLicenseAgreement=Yes&ResponseType=application/x-ups-pld&VersionNumber=V4R1&UserId=userid&Password=password<cr/lf>


This is from cULR trace file:
...
0044: Content-Length: 1211
005a: Content-Type: multipart/mixed; boundary=BOUNDARY; boundary=-----
009a: -----------------------814d8b8ab9f8
00bf:
=> Send data, 144 bytes (0x90)
0000: ------------------------------814d8b8ab9f8
002c: Content-Disposition: form-data; name="Filename"
005d: Content-Type: application/x-www-form-urlencoded
008e:
=> Send data, 163 bytes (0xa3)
0000: Content-length: 136
0015:
0017: AppVersion=1.0&AcceptUPSLicenseAgreement=Yes&ResponseType=applic
0057: ation/x-ups-pld&VersionNumber=V4R1&UserId=userid&Password=passwo
0097: rd
00a1:
=> Send data, 137 bytes (0x89)
0000:
0002: ------------------------------814d8b8ab9f8
002e: Content-Disposition: form-data; name="Filename"
005f: Content-Type: application/x-ups-binary
0087:
=> Send data, 719 bytes (0x2cf)
<DATA FROM FILE DATA2.TXT>
=> Send data, 48 bytes (0x30)
0000:
0002: ------------------------------814d8b8ab9f8--
<= Recv header, 17 bytes (0x11)
0000: HTTP/1.1 200 OK
...


This is response from the server:
...
00c9: --BOUNDARY
00d5: Content-type: application/x-ups-psmpld
00fd: Content-length: 104
0112:
0114: UPSOnLine%null%6931%6800AppVersion missing.
0154:
017a:
017c: --BOUNDARY
0188: Content-type: application/x-ups-pld
01ad: Content-length: 110
01c2:
01c4: 00010000009710000008800026800AppVersion missing.
0204:
0233:
0235: --BOUNDARY--
...


How the content of the file DATA1.TXT need to send in order for server to
see the AppVersion...?

This electronic message contains information from Russell A. Farrow
Limited, Russell A. Farrow (U.S.) Inc., canadaplus.com or RAF Express
Inc., that may be confidential or privileged. The information in this
electronic message is only for the use of the intended recipient.
Inappropriate disclosure, copying, distribution or other use of the
contents of this electronic message is strictly prohibited. If you have
received this electronic transmission in error, please notify the sender
immediately by return e-mail. Thank you.

[Daniel Stenberg]

Post by J***@farrow.com
Content-type: multipart/mixed; boundary=BOUNDARY<cr/lf>

You can't set your own boundary string with curl. If you NEED that, you must
instead generate the entire thing yourself and send it with -d. It should be
very rare though.

Post by J***@farrow.com
Content-type: application/x-www-form-urlencoded<cr/lf>
Content-length: 136<cr/lf>

curl won't use Content-length: for individual sub-parts.

Post by J***@farrow.com
-H "Content-Type: multipart/mixed; boundary=BOUNDARY"

This doesn't really work. curl (libcurl actually) generates and uses its own
boundary whatever you try to tell it.

Post by J***@farrow.com
Content-length: 136<cr/lf>

This won't work, as curl expects you to pass on data, not headers.

Post by J***@farrow.com
0044: Content-Length: 1211
005a: Content-Type: multipart/mixed; boundary=BOUNDARY; boundary=-----
009a: -----------------------814d8b8ab9f8

See? It got confused by your attempt to replace it. :-)

Post by J***@farrow.com
0000: ------------------------------814d8b8ab9f8
002c: Content-Disposition: form-data; name="Filename"
005d: Content-Type: application/x-www-form-urlencoded
=> Send data, 163 bytes (0xa3)
0000: Content-length: 136

And this passes "Content-length: 136" as part of the actual payload.

Post by J***@farrow.com
How the content of the file DATA1.TXT need to send in order for server to
see the AppVersion...?

I suggest you cut out the Content-length: line. I've never seen a server-side
receiver of multipart formposts require content-length for a subpart.

Post by J***@farrow.com
Inappropriate disclosure, copying, distribution or other use of the contents
of this electronic message is strictly prohibited.

Is replying to the mail considered "other use"? Is this strictly prohibited?
Man, those dislaimers are stupid and pointless. Whine on your managers for
polluting the world with such crap.

[J***@farrow.com]

Well I have tried 2 things:

1. I now understand the 'boundary' logic, so I left out of my command line
"boundary=BOUNDARY" and also left out the Content-length: for individual
sub-parts.
But I'm still getting the feedback from server 'Missing AppVersion'.
- also tried with just a 'AppVersion=1.0' in that DATA1.TXT file -
same result: 'Missing AppVersion'
- also tried with url-encoding a 'AppVersion=1.0' as
'AppVersion=1%2E0' in that DATA1.TXT file - same result: 'Missing
AppVersion'

2.

Post by Daniel Stenberg
You can't set your own boundary string with curl. If you NEED that, you must
instead generate the entire thing yourself and send it with -d. It should be
very rare though.

This will be a preferred solution for us (to construct the entire thing
and send it with -d). I have tied to send the following:

Content-Type: multipart/mixed;
boundary=------------------------------814d8b8ab9f8
Content-length: 1036

------------------------------814d8b8ab9f8
Content-type: application/x-www-form-urlencoded
Content-length: 136

AppVersion=1.0&AcceptUPSLicenseAgreement=Yes&ResponseType=application/x-ups-pld&VersionNumber=V4R1&UserId=userid&Password=password

------------------------------814d8b8ab9f8
Content-type: application/x-ups-binary
Content-length: 719

020093aaa-bbbb V2010.03
2010031000000006621400000085000000001

------------------------------814d8b8ab9f8--


cURL sent the following:

=> Send data, 560 bytes (0x230)
0000: Content-Type: multipart/mixed; boundary=------------------------
0040: ------814d8b8ab9f8Content-length: 1036--------------------------
0080: ----814d8b8ab9f8Content-type: application/x-www-form-urlencodedC
00c0: ontent-length: 136AppVersion=1.0&AcceptUPSLicenseAgreement=Yes&R
0100: esponseType=application/x-ups-pld&VersionNumber=V4R1&UserId=user
0140: id&Password=password------------------------------814d8b8ab9f8Co
0180: ntent-type: application/x-ups-binaryContent-length: 719020093aaa
01c0: -bbbb V2010.03 201003100000000662140000008500000
0200: 0001------------------------------814d8b8ab9f8--
<= Recv header, 17 bytes (0x11)
0000: HTTP/1.1 200 OK


Response from server was the following:

00c9: --BOUNDARY
00d5: Content-type: application/x-ups-psmpld
00fd: Content-length: 104
0112:
0114: UPSOnLine%null%6932%6823PLD Internal Processing Error Encountere
0154: d.
017a:
017c: --BOUNDARY
0188: Content-type: application/x-ups-pld
01ad: Content-length: 110
01c2:
01c4: 00010000009710000008800036823PLD Internal Processing Error Encou
0204: ntered.
0233:
0235: --BOUNDARY--


Questions:
- To construct it ourselves do we need to url-encode every line, every
string.
- I have tried with url-encoding CR/LF at the end of each line as %0D%0A
but it didn't work.
- What else we need to do to construct it properly.

3. Sorry for the disclaimers, but I can not do nothing about them.

I obviously don't know much about cURL, even less about posting HTTP
Request
formatted as requested. Thanks for your replies and patience.

Jan

This electronic message contains information from Russell A. Farrow
Limited, Russell A. Farrow (U.S.) Inc., canadaplus.com or RAF Express
Inc., that may be confidential or privileged. The information in this
electronic message is only for the use of the intended recipient.
Inappropriate disclosure, copying, distribution or other use of the
contents of this electronic message is strictly prohibited. If you have
received this electronic transmission in error, please notify the sender
immediately by return e-mail. Thank you.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html

[Daniel Stenberg]

Post by J***@farrow.com
This will be a preferred solution for us (to construct the entire thing
and send it with -d).

--data-binary is probably better for you. You want the newlines etc preserved.

and you want to provide the proper Content-Type: header with -H too, which
then needs to include the correct boundary that you use in the file.

Post by J***@farrow.com
- To construct it ourselves do we need to url-encode every line, every
string.

No. Multipart formposts never URL-encode anything. The fact that you insist on
having one part url-encoded is a very rare thing. In fact I've not ever seen
it "live".

Post by J***@farrow.com
- I have tried with url-encoding CR/LF at the end of each line as %0D%0A
but it didn't work.

That's completely wrong, unless you want to actually pass on content that is
exactly the combination of bytes '%' '0' 'D' etc...

[J***@farrow.com]

Well, I tried with --data-binary, and it did preserved the new lines, but
it came back with response from the server 'Internal processing error'.

i also tried with
-H Content-Type: multipart/mixed;
boundary=------------------------------814d8b8ab9f8
and
-H "Content-Type: multipart/mixed;
boundary=------------------------------814d8b8ab9f8"

but got the same response 'Internal processing error'.

It is not that we insisting on formatting the request this way.
It is a vendor defined format. Every <cr/lf>, 'Content-length:' on
sub-parts, 2 sections of data, everything has to be exactly as they
specified.
Since the header section is fairly static (except first and last
'Content-length'), and because the Request message is too complex to
handle
it with cURL command line parameters, easiest solution for us will be if
we can construct the whole message within our application, and have
cURL pass it over to the server but it looks like that will not be
possible also.

If you can suggest anything else that might work, I am open for
suggestions.
Thanks again for all your replies.

Jan

This electronic message contains information from Russell A. Farrow
Limited, Russell A. Farrow (U.S.) Inc., canadaplus.com or RAF Express
Inc., that may be confidential or privileged. The information in this
electronic message is only for the use of the intended recipient.
Inappropriate disclosure, copying, distribution or other use of the
contents of this electronic message is strictly prohibited. If you have
received this electronic transmission in error, please notify the sender
immediately by return e-mail. Thank you.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html

[Daniel Stenberg]

Post by J***@farrow.com
i also tried with
-H Content-Type: multipart/mixed;
boundary=------------------------------814d8b8ab9f8
and
-H "Content-Type: multipart/mixed;
boundary=------------------------------814d8b8ab9f8"

You need the latter, the first version is major wrong.

Post by J***@farrow.com
It is not that we insisting on formatting the request this way.
It is a vendor defined format.

A stupidly defined vendor format too...

Post by J***@farrow.com
Every <cr/lf>, 'Content-length:' on
sub-parts, 2 sections of data, everything has to be exactly as they
specified.

Then you really need to handicraft your own.

Post by J***@farrow.com
easiest solution for us will be if we can construct the whole message within
our application, and have cURL pass it over to the server but it looks like
that will not be possible also.

I think it is possible. In fact I can't see why it wouldn't be possible.

Your problem is that you don't quite understand what you're doing and then
you're asking us when your server responds with errors instead of checking the
server logs or asking the guys with the server what those particular errors
mean. It is the server that's annoying here.

I would seriously advice you to just ask or hire someone who understands a
little bit about HTTP or has some curl experience to help you out. This really
isn't rocket science.