Read ~/.netrc only once

[Morten Kjærulff via curl-users]

Hi,

If http://example.com needs my userid+password, this:
curl -n --anyauth http://example.com
will read my ~/.netrc to find my userid+password.
This will do the same:
curl -n --anyauth --netrc-file ~/.netrc http://example.com

This one, hangs:
curl -n --anyauth --netrc-file <( cat ~/.netrc) http://example.com
I assume this is because curl reads the file more than once.

It would be cool if curl would read the ~/.netrc / --netrc-file only once.
I could then replace "cat ~/.netrc" with something more secure.

Or can I do it another way?

/Morten
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquet

[Daniel Stenberg via curl-users]

Post by Morten Kjærulff via curl-users
curl -n --anyauth --netrc-file ~/.netrc http://example.com
I assume this is because curl reads the file more than once.

When I try that command line and set a break-point in Curl_parsenetrc() I
only see it called once.

[Morten Kjærulff via curl-users]

Post by Daniel Stenberg via curl-users
When I try that command line and set a break-point in Curl_parsenetrc() I
only see it called once.

I am on cygwin.
http://mysite/... is an internal sharepoint site.

When I do
strace curl -n --anyauth --netrc-file ~/.netrc http://mysite/... |
grep \.netrc
It seems the file is opened 3 times:
46 185075 [main] curl 8213 build_argv: argv[3] = '--netrc-file'
55 185130 [main] curl 8213 build_argv: argv[4] = '/home/vp01mkf/.netrc'
1311 747547 [main] curl 8213 open: open(/home/vp01mkf/.netrc, 0x20000)
48 747595 [main] curl 8213 normalize_posix_path: src /home/vp01mkf/.netrc
47 747642 [main] curl 8213 normalize_posix_path:
/home/vp01mkf/.netrc = normalize_posix_path (/home/vp01mkf/.netrc)
50 747692 [main] curl 8213 mount_info::conv_to_win32_path:
conv_to_win32_path (/home/vp01mkf/.netrc)
49 747741 [main] curl 8213 mount_info::conv_to_win32_path:
src_path /home/vp01mkf/.netrc, dst C:\mhome\.netrc, flags 0x2008, rc 0
148 747889 [main] curl 8213 symlink_info::check: 0x0 = NtCreateFile
(\??\C:\mhome\.netrc)
41 748000 [main] curl 8213 symlink_info::check: 0 =
symlink.check(C:\mhome\.netrc, 0xFFFF9E30) (mount_flags 0x2008,
path_flags 0x0)
53 748053 [main] curl 8213 path_conv::check:
this->path(C:\mhome\.netrc), has_acls(1)
98 748205 [main] curl 8213 fhandler_base::open:
(\??\C:\mhome\.netrc, 0x28000)
48 748465 [main] curl 8213 fhandler_base::open: 0x0 = NtCreateFile
(0x328, 0x80100000, \??\C:\mhome\.netrc, io, NULL, 0x0, 0x7, 0x1,
0x4020, NULL, 0)
48 748513 [main] curl 8213 fhandler_base::open: 1 =
fhandler_base::open(\??\C:\mhome\.netrc, 0x28000)
56 748569 [main] curl 8213 fhandler_base::open_fs: 1 =
fhandler_disk_file::open(\??\C:\mhome\.netrc, 0x28000)
63 748632 [main] curl 8213 open: 5 = open(/home/vp01mkf/.netrc, 0x28000)
373 749248 [main] curl 8213 fhandler_base::fstat_helper: 0 = fstat
(\??\C:\mhome\.netrc, 0xFFFFB060) st_size=432, st_mode=0100644,
st_ino=12666373951983468st_atim=60AABD41.1F6A3DAC
st_ctim=60A65D74.1BC6C0BC st_mtim=60A654B7.4FC71EC
st_birthtim=60A654B7.4ED2FAC
47 749551 [main] curl 8213 _setmode: (5<\??\C:\mhome\.netrc>,
0x10000) returning text
47 749945 [main] curl 8213 _setmode: (5<\??\C:\mhome\.netrc>,
0x20000) returning binary
49 751659 [main] curl 8213 fhandler_base::close: closing
'/home/vp01mkf/.netrc' handle 0x328
2955 3289168 [main] curl 8213 open: open(/home/vp01mkf/.netrc, 0x20000)
51 3289219 [main] curl 8213 normalize_posix_path: src /home/vp01mkf/.netrc
52 3289271 [main] curl 8213 normalize_posix_path:
/home/vp01mkf/.netrc = normalize_posix_path (/home/vp01mkf/.netrc)
63 3289334 [main] curl 8213 mount_info::conv_to_win32_path:
conv_to_win32_path (/home/vp01mkf/.netrc)
46 3289380 [main] curl 8213 mount_info::conv_to_win32_path:
src_path /home/vp01mkf/.netrc, dst C:\mhome\.netrc, flags 0x2008, rc 0
183 3289563 [main] curl 8213 symlink_info::check: 0x0 = NtCreateFile
(\??\C:\mhome\.netrc)
53 3289696 [main] curl 8213 symlink_info::check: 0 =
symlink.check(C:\mhome\.netrc, 0xFFFF9E30) (mount_flags 0x2008,
path_flags 0x0)
50 3289746 [main] curl 8213 path_conv::check:
this->path(C:\mhome\.netrc), has_acls(1)
98 3289913 [main] curl 8213 fhandler_base::open:
(\??\C:\mhome\.netrc, 0x28000)
40 3290183 [main] curl 8213 fhandler_base::open: 0x0 = NtCreateFile
(0x354, 0x80100000, \??\C:\mhome\.netrc, io, NULL, 0x0, 0x7, 0x1,
0x4020, NULL, 0)
41 3290224 [main] curl 8213 fhandler_base::open: 1 =
fhandler_base::open(\??\C:\mhome\.netrc, 0x28000)
54 3290278 [main] curl 8213 fhandler_base::open_fs: 1 =
fhandler_disk_file::open(\??\C:\mhome\.netrc, 0x28000)
59 3290337 [main] curl 8213 open: 6 = open(/home/vp01mkf/.netrc, 0x28000)
386 3290962 [main] curl 8213 fhandler_base::fstat_helper: 0 = fstat
(\??\C:\mhome\.netrc, 0xFFFFB060) st_size=432, st_mode=0100644,
st_ino=12666373951983468st_atim=60AABDC4.36768140
st_ctim=60A65D74.1BC6C0BC st_mtim=60A654B7.4FC71EC
st_birthtim=60A654B7.4ED2FAC
58 3291302 [main] curl 8213 _setmode: (6<\??\C:\mhome\.netrc>,
0x10000) returning text
53 3291698 [main] curl 8213 _setmode: (6<\??\C:\mhome\.netrc>,
0x20000) returning binary
56 3295269 [main] curl 8213 fhandler_base::close: closing
'/home/vp01mkf/.netrc' handle 0x354
2535 3404917 [main] curl 8213 open: open(/home/vp01mkf/.netrc, 0x20000)
47 3404964 [main] curl 8213 normalize_posix_path: src /home/vp01mkf/.netrc
52 3405016 [main] curl 8213 normalize_posix_path:
/home/vp01mkf/.netrc = normalize_posix_path (/home/vp01mkf/.netrc)
57 3405073 [main] curl 8213 mount_info::conv_to_win32_path:
conv_to_win32_path (/home/vp01mkf/.netrc)
57 3405130 [main] curl 8213 mount_info::conv_to_win32_path:
src_path /home/vp01mkf/.netrc, dst C:\mhome\.netrc, flags 0x2008, rc 0
159 3405289 [main] curl 8213 symlink_info::check: 0x0 = NtCreateFile
(\??\C:\mhome\.netrc)
45 3405411 [main] curl 8213 symlink_info::check: 0 =
symlink.check(C:\mhome\.netrc, 0xFFFF9E30) (mount_flags 0x2008,
path_flags 0x0)
53 3405464 [main] curl 8213 path_conv::check:
this->path(C:\mhome\.netrc), has_acls(1)
96 3405618 [main] curl 8213 fhandler_base::open:
(\??\C:\mhome\.netrc, 0x28000)
125 3407242 [main] curl 8213 fhandler_base::open: 0x0 = NtCreateFile
(0x1E8, 0x80100000, \??\C:\mhome\.netrc, io, NULL, 0x0, 0x7, 0x1,
0x4020, NULL, 0)
85 3407327 [main] curl 8213 fhandler_base::open: 1 =
fhandler_base::open(\??\C:\mhome\.netrc, 0x28000)
80 3407407 [main] curl 8213 fhandler_base::open_fs: 1 =
fhandler_disk_file::open(\??\C:\mhome\.netrc, 0x28000)
69 3407476 [main] curl 8213 open: 6 = open(/home/vp01mkf/.netrc, 0x28000)
478 3408193 [main] curl 8213 fhandler_base::fstat_helper: 0 = fstat
(\??\C:\mhome\.netrc, 0xFFFFB060) st_size=432, st_mode=0100644,
st_ino=12666373951983468st_atim=60AABDC7.1A5D3418
st_ctim=60A65D74.1BC6C0BC st_mtim=60A654B7.4FC71EC
st_birthtim=60A654B7.4ED2FAC
62 3408589 [main] curl 8213 _setmode: (6<\??\C:\mhome\.netrc>,
0x10000) returning text
69 3409064 [main] curl 8213 _setmode: (6<\??\C:\mhome\.netrc>,
0x20000) returning binary
94 3421740 [main] curl 8213 fhandler_base::close: closing
'/home/vp01mkf/.netrc' handle 0x1E8


When I do
strace curl -n --ntlm --netrc-file ~/.netrc http://mysite/... | grep \.netrc
(--ntlm instead of --anyauth)
It seems the file is opened 2 times:
50 194388 [main] curl 8220 build_argv: argv[3] = '--netrc-file'
54 194442 [main] curl 8220 build_argv: argv[4] = '/home/vp01mkf/.netrc'
1262 722561 [main] curl 8220 open: open(/home/vp01mkf/.netrc, 0x20000)
51 722612 [main] curl 8220 normalize_posix_path: src /home/vp01mkf/.netrc
42 722654 [main] curl 8220 normalize_posix_path:
/home/vp01mkf/.netrc = normalize_posix_path (/home/vp01mkf/.netrc)
47 722701 [main] curl 8220 mount_info::conv_to_win32_path:
conv_to_win32_path (/home/vp01mkf/.netrc)
55 722756 [main] curl 8220 mount_info::conv_to_win32_path:
src_path /home/vp01mkf/.netrc, dst C:\mhome\.netrc, flags 0x2008, rc 0
169 722925 [main] curl 8220 symlink_info::check: 0x0 = NtCreateFile
(\??\C:\mhome\.netrc)
48 723051 [main] curl 8220 symlink_info::check: 0 =
symlink.check(C:\mhome\.netrc, 0xFFFF9E30) (mount_flags 0x2008,
path_flags 0x0)
45 723096 [main] curl 8220 path_conv::check:
this->path(C:\mhome\.netrc), has_acls(1)
95 723242 [main] curl 8220 fhandler_base::open:
(\??\C:\mhome\.netrc, 0x28000)
53 723546 [main] curl 8220 fhandler_base::open: 0x0 = NtCreateFile
(0x348, 0x80100000, \??\C:\mhome\.netrc, io, NULL, 0x0, 0x7, 0x1,
0x4020, NULL, 0)
50 723596 [main] curl 8220 fhandler_base::open: 1 =
fhandler_base::open(\??\C:\mhome\.netrc, 0x28000)
45 723641 [main] curl 8220 fhandler_base::open_fs: 1 =
fhandler_disk_file::open(\??\C:\mhome\.netrc, 0x28000)
43 723684 [main] curl 8220 open: 5 = open(/home/vp01mkf/.netrc, 0x28000)
367 724266 [main] curl 8220 fhandler_base::fstat_helper: 0 = fstat
(\??\C:\mhome\.netrc, 0xFFFFB060) st_size=432, st_mode=0100644,
st_ino=12666373951983468st_atim=60AABDC7.2222DBA8
st_ctim=60A65D74.1BC6C0BC st_mtim=60A654B7.4FC71EC
st_birthtim=60A654B7.4ED2FAC
41 724594 [main] curl 8220 _setmode: (5<\??\C:\mhome\.netrc>,
0x10000) returning text
83 724957 [main] curl 8220 _setmode: (5<\??\C:\mhome\.netrc>,
0x20000) returning binary
48 726770 [main] curl 8220 fhandler_base::close: closing
'/home/vp01mkf/.netrc' handle 0x348
2613 3199265 [main] curl 8220 open: open(/home/vp01mkf/.netrc, 0x20000)
47 3199312 [main] curl 8220 normalize_posix_path: src /home/vp01mkf/.netrc
52 3199364 [main] curl 8220 normalize_posix_path:
/home/vp01mkf/.netrc = normalize_posix_path (/home/vp01mkf/.netrc)
50 3199414 [main] curl 8220 mount_info::conv_to_win32_path:
conv_to_win32_path (/home/vp01mkf/.netrc)
58 3199472 [main] curl 8220 mount_info::conv_to_win32_path:
src_path /home/vp01mkf/.netrc, dst C:\mhome\.netrc, flags 0x2008, rc 0
160 3199632 [main] curl 8220 symlink_info::check: 0x0 = NtCreateFile
(\??\C:\mhome\.netrc)
51 3199762 [main] curl 8220 symlink_info::check: 0 =
symlink.check(C:\mhome\.netrc, 0xFFFF9E30) (mount_flags 0x2008,
path_flags 0x0)
54 3199816 [main] curl 8220 path_conv::check:
this->path(C:\mhome\.netrc), has_acls(1)
102 3199982 [main] curl 8220 fhandler_base::open:
(\??\C:\mhome\.netrc, 0x28000)
55 3200277 [main] curl 8220 fhandler_base::open: 0x0 = NtCreateFile
(0x308, 0x80100000, \??\C:\mhome\.netrc, io, NULL, 0x0, 0x7, 0x1,
0x4020, NULL, 0)
55 3200332 [main] curl 8220 fhandler_base::open: 1 =
fhandler_base::open(\??\C:\mhome\.netrc, 0x28000)
70 3200402 [main] curl 8220 fhandler_base::open_fs: 1 =
fhandler_disk_file::open(\??\C:\mhome\.netrc, 0x28000)
48 3200450 [main] curl 8220 open: 6 = open(/home/vp01mkf/.netrc, 0x28000)
430 3201104 [main] curl 8220 fhandler_base::fstat_helper: 0 = fstat
(\??\C:\mhome\.netrc, 0xFFFFB060) st_size=432, st_mode=0100644,
st_ino=12666373951983468st_atim=60AABDE1.22F5E318
st_ctim=60A65D74.1BC6C0BC st_mtim=60A654B7.4FC71EC
st_birthtim=60A654B7.4ED2FAC
60 3201405 [main] curl 8220 _setmode: (6<\??\C:\mhome\.netrc>,
0x10000) returning text
43 3201781 [main] curl 8220 _setmode: (6<\??\C:\mhome\.netrc>,
0x20000) returning binary
48 3203554 [main] curl 8220 fhandler_base::close: closing
'/home/vp01mkf/.netrc' handle 0x308

/Morten
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette:

[Daniel Stenberg via curl-users]

Post by Morten Kjærulff via curl-users
strace curl -n --ntlm --netrc-file ~/.netrc http://mysite/... | grep \.netrc
(--ntlm instead of --anyauth)

It will be opened and read once per connection, so I presume your auth use
there forces curl to create more than one connection to perform the transfer.

[Morten Kjærulff via curl-users]

Post by Daniel Stenberg via curl-users
It will be opened and read once per connection, so I presume your auth use
there forces curl to create more than one connection to perform the transfer.

This seems to work:

mkfifo tmppip
while true ; do cat .netrc > tmppip ; done &
curl -n --anyauth --netrc-file tmppip http://mysite/...

But it seems a bit stupid.

Put it on the wishlist that I wish curl would read .netrc only once
per invocation.

/Morten
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https

[Daniel Stenberg via curl-users]

Post by Morten Kjærulff via curl-users
mkfifo tmppip
while true ; do cat .netrc > tmppip ; done &
curl -n --anyauth --netrc-file tmppip http://mysite/...
But it seems a bit stupid.

You could also provide the credentials using other means than netrc!

Post by Morten Kjærulff via curl-users
Put it on the wishlist that I wish curl would read .netrc only once per
invocation.

Yeps, the TODO list was getting a little thin... =)

[Morten Kjærulff via curl-users]

Post by Daniel Stenberg via curl-users
You could also provide the credentials using other means than netrc!

Yes, I could use -u --user, but `man curl`says "Such sensitive data
should be retrieved from a file instead or similar and never used in
clear text in a command line."

So back to .netrc or --netrc-file. But this might end up in backups or
could be seen by sysadmins.

Any other options?

That's why I wish --netrc-file <(cat ~/.netrc), but instead of cat
~/.netrc I would use
gpg --decrypt ...
or
echo "cat A.RACF.PROTECTED.DATASET" | ssh MY.GOOD.OLD.ZOS.HOST
Which I *guess*/*feel* is more safe.

/Morten
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiq