Discussion:
Feedback wanted: bold headers?
(too old to reply)
Daniel Stenberg
2018-04-27 22:26:16 UTC
Permalink
Hi,

I've put together a little proof of concept PR that shows how curl can output
HTTP headers using bold font style for the name parts when the output is sent
to the terminal. Personally I think it helps making the headers easier to
read.

What do you think? Is this a good idea to forward on, should I polish it
further? Do we need cutomizable colors/styles? Do we risk any backlash if we
enable this styled output by default?

I presume we need an option to all users to explicitly turn it on/off, but I
haven't made any such yet.

The PR is here:

https://github.com/curl/curl/pull/2538

A sample screenshot is here:

Loading Image...
--
/ daniel.haxx.se
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://
Jeremy Nicoll
2018-04-28 12:34:42 UTC
Permalink
Post by Daniel Stenberg
Hi,
I've put together a little proof of concept PR that shows how curl can output
HTTP headers using bold font style for the name parts when the output is sent
to the terminal. Personally I think it helps making the headers easier to
read.
I agree...
Post by Daniel Stenberg
I presume we need an option to all users to explicitly turn it on/off, but I
haven't made any such yet.
Assuming that the bold feature is acheived with control codes enclosing the
bold parts of the text, I think it would need to be possible to turn it off, for
users who pipe terminal output elsewhere for subsequent processing.

Are there any OSes whose terminals don't honour these control codes?

Presumably those who explicitly direct such output to named files wouldn't
see the control codes?
--
Jeremy Nicoll - my opinions are my own.
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se
Daniel Stenberg
2018-04-28 21:47:11 UTC
Permalink
Post by Jeremy Nicoll
Assuming that the bold feature is acheived with control codes enclosing the
bold parts of the text, I think it would need to be possible to turn it off,
for users who pipe terminal output elsewhere for subsequent processing.
Yes. But it already now disables itself if the output is not a terminal, just
so piping the output into something else shouldn't need any special flags.

I'm thinking about leaving the default to auto-sense and adding a
--no-styled-output flag that disables it unconditionally. My still considering
if --styled-output should enable bold headers unconditionally or just enable
the auto-sense mode...
Post by Jeremy Nicoll
Are there any OSes whose terminals don't honour these control codes?
The Windows command prompt doesn't apparently. But as was mentioned in the PR,
I'm curious if we should determine that in run-time rather than just disabling
styled output completely from the Windows executable...
--
/ daniel.haxx.se
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.h
s***@gmail.com
2018-04-28 22:10:55 UTC
Permalink
I think this is a phenomenal idea, but I'd personally like to see the ability to configure certain colours for various flags. For example, instead of just bold, if I could set RED for the header tags for spam or stuff like that, that'd be amazing! Or blue for people in my contacts list (obviously, I'd write the program to check the contacts list, etc.)

Just my two cents.

-----Original Message-----
From: curl-users <curl-users-***@cool.haxx.se> On Behalf Of Daniel Stenberg
Sent: Friday, April 27, 2018 6:26 PM
To: curl users <curl-***@cool.haxx.se>
Subject: Feedback wanted: bold headers?

Hi,

I've put together a little proof of concept PR that shows how curl can output HTTP headers using bold font style for the name parts when the output is sent to the terminal. Personally I think it helps making the headers easier to read.

What do you think? Is this a good idea to forward on, should I polish it further? Do we need cutomizable colors/styles? Do we risk any backlash if we enable this styled output by default?

I presume we need an option to all users to explicitly turn it on/off, but I haven't made any such yet.

The PR is here:

https://github.com/curl/curl/pull/2538

A sample screenshot is here:

https://daniel.haxx.se/media/bold-headers-terminal.png
--
/ daniel.haxx.se
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html


-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiq
Daniel Stenberg
2018-04-29 10:45:24 UTC
Permalink
Post by s***@gmail.com
I think this is a phenomenal idea, but I'd personally like to see the
ability to configure certain colours for various flags.
I was looking for inspiration in how 'ls' does this on Linux as it can make
specific files or extensions use specific colors and styles:

https://twistedcode.blogspot.se/2008/04/lscolors-explained.html
Post by s***@gmail.com
For example, instead of just bold, if I could set RED for the header tags
for spam or stuff like that, that'd be amazing! Or blue for people in my
contacts list (obviously, I'd write the program to check the contacts list,
etc.)
A possible syntax in similar vein could be to use a CURL_COLORS environment
variable (and command line option) and allow it to set a colon-separated list
of headers and what style/colors to use:

server=1;35:content-length=4;32:*=1

Makes "server" headers bold purple, "content-length" headers underlined green
and the rest plain bold.

But I'm not entirely convinced people actually want or would use such a
flexible approach...
--
/ daniel.haxx.se
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/e
s***@gmail.com
2018-04-29 18:18:22 UTC
Permalink
Daniel,

I'm with Timothe on this and don't think using the ANSI escape sequences are the way to set the colours. I think termcap should be used, personally, for portability sakes.

-----Original Message-----
From: curl-users <curl-users-***@cool.haxx.se> On Behalf Of Daniel Stenberg
Sent: Sunday, April 29, 2018 6:45 AM
To: the curl tool <curl-***@cool.haxx.se>
Subject: RE: Feedback wanted: bold headers?
Post by s***@gmail.com
I think this is a phenomenal idea, but I'd personally like to see the
ability to configure certain colours for various flags.
I was looking for inspiration in how 'ls' does this on Linux as it can make
specific files or extensions use specific colors and styles:

https://twistedcode.blogspot.se/2008/04/lscolors-explained.html
Post by s***@gmail.com
For example, instead of just bold, if I could set RED for the header tags
for spam or stuff like that, that'd be amazing! Or blue for people in my
contacts list (obviously, I'd write the program to check the contacts list,
etc.)
A possible syntax in similar vein could be to use a CURL_COLORS environment
variable (and command line option) and allow it to set a colon-separated list
of headers and what style/colors to use:

server=1;35:content-length=4;32:*=1

Makes "server" headers bold purple, "content-length" headers underlined green
and the rest plain bold.

But I'm not entirely convinced people actually want or would use such a
flexible approach...
--
/ daniel.haxx.se
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html


-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https
Georg Horn
2018-04-30 11:50:53 UTC
Permalink
Hello ,

i personally always turn colors of. I hate it if i install a new
linux system and the first ls -la turns the screen into an ocean of
blinking colors... ;-)

I personaly believe curl should concentrate on fetching URLs. The
presentation or further processing of the fetched data should be left to
B="`tput bold`"
b="`tput sgr0`"
2>&1 curl -v http://www.spamhaus.org/drop/dropv6.txt | \
sed -e"s/^< \([A-Z}[A-Za-z-]*:\)/< $B\1$b/" | less -R
But thats just my personal opinion. If the feature is switchable and is
preferably switched off by default i can live with it...

Kind Regards,
Georg
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquet
Rodrigo Zanatta Silva
2018-04-30 15:59:00 UTC
Permalink
Just to say my opinion:* I really love the idea. *

I don't see the problem if you use a non-color by default, but have an
option to enable it. Color, bold.. Everything to make it easier is better.

Maybe we should only debate one thing: The bold/color will be enabled by
default or not?

You need to put it.
Post by Georg Horn
Hello ,
i personally always turn colors of. I hate it if i install a new
linux system and the first ls -la turns the screen into an ocean of
blinking colors... ;-)
I personaly believe curl should concentrate on fetching URLs. The
presentation or further processing of the fetched data should be left to
B="`tput bold`"
b="`tput sgr0`"
2>&1 curl -v http://www.spamhaus.org/drop/dropv6.txt | \
sed -e"s/^< \([A-Z}[A-Za-z-]*:\)/< $B\1$b/" | less -R
But thats just my personal opinion. If the feature is switchable and is
preferably switched off by default i can live with it...
Kind Regards,
Georg
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Timothe Litt
2018-04-30 15:52:42 UTC
Permalink
Message: 1
Date: Sun, 29 Apr 2018 12:45:24 +0200 (CEST)
Subject: RE: Feedback wanted: bold headers?
Content-Type: text/plain; charset=US-ASCII; format=flowed
A possible syntax in similar vein could be to use a CURL_COLORS environment
variable (and command line option) and allow it to set a colon-separated list
server=1;35:content-length=4;32:*=1
Makes "server" headers bold purple, "content-length" headers underlined green
and the rest plain bold.
But I'm not entirely convinced people actually want or would use such a
flexible approach...
This is what I expected; the requirement never stays "just bold".
The colors supported by terminals vary - as do the colors mapped to the
color numbers.
People with visual impairments (e.g. color blindness) have issues - and
modify termcap.
So, again, if you do this, use termcap; it has solved these problems for
many years.
Don't do your own thing.  It would be like me writing my own utility to
fetch http resources
because "curl is too heavyweight".  Wouldn't be long before it looked a
lot like curl :-)
------------------------------ Message: 4 Date: Sun, 29 Apr 2018
text/plain; charset="utf-8"; Format="flowed" On Sun, 29 Apr 2018,
Use termcap.  Or (n)curses.  Don't hardcode even the ANSI bold controls.
That would be taking things (much) further than I'd be comfortable with.
Then don't do it at all.  Termcap is easy to use; there are libraries to
fetch the right control
sequence for whatever terminal (as may be modified by user
capabilities).  Hardcoding your
own idea of beauty and/or terminal/user capabilities is something that
you (or your users)
will regret. 

If you mean that you don't want to do a fancy ncurses windowed gui at
this time, I'm OK
with that.  Though scrolling regions for client and server headers would
be much more
useful than just bolding header names.  As I noted, these days the list
of headers can
run into the dozens, and when debugging one often would like to compare
request with
response.  E.g. If-Modified-Since with Last-Modified.  My current
approach is to copy the
whole trace into an emacs buffer, split the window horizontally, and
scroll as needed.
Note that this discards your bolding... and that an emacs face can
easily highlight header names.
(Or vim, or whatever editor you like.)
Make sure that you don't break copy and paste.
How would I break or not break that?
A paste buffer may (depends on OS) hold multiple formats: graphics,
plain text, rich text, html, ...
If you enhance text, the terminal may store what you copy differently
(or without plain text).
Since, as noted above, today's use model is often "do something, copy
screen, diagnose", you
need to make sure that the new behavior doesn't break it.

If you default to no enhancements, you should be OK.
If you default to enhancements on a terminal, you may not.  E.g. if bold
triggers storing only
rich text in the paste buffer.  That's a per-terminal emulator behavior.

As a user, I don't want to be doubly surprised: why are the header names
in bold?  and why
doesn't copy & paste work anymore?
2>&1 curl -v http://www.spamhaus.org/drop/dropv6.txt | \
sed -e's/^< \([A-Z}[A-Za-z-]*:\)/< \x1b[1m\1\x1b[m/' | less -R
Of course that breaks in 22 different ways when you add other factors in. But
sure, that works for the simple use case.
I didn't propose it as an engineered solution; merely to show that one
can have your
enhancement without modifying curl.  A fully engineered wrapper would
be a small project.  And would use termcap.
I'd find that to be a too huge penalty for just doing bold escape codes.
What's "huge" about retrieving the 'bold' and 'end bold' codes from termcap?
Inside the library, it's just a hash lookup that happens once per run of
curl. The
library and termcap file are frequently used; they are likely to be in
the filesystem
cache(s).

If you mean running tput twice - well, it's a shell script - that's the
API; shell scripts
do lots of process activations.  (The second is likely to be cached.)
But compared
to the network traffic, who cares?  It's a few milliseconds.  If you do
the work in
curl, you'll be in the microseconds range - a small price to pay for
portability.

As another alternative, how about providing an option for curl to write
a pcap file?
e.g., it might open the connection, then spawn a tcpdump command with a
precise filter
covering just that connection.
Or call libpcap itself.  Or generate a synthetic trace with the data in
plaintext.

With a capture file in hand, wireshark provides a GUI with all the fancy
colorizing options that you
might want.  And so do many other network analysis tools.

Such a capture would also be useful in diagnosing transfer issues at all
levels of the stack.  wireshark dissectors will link requests and
responses & provide multiple levels of detail.

Why have curl help? 
using wireshark directly is a pain.  You don't know the port numbers in
advance;
you need to deal with two different filter syntaxes ; you often end up
capturing a huge
amount of extraneous data.  And you need to use multiple tools
simultaneously.
(How often have I tried to click "start" on wireshark just before "<CR>"
on an application
window?  Lots.)
Plus, with any encrypted protocol, you need private keys.  (Good luck
with that.) 
curl has the cleartext, so could turn https into http (just generate a
fake TCp stream
trace).  And a synthetic trace doesn't need privileges.

Yes, I know about curl's trace files - which are very useful.  But a
.pcap file can be consumed
by wireshark (and other tools) - which allows enhanced presentation to
be their problem, not
curl's.

If you have time to invest, this seems to address a more general problem
and produce a much more flexible result with a bounded investment of
time. In any case, curl is your tool & you'll do what you wish. You have
the feedback that you asked for. I hope it helps.

Timothe Litt
ACM Distinguished Engineer
--------------------------
This communication may not represent the ACM or my employer's views,
if any, on the matters discussed.

 
Bill Mercer
2018-04-30 19:48:21 UTC
Permalink
As a non-developer/ command line user, I like the bolded headers for easier reading, but I'm a little concerned about the possibility of issues with non-standard terminals. I also like the idea of separating content from presentation.

-----Original Message-----
From: curl-users [mailto:curl-users-***@cool.haxx.se] On Behalf Of Daniel Stenberg
Sent: Friday, April 27, 2018 18:26 PM
To: curl users <curl-***@cool.haxx.se>
Subject: Feedback wanted: bold headers?

Hi,

I've put together a little proof of concept PR that shows how curl can output HTTP headers using bold font style for the name parts when the output is sent to the terminal. Personally I think it helps making the headers easier to read.

What do you think? Is this a good idea to forward on, should I polish it further? Do we need cutomizable colors/styles? Do we risk any backlash if we enable this styled output by default?

I presume we need an option to all users to explicitly turn it on/off, but I haven't made any such yet.

The PR is here:

https://github.com/curl/curl/pull/2538

A sample screenshot is here:

https://daniel.haxx.se/media/bold-headers-terminal.png
--
/ daniel.haxx.se
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette
Loading...