Get specified site browser like do

[callem]

Hi,
im trying to get this site https://www.raks.pl/aktualne-wersje by curl, but it show no content. I've tried to send special headers, etc., no success.
I need to get software version info ("Instalator - wersja" string) for my bash script, that informs me about new version release.
Thanks for any help.
Best regards
Callem

[Daniel Stenberg]

Post by callem
im trying to get this site https://www.raks.pl/aktualne-wersje by curl, but
it show no content. I've tried to send special headers, etc., no success.

I get lots of content when I try. Lots of javascript.

[Nick Chambers]

im trying to get this site https://www.raks.pl/aktualne-wersje by curl, but
it show no content. I've tried to send special headers, etc., no success.

I get lots of content when I try. Lots of javascript.

--
/ daniel.haxx.se


You can also export a cURL request that is equivalent to the request the browser makes, in some browsers.
https://lornajane.net/posts/2013/chrome-feature-copy-as-curl



-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquett

[callem]

Yes, but that content is much less than in normal browser, like chrome.

Browser gets text like "wersja", curl gets only tose java scripts, and I dont know why and how to get full content of this site. Is it at all possible ?

Post by Daniel Stenberg

im trying to get this site https://www.raks.pl/aktualne-wersje by curl, but
it show no content. I've tried to send special headers, etc., no success.

I get lots of content when I try. Lots of javascript.
--
/ daniel.haxx.se-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette

[Daniel Stenberg]

Post by callem
Yes, but that content is much less than in normal browser, like chrome.
Browser gets text like "wersja", curl gets only tose java scripts, and I
dont know why and how to get full content of this site. Is it at all
possible ?

(Please don't top-post, now we lost the context.)

Browsers don't get more content for that URL but browsers will *execute* that
javascript and that will then produce what you see, potentially by doing more
requests and whatever. Modern web sites are often not visible unless you run
the javascript.

[callem]

Post by Nick Chambers
You can also export a cURL request that is equivalent to the request the browser makes, in some browsers.
https://lornajane.net/posts/2013/chrome-feature-copy-as-curl

It doesnt work eather.

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.h

[Ray Satiro via curl-users]

Post by callem

Post by Nick Chambers
You can also export a cURL request that is equivalent to the request the browser makes, in some browsers.
https://lornajane.net/posts/2013/chrome-feature-copy-as-curl

It doesnt work eather.

You can use a debugging proxy like fiddler to monitor the traffic and
see what the javasript is retrieving. In this case it's retrieving a
json file with the version information. You would then pass that to jq
to process it. For example let's say you want the first array item in
wersjedos and that version information, it would look like this:

curl --proto =https -fLsS https://www.raks.pl/assets/wersje.json | jq
--raw-output ".wersjedos | .[0] | .numer | select(type == \"string\")"

To see how to use curl with jq see https://stedolan.github.io/jq/tutorial/

jq questions are out of the scope of this mailing list, you will have to
ask a jq support channel (?) for questions if you need help parsing the
json.

[callem]

Post by Daniel Stenberg
Browsers don't get more content for that URL but browsers will *execute* that
javascript and that will then produce what you see, potentially by doing more
requests and whatever. Modern web sites are often not visible unless you run
the javascript.

So is it possible to run that js with curl and get that content ? Or any other method for linux console ?

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users

[callem]

curl --proto =https -fLsS https://www.raks.pl/assets/wersje.json | jq --raw-output ".wersjedos | .[0] | .numer | select(type == \"string\")"
To see how to use curl with jq see https://stedolan.github.io/jq/tutorial/
jq questions are out of the scope of this mailing list, you will have to ask a jq support channel (?) for questions if you need help parsing the json.

Thanks !

I was going to dig with htmlUnit or phantomJS, but now this is all I need.

-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etique