Basic CURL commands

List HTTP methods:

curl -i -X OPTIONS http://10.10.10.57
-i flag to include protocol response headers in the output. Screenshot 2018-03-30 14.38.02

Grab HTTP Server Banners:

curl --head http://10.10.10.63
--head flag to only grab headers of the site. Screenshot 2018-03-30 14.33.17

Grab robots.txt file:

curl https://www.bbc.co.uk/robots.txt
Screenshot 2018-03-30 14.42.12
If you want to include the headers in this request, apply the -i flag. (Shown in screenshot above.)