User-Agent is a header that identifies the client making a request to a web server. It normally contains information about the client's operating system, browser, and other details. cURL, by default, uses its name and version as the User-Agent string.

Web servers and applications often deliver content based on the User-Agent. For instance, a website might serve a mobile-optimized version of its content when it detects requests coming from smartphone browsers. Some websites might block requests from bots and crawlers by checking the User-Agent string, which oftentimes include cURL.

Changing the User-Agent string in cURL allows developers and testers to mimic requests from various clients. This is useful for debugging and website behavior analysis. It could also be used to bypass restrictions based on the User-Agent.

Related: How to add custom headers to cURL request

Steps to modify the cURL User-Agent string:

  1. Open the terminal.
  2. Make an HTTP request with cURL to a URL. 

    $ curl  http://www.example.com -v 2>&1 | grep '^>'
> GET / HTTP/1.1
> Host: www.example.com
> User-Agent: curl/8.1.2
> Accept: */*
>

    Use the -v or –verbose parameter to display detailed information about the request and the server's response, and 2>&1 | grep '^>' parameter to filter the output. These are for demonstration purposes only and are not required for normal cURL requests.

  3. Modify the User-Agent to imitate a Mozilla Firefox browser request. 

    $ curl -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"  http://www.example.com -v  2>&1 | grep '^>'
> GET / HTTP/1.1
> Host: www.example.com
> User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
> Accept: */*
>

    Related: List of Browser User Agents

  4. Use the longer –user-agent option to achieve the same result. 

    $ curl --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"  http://www.example.com -v  2>&1 | grep '^>'
> GET / HTTP/1.1
> Host: www.example.com
> User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
> Accept: */*
>
  5. Manually set User-Agent header to imitate a Mozilla Firefox browser request. 

     $ curl -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"  http://www.example.com -v  2>&1 | grep '^>'
> GET / HTTP/1.1
> Host: www.example.com
> Accept: */*
> User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
>

    Related: How to add custom headers to cURL request

  6. Check server's response for content or headers that may vary based on User-Agent. 

    $ curl -I www.example.com
HTTP/1.1 200 OK
Server: nginx
Date: Sat, 30 Sep 2023 04:28:05 GMT
Content-Type: text/html
Content-Length: 1991
Connection: keep-alive
Keep-Alive: timeout=20
Vary: Accept-Encoding
Last-Modified: Thu, 26 Aug 2021 07:13:07 GMT
ETag: "61273f03-7c7"
Accept-Ranges: bytes

    The -I flag sends a HEAD request, which will only return headers, allowing you to observe possible variations in server responses based on User-Agent.

  7. Add User-Agent option to cURL's config file to change the User-Agent permanently. 

     $ echo 'user-agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"' >> ~/.curlrc

    Encapsulate the User-Agent string in double quotes and the entire command in single quotes.

  8. Make a request to a website without specifying the User-Agent. 

    $ curl  http://www.example.com -v  2>&1 | grep '^>'
> GET / HTTP/1.1
> Host: www.example.com
> User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0
> Accept: */*
>