Can PHP cURL retrieve response headers AND body in a single request?

php http curl

Is there any way to get both headers and body for a cURL request using PHP? I found that this option:

curl_setopt($ch, CURLOPT_HEADER, true);

is going to return the body plus headers, but then I need to parse it to get the body. Is there any way to get both in a more usable (and secure) way?

Note that for "single request" I mean avoiding issuing a HEAD request prior of GET/POST.

There is a built in solution for this, see this answer: stackoverflow.com/a/25118032/1334485 (added this comment 'coz this post still gets many views)

Look at this nice comment: secure.php.net/manual/en/book.curl.php#117138

a-curl_multi-working-example---that-actually-works

I was told my question was a duplicate to this question. If it is not a duplicate can someone please reopen it? stackoverflow.com/questions/43770246/… In my question I have a concrete requirement to use a method that returns an object with headers and body separate and not one string.

laurent

One solution to this was posted in the PHP documentation comments: http://www.php.net/manual/en/function.curl-exec.php#80442

Code example:

$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
// ...

$response = curl_exec($ch);

// Then, after your curl_exec call:
$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$header = substr($response, 0, $header_size);
$body = substr($response, $header_size);

Warning: As noted in the comments below, this may not be reliable when used with proxy servers or when handling certain types of redirects. @Geoffrey's answer may handle these more reliably.

You can also list($header, $body) = explode("\r\n\r\n", $response, 2), but this might take a bit longer, depending on your request size.

this is bad solution because if you use proxy server and your proxy server(fiddler for example) add own headers to response - this headers broke all offsets and you should use list($header, $body) = explode("\r\n\r\n", $response, 2) as only working variant

@msangel Your solution doesn't work when there are multiple headers in the response, such as when the server does a 302 redirect. Any suggestions?

@Nate, yes, i know this. AFAIK, but there is only one possible additional header - with code 100 (Continue). For this header you can go around with correctly defining request option: curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:')); , disabling sending this header response. As for 302, this should not be happened, because 302 header is redirect, it not expecting body, however i know, sometimes servers send some body with 302 response, but it will be anyway ignored by browsers, so far, why curl should handle this?)

CURLOPT_VERBOSE is intended to output process information to STDERR (may bother in CLI) and for discussed problem is useless.

hakre

Many of the other solutions offered this thread are not doing this correctly.

Splitting on \r\n\r\n is not reliable when CURLOPT_FOLLOWLOCATION is on or when the server responds with a 100 code RFC-7231, MDN.

Not all servers are standards compliant and transmit just a \n for new lines (and a recipient may discard the \r in the line terminator) Q&A.

Detecting the size of the headers via CURLINFO_HEADER_SIZE is also not always reliable, especially when proxies are used Curl-1204 or in some of the same redirection scenarios.

The most correct method is using CURLOPT_HEADERFUNCTION.

Here is a very clean method of performing this using PHP closures. It also converts all headers to lowercase for consistent handling across servers and HTTP versions.

This version will retain duplicated headers

This complies with RFC822 and RFC2616, please do not make use of the mb_ (and similar) string functions, it is a not only incorrect but even a security issue^RFC-7230!

$ch = curl_init();
$headers = [];
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

// this function is called by curl for each header received
curl_setopt($ch, CURLOPT_HEADERFUNCTION,
  function($curl, $header) use (&$headers)
  {
    $len = strlen($header);
    $header = explode(':', $header, 2);
    if (count($header) < 2) // ignore invalid headers
      return $len;

    $headers[strtolower(trim($header[0]))][] = trim($header[1]);
    
    return $len;
  }
);

$data = curl_exec($ch);
print_r($headers);

IMO this is the best answer in this thread and fixes problems with redirects that occurred with other answers. Best to read the documentation for CURLOPT_HEADERFUNCTION to understand how it works and potential gotchas. I've also made some improvements to the answer to help others out.

Great, I have updated the answer to cater for duplicated headers. In future do not re-format the code to what you believe it should be. This is written in a way to make it clear where the closure function boundaries are.

@thealexbaron Yes it is as of PHP 5.4, see: php.net/manual/en/migration54.new-features.php

This answer is highly underrated for such a neat and RFC compliant approach. This should be made sticky answer and moved to the top. I just wish there were a faster approach to get value of a desired header instead of parsing all headers first.

@Mahesh.D no, $data = curl_exec($ch); returns the content when CURLOPT_RETURNTRANSFER is set as per the provided example.

hakre

Curl has a built in option for this, called CURLOPT_HEADERFUNCTION. The value of this option must be the name of a callback function. Curl will pass the header (and the header only!) to this callback function, line-by-line (so the function will be called for each header line, starting from the top of the header section). Your callback function then can do anything with it (and must return the number of bytes of the given line). Here is a tested working code:

function HandleHeaderLine( $curl, $header_line ) {
    echo "<br>YEAH: ".$header_line; // or do whatever
    return strlen($header_line);
}


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.google.com");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADERFUNCTION, "HandleHeaderLine");
$body = curl_exec($ch);

The above works with everything, different protocols and proxies too, and you dont need to worry about the header size, or set lots of different curl options.

P.S.: To handle the header lines with an object method, do this:

curl_setopt($ch, CURLOPT_HEADERFUNCTION, array($object, 'methodName'))

As a note, the callback function is called for each header and it seems they are not trimmed. You can use a global variable to hold all headers or you can use an anonymous function for the callback and use a local variable (local for the parent scope, not the anonymous function).

@MV Thanks, yes, by "line-by-line" I meant "each header". I edited my answer for clarity. To get the entire header section (aka. all headers), you can also use an object method for the callback so an object property can hold all of them.

This is the best answer IMO. It doesn't cause problems with multiple "\r\n\r\n" when using CURLOPT_FOLLOWLOCATION and I guess it won't be affected by additional headers from proxies.

Worked very well for me, also see stackoverflow.com/questions/6482068/… in case of issues

Yes, this is the best approach however @Geoffrey's answer makes this cleaner by using an anonymous function with no need for global variables and such.

papsy

is this what are you looking to?

curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));
$response = curl_exec($ch); 
list($header, $body) = explode("\r\n\r\n", $response, 2);

This works normally except when there's a HTTP/1.1 100 Continue followed by a break then HTTP/1.1 200 OK. I'd go with the other method.

Take a look to the selected answer of stackoverflow.com/questions/14459704/… before implementing something like this. w3.org/Protocols/rfc2616/rfc2616-sec14.html (14.20)

A server that does not understand or is unable to comply with any of the expectation values in the Expect field of a request MUST respond with appropriate error status. The server MUST respond with a 417 (Expectation Failed) status if any of the expectations cannot be met or, if there are other problems with the request, some other 4xx status.

@ghostfly, i have the solution for 100 in comment under "correct" answer.

This method also fails on 302 redirects when curl is set to follow the location header.

This doesn't work on some POST requests too.

pr1001

If you specifically want the Content-Type, there's a special cURL option to retrieve it:

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$response = curl_exec($ch);
$content_type = curl_getinfo($ch, CURLINFO_CONTENT_TYPE);

The OP asked if there is a way to retrieve the headers, not one specific header, this doesn't answer the OP's question.

@Geoffrey no, it can be useful for others who needs to get only Content-Type

@Acuna it doesn't matter how useful it is, it does not answer the OPs question!

@Geoffrey yes, but all answers are useful for other users too, don't forget about it, and OP has found an answer too, so everybody is satisfied

Cyril H.

Just set options :

CURLOPT_HEADER, 0

CURLOPT_RETURNTRANSFER, 1

and use curl_getinfo with CURLINFO_HTTP_CODE (or no opt param and you will have an associative array with all the informations you want)

More at : http://php.net/manual/fr/function.curl-getinfo.php

This does not seem to return the response headers to you at all. Or at least there's no way to retrieve them using curl_getinfo().

Enyby

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);

$parts = explode("\r\n\r\nHTTP/", $response);
$parts = (count($parts) > 1 ? 'HTTP/' : '').array_pop($parts);
list($headers, $body) = explode("\r\n\r\n", $parts, 2);

Works with HTTP/1.1 100 Continue before other headers.

If you need work with buggy servers which sends only LF instead of CRLF as line breaks you can use preg_split as follows:

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);

$parts = preg_split("@\r?\n\r?\nHTTP/@u", $response);
$parts = (count($parts) > 1 ? 'HTTP/' : '').array_pop($parts);
list($headers, $body) = preg_split("@\r?\n\r?\n@u", $parts, 2);

Shouldn't $parts = explode("\r\n\r\nHTTP/", $response); have 3rd parameter for explode as 2?

@user4271704 No. It allow find last HTTP message. HTTP/1.1 100 Continue can appear many times.

But he says something else: stackoverflow.com/questions/9183178/… which one of you are correct?

HTTP/1.1 100 Continue can appear many times. He view case if it appear only one time, but it wrong in common case. For example for HTTP/1.1 100 Continue\r\n\r\nHTTP/1.1 100 Continue\r\n\r\nHTTP/1.1 200 OK...\r\n\r\n... his code don't work properly

Splitting on \r\n is not reliable, some servers do not conform to HTTP specifications and will only send a \n. The RFC standard states that applications should ignore \r and split on \n for greatest reliability.

tony gil

My way is

$response = curl_exec($ch);
$x = explode("\r\n\r\n", $v, 3);
$header=http_parse_headers($x[0]);
if ($header=['Response Code']==100){ //use the other "header"
    $header=http_parse_headers($x[1]);
    $body=$x[2];
}else{
    $body=$x[1];
}

If needed apply a for loop and remove the explode limit.

Antony

Here is my contribution to the debate ... This returns a single array with the data separated and the headers listed. This works on the basis that CURL will return a headers chunk [ blank line ] data

curl_setopt($ch, CURLOPT_HEADER, 1); // we need this to get headers back
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, true);

// $output contains the output string
$output = curl_exec($ch);

$lines = explode("\n",$output);

$out = array();
$headers = true;

foreach ($lines as $l){
    $l = trim($l);

    if ($headers && !empty($l)){
        if (strpos($l,'HTTP') !== false){
            $p = explode(' ',$l);
            $out['Headers']['Status'] = trim($p[1]);
        } else {
            $p = explode(':',$l);
            $out['Headers'][$p[0]] = trim($p[1]);
        }
    } elseif (!empty($l)) {
        $out['Data'] = $l;
    }

    if (empty($l)){
        $headers = false;
    }
}

Community

The problem with many answers here is that "\r\n\r\n" can legitimately appear in the body of the html, so you can't be sure that you're splitting headers correctly.

It seems that the only way to store headers separately with one call to curl_exec is to use a callback as is suggested above in https://stackoverflow.com/a/25118032/3326494

And then to (reliably) get just the body of the request, you would need to pass the value of the Content-Length header to substr() as a negative start value.

It can appear legitimately, but your answer is incorrect. Content-Length does not have to be present in a HTTP response. The correct method to manually parse the headers is to look for the first instance of \r\n (or \n\n). This could be done simply by limiting explode to return only two elements, ie: list($head, $body) = explode("\r\n\r\n", $response, 2);, however CURL already does this for you if you use curl_setopt($ch, CURLOPT_HEADERFUNCTION, $myFunction);

james-geldart

A better way is to use the verbose CURL response which can be piped to a temporary stream. Then you can search the response for the header name. This could probably use a few tweaks but it works for me:

class genericCURL {
    /**
     * NB this is designed for getting data, or for posting JSON data
     */
    public function request($url, $method = 'GET', $data = array()) {
        $ch = curl_init();
        
        if($method == 'POST') {
            
            curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
            curl_setopt($ch, CURLOPT_POSTFIELDS, $string = json_encode($data));
            
        }

        
        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_VERBOSE, true);
        
        //open a temporary stream to output the curl log, which would normally got to STDERR
        $err = fopen("php://temp", "w+");
        curl_setopt($ch, CURLOPT_STDERR, $err);
        

        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        $server_output = curl_exec ($ch);
        
        //rewind the temp stream and put it into a string   
        rewind($err);
        $this->curl_log = stream_get_contents($err);
        
        curl_close($ch);
        fclose($err);

    
        return $server_output;
        
    }
    
    /**
     * use the curl log to get a header value
     */
    public function getReturnHeaderValue($header) {
        $log = explode("\n", str_replace("\r\n", "\n", $this->curl_log));
        foreach($log as $line) {
            //is the requested header there
            if(stripos($line, '< ' . $header . ':') !== false) {
                $value = trim(substr($line, strlen($header) + 3));
                return $value;
            }
        }
        //still here implies not found so return false
        return false;
        
    }
}

sneaky

Improvement of Geoffreys answer:

I couldn't get the right length for header with $headerSize = curl_getinfo($this->curlHandler, CURLINFO_HEADER_SIZE);- i had to calculate header size on my own.

In addition some improvements for better readability.

$headerSize = 0;
curl_setopt_array($this->curlHandler, [
CURLOPT_URL => $yourUrl,
CURLOPT_POST => 0,
CURLOPT_RETURNTRANSFER => 1,
// this function is called by curl for each header received
CURLOPT_HEADERFUNCTION =>
         function ($curl, $header) use (&$headers, &$headerSize) {
              $lenghtCurrentLine = strlen($header);
              $headerSize += $lenghtCurrentLine;
              $header = explode(':', $header, 2);
              if (count($header) > 1) { // store only vadid headers
                   $headers[strtolower(trim($header[0]))][] = trim($header[1]);
              }
              return $lenghtCurrentLine;
           },
]);
$fullResult = curl_exec($this->curlHandler);
$result = substr($fullResult, $headerSize);

K-Gun

Just in case you can't / don't use CURLOPT_HEADERFUNCTION or other solutions;

$nextCheck = function($body) {
    return ($body && strpos($body, 'HTTP/') === 0);
};

[$headers, $body] = explode("\r\n\r\n", $result, 2);
if ($nextCheck($body)) {
    do {
        [$headers, $body] = explode("\r\n\r\n", $body, 2);
    } while ($nextCheck($body));
}

tosturaw

Try this if you are using GET:

$curl = curl_init($url);

curl_setopt_array($curl, array(
    CURLOPT_URL => $url,
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_ENCODING => "",
    CURLOPT_MAXREDIRS => 10,
    CURLOPT_TIMEOUT => 30,
    CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
    CURLOPT_CUSTOMREQUEST => "GET",
    CURLOPT_HTTPHEADER => array(
        "Cache-Control: no-cache"
    ),
));

$response = curl_exec($curl);
curl_close($curl);

diyism

Return response headers with a reference parameter:

<?php
$data=array('device_token'=>'5641c5b10751c49c07ceb4',
            'content'=>'测试测试test'
           );
$rtn=curl_to_host('POST', 'http://test.com/send_by_device_token', array(), $data, $resp_headers);
echo $rtn;
var_export($resp_headers);

function curl_to_host($method, $url, $headers, $data, &$resp_headers)
         {$ch=curl_init($url);
          curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $GLOBALS['POST_TO_HOST.LINE_TIMEOUT']?$GLOBALS['POST_TO_HOST.LINE_TIMEOUT']:5);
          curl_setopt($ch, CURLOPT_TIMEOUT, $GLOBALS['POST_TO_HOST.TOTAL_TIMEOUT']?$GLOBALS['POST_TO_HOST.TOTAL_TIMEOUT']:20);
          curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
          curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
          curl_setopt($ch, CURLOPT_HEADER, 1);

          if ($method=='POST')
             {curl_setopt($ch, CURLOPT_POST, true);
              curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($data));
             }
          foreach ($headers as $k=>$v)
                  {$headers[$k]=str_replace(' ', '-', ucwords(strtolower(str_replace('_', ' ', $k)))).': '.$v;
                  }
          curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
          $rtn=curl_exec($ch);
          curl_close($ch);

          $rtn=explode("\r\n\r\nHTTP/", $rtn, 2);    //to deal with "HTTP/1.1 100 Continue\r\n\r\nHTTP/1.1 200 OK...\r\n\r\n..." header
          $rtn=(count($rtn)>1 ? 'HTTP/' : '').array_pop($rtn);
          list($str_resp_headers, $rtn)=explode("\r\n\r\n", $rtn, 2);

          $str_resp_headers=explode("\r\n", $str_resp_headers);
          array_shift($str_resp_headers);    //get rid of "HTTP/1.1 200 OK"
          $resp_headers=array();
          foreach ($str_resp_headers as $k=>$v)
                  {$v=explode(': ', $v, 2);
                   $resp_headers[$v[0]]=$v[1];
                  }

          return $rtn;
         }
?>

Are you sure $rtn=explode("\r\n\r\nHTTP/", $rtn, 2); is correct? Shouldn't 3rd parameter of explode be removed?

@user4271704, the 3rd param is to deal with "HTTP/1.1 100 Continue\r\n\r\nHTTP/1.1 200 OK...\r\n\r\n..." header

But he said something else: stackoverflow.com/questions/9183178/… which one of you are correct?

@user4271704 the link you are referring to also use: explode("\r\n\r\n", $parts, 2); so both are right.

Bevan

If you don't really need to use curl;

$body = file_get_contents('http://example.com');
var_export($http_response_header);
var_export($body);

Which outputs

array (
  0 => 'HTTP/1.0 200 OK',
  1 => 'Accept-Ranges: bytes',
  2 => 'Cache-Control: max-age=604800',
  3 => 'Content-Type: text/html',
  4 => 'Date: Tue, 24 Feb 2015 20:37:13 GMT',
  5 => 'Etag: "359670651"',
  6 => 'Expires: Tue, 03 Mar 2015 20:37:13 GMT',
  7 => 'Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT',
  8 => 'Server: ECS (cpm/F9D5)',
  9 => 'X-Cache: HIT',
  10 => 'x-ec-custom-error: 1',
  11 => 'Content-Length: 1270',
  12 => 'Connection: close',
)'<!doctype html>
<html>
<head>
    <title>Example Domain</title>...

See http://php.net/manual/en/reserved.variables.httpresponseheader.php

uhm, you don't really need PHP either, but that just happens to be what the question is about...

Can PHP cURL retrieve response headers AND body in a single request?

Follow WeChat

Want to stay one step ahead of the latest teleworks?

相似问题

Platform

Support

Links

Contact US