Recursive Twitter search with PHP

I’ve been playing with the Twitter API recently and more specifically the Twitter search API. Sometimes the results from a Twitter search span across multiple pages and because of this, the result from a call to the Twitter search API may include a next_page variable. Using this variable we can recursively search Twitter until there are no more results. In this blogpost I’ll show you how.

I didn’t write my own Twitter search library. Instead, I extended the excellent PHPTwitterSearch library by Ryan Faerman. Don’t forget to download that and include or require it before using this class. Here is the code:

<?php
class RecursiveTwitterSearch extends TwitterSearch {
    var $request_count = 0;
    var $max_request_count = 5;
    var $max_id;
 
    function recursive_results() {
        $request  = 'http://search.twitter.com/search.'.$this->type;
        $request .= '?q='.urlencode($this->query);
 
        if(isset($this->rpp)) {
            $request .= '&rpp='.$this->rpp;
        }
 
        if(isset($this->page)) {
            $request .= '&page='.$this->page;
        }
 
        if(isset($this->lang)) {
            $request .= '?='.$this->lang;
        }
 
        if(isset($this->since)) {
            $request .= '&since_id='.$this->since;
        }
 
        if($this->show_user) {
            $request .= '&show_user=true';
        }
 
        if(isset($this->geocode)) {
            $request .= '&geocode='.$this->geocode;
        }
 
        if(isset($this->max_id)) {
            $request .= '&max_id='.$this->max_id;
        }
 
        $response = $this->objectify($this->process($request));
        $this->request_count++;
        if ($response) {
            $results = $response->results;
            if(!empty($response->next_page)) {
                preg_match('|\?page=([0-9]*)&max_id=([0-9]*)&|', $response->next_page, $matches);
                $this->page = $matches[1];
                $this->max_id = $matches[2];
                if ($this->request_count < $this->max_request_count) {
                    $results = array_merge($results, $this->recursive_results());
                }
            }
            return $results;
        }
        else return false;
    }
    function max_request_count($n) {
        $this->max_request_count = $n;
        return $this;
    }
}
?>

Basically what I did is duplicate the results() function from the parent class and add some code to it. Now, recursive_results() calls the API and when the variable next_page is present it will call itself again and merge the result sets. This process keeps on going either until we’re at the last page of search results or until a certain number of requests have been made. You can set this number yourself with the function max_request_count().

Here is some example code:

<?php
    $rts = new RecursiveTwitterSearch('starbucks');
    print_r($rts->recursive_results());
    echo 'It took ' . $rts->request_count . ' request(s) to get this result.';
?>

Got questions? Found a bug? Ideas for optimizations? General shout-outs? I’d love to hear them! :-)

2 comments on Recursive Twitter search with PHP

  1. Timo zegt:

    Great script / extension! Loving it already!

    Can I also set the max number of results per page? ($this->rpp)? This significantly influences the number of results (tweets) that are retrieved..

    June 9th, 2011

  2. admin zegt:

    I believe you can, Timo. Thanks for your comment :)

    June 9th, 2011

Leave a response: