YouTube : get more than 20 search results from the command line

searchyoutube

I want to get the YouTube video IDs for this search pattern "joseph prince" from curl :

curl "https://www.youtube.com/results?sp=CAISAhgC&search_query=joseph+prince&max_results=50" 2>&1 | grep -oP "watch\?v=[\w-]{10,}" | sort | uniq | wc -l
20

My problem is that I get only 20 results.

If I use the webbrowser to go downto the end of the list of the results, the browser automatically queries for more results and displays them.

How can I get more from the command line?

Best Answer

Use the API, Luke!

It’ll spoil you with a neat JSON formatted response (which you can grep as well, or better use a parsing tool): https://www.googleapis.com/youtube/v3/search?part=snippet&maxResults=50&q=joseph+prince&type=video&videoDuration=long&fields=items(id(videoId)%2Csnippet(description%2Ctitle))%2CnextPageToken%2CpageInfo%2CprevPageToken%2CregionCode&key=YOUR_API_KEY

Replace the YOUR_API_KEY part by an API key you generate at https://console.cloud.google.com/apis/credentials (you need log into a Google account for that).

The API supports up to 50 results per ‘page’; if you query for the field nextPageToken (as I did in the upper URI), you can use the returned token in a pageToken parameter (&pageToken=THE_NEXT_PAGE_TOKEN) in a subsequent query for the respective next batch of 50 results, and so forth.

The docs [ https://developers.google.com/youtube/v3/docs/search/list ] and API explorer [ https://developers.google.com/apis-explorer/#p/youtube/v3/youtube.search.list?part=snippet&maxResults=50&q=joseph+prince&type=video&videoDuration=long&fields=items(id(videoId)%2Csnippet(description%2Ctitle))%2CnextPageToken%2CpageInfo%2CprevPageToken%2CregionCode ] are helpful.

Related Topic