Powershell – Download URL content using PowerShell

powershell

I am working in a script, where I am able to browse the web content or the 'url' but I am not able to copy the web content in it & download as a file.
This is what I have made so far:

$url = "http://sp-fin/sites/arindam-sites/_layouts/xlviewer.aspx?listguid={05DA1D91-F934-4419-8AEF-B297DB81A31D}&itemid=4&DefaultItemOpen=1"
$ie=new-object -com internetexplorer.application
$ie.visible=$true
$ie.navigate($url)
while($ie.busy) {start-sleep 1}

How can I copy the content of $url and save it to local drive as a file?

Update:

I got these errors:

Exception calling "DownloadFile" with "2" argument(s): "The remote server returned an error: (401) Unauthorized." At :line:6 char:47 + (New-Object system.net.webclient).DownloadFile( <<<< "$url/download-url-content", 'save.html' )

Missing ')' in method call. At :line:6 char:68 + (New-Object system.net.webclient).DownloadFile( "$url", 'save.html' <<<<

Exception calling "DownloadFile" with "2" argument(s): "The remote server returned an error: (401) Unauthorized." At :line:6 char:47 + (New-Object system.net.webclient).DownloadFile( <<<< "$url", 'save.html' )

Ok, let me explain more, on what I am trying to do: I have a excel file in our share point site & this is the file I am trying to download locally(any format), which is a part of the script, so that for the later part of the script, I can compare this file with other data & get an output.

Now if I can somehow map "my documents" from the site & able to download the file, that will also work for me.

Best Answer

Update Jan 2014: With Powershell v3, released with Windows 8, you can do this:

 (Invoke-webrequest -URI "http://www.kernel.org").Content

Original Post, valid for Powershell Version 2

This solution is very similar to the other answers from stej, Jay Bazusi and Marco Shaw. It is a bit more general, by installing a new module into your module directory, psurl. The module psurl adds new commands in case you have to do a lot of html-fetching (and POSTing) with powershell.

(new-object Net.WebClient).DownloadString("http://psget.net/GetPsGet.ps1") | iex

See the homepage of the code-sharing website http://psget.net/.

This nice line of PowerShell script will dowload GetPsGet.ps1 and send it to Invoke-Expression to install PsGet Module.

Then install PsUrl, a Powershell Module inspired by curl:

To install something (in our case PsUrl) from central directory just type:

install-module PsUrl

get-module -name psurl

Output: 

ModuleType Name                      ExportedCommands
---------- ----                      ----------------
Script     psurl                     {Get-Url, Send-WebContent, Write-Url, Get-WebContent}

Command:

get-command -module psurl

Output: 

CommandType     Name                                                Definition
-----------     ----                                                ----------
Function        Get-Url                                             ...
Function        Get-WebContent                                      ...
Alias           gwc                                                 Get-WebContent
Function        Send-WebContent                                     ...
Alias           swc                                                 Send-WebContent
Function        Write-Url                                           ...

You need to do this only once.

Note that this error might occur:

Q: Error "File xxx cannot be loaded because the execution of scripts is disabled on this system. Please see "get-help about_signing" for more details."

A: By default, PowerShell restricts execution of all scripts. This is all about security. To "fix" this run PowerShell as Administrator and call

Set-ExecutionPolicy RemoteSigned

From now on, in your new powershell sessions/scripts, do this: 

import-module psurl
get-url "http://www.google.com"

To download and save to a file, do this:

get-url "http://www.google.com" | out-file -filepath "myfile.html"
Related Topic