Selenium – Headless browser testing with download functionality

casperjsheadless-browserphantomjsselenium

I've been looking for a solution to do headless testing in osx. But I need the ability to save files returned by the server.

I've tested selenium, phantomjs, casperjs and have looked into anything I could find online.

none of them supports downloading. am I missing something? are there any headless browser/testing frameworks that support downloads?

Best Answer

What you can do is:

start a virtual display (see Xvfb)
start up Firefox browser with preferences configured to automatically save csv files

Working example in python with additional comments (using pyvirtualdisplay xvfb wrapper):

from os import getcwd
import time

from pyvirtualdisplay import Display
from selenium import webdriver

# start the virtual display
display = Display(visible=0, size=(800, 600))
display.start()

# configure firefox profile to automatically save csv files in the current directory
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.download.folderList", 2)
fp.set_preference("browser.download.manager.showWhenStarting", False)
fp.set_preference("browser.download.dir", getcwd())
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/csv")

browser = webdriver.Firefox(firefox_profile=fp)
browser.get('http://www.nationale-loterij.be/nl/onze-spelen/lotto/resultaten')

# check the option
browser.find_element_by_id('corporatebody_3_corporategrid93961a8f9b424ed6bd0697df356d9483_1_rblType_0').click()

# click the link
browser.find_element_by_name('corporatebody_3$corporategrid93961a8f9b424ed6bd0697df356d9483_1$btnDownload').click()

# hardcoded delay for waiting a file download (better check for the downloaded file to appear on the disk)
time.sleep(2)

# quit the browser
browser.quit()

# stop the display
display.stop()

Related Solutions

Selenium – Does Selenium support headless browser testing

you need not use PhantomJS as an alternative to Selenium. Selenium includes a PhantomJS webdriver class, which rides on the GhostDriver platform. Simply install the PhantomJS binary to your machine. in python, you can then use:

from selenium import webdriver
dr = webdriver.PhantomJS()

and voila.

Python – Using Selenium with Python and PhantomJS to download file to filesystem

Despite this question is quite old, downloading files through PhantomJS is still a problem. But we can use PhantomJS to get download link and fetch all needed cookies such as csrf tokens and so on. And then we can use requests to download it actually:

import requests
from selenium import webdriver

driver = webdriver.PhantomJS()
driver.get('page_with_download_link')
download_link = driver.find_element_by_id('download_link')
session = requests.Session()
cookies = driver.get_cookies()

for cookie in cookies: 
    session.cookies.set(cookie['name'], cookie['value'])
response = session.get(download_link)

And now in response.content actual file content should appear. We can next write it with open or do whatever we want.

Best Answer

Related Solutions

Selenium – Does Selenium support headless browser testing

Python – Using Selenium with Python and PhantomJS to download file to filesystem

Related Topic