Google Sheets – Extract Information from Webpage Using IMPORTXML

formulasgoogle sheetsimportxml

I've been trying to grab some data from this webpage: https://seekingalpha.com/symbol/BZUN. I want to import the follower count (under the add to portfolio button) into a cell on google sheets. For example, the data I'd try to import under this page would be the (22,201 followers).

I think that I need to use =importXML with this. However, it's not working. From the research that I've done, it's because that value isn't static and comes from another source.

I believe that the remedy to my problem is do use Regex Extract, but I have no idea how to do this.

Best Answer

From the research that I've done, it's because that value isn't static and comes from another source

You are not wrong there. after disabling JavaScript you can observe that followers count case to exist, and therefore not even regex will help.

First, I should warn you, that down this path lies madness.

It may seem like something obvious to do, and it doesn't look that hard, but generating HTML using a spreadsheet (be it Google Sheets, Excel or any other spreadsheet package) may seem simple, but things can and will get over-complicated, or you will very soon reach a point in which you will need to write a macro to achieve your desired output.

Moreover, once you take the plunge and start writing your first couple of macros, if your sheet is not already overly complicated, you will hit the limits of the macro language pretty soon too.

But why? I'm not doing anything too intricate!

The reason you will quickly exhaust the features of a spreadsheet when using it to generate HTML is that generating structured markup is not well suited to the overall design of grid calculation paradigm. Which is the reason we have Word and Excel and that

"In the beginning there was Visicalc, and Wordstar, 
 and each did what they did 
 and they were not a single program" 

(From 'The book of Mozilla', I forget which verses).

Word processing is in itself, basically the process of generating a structured markup document. So there are software packages more suited to the task.

So what would be a wise thing to do?

My advice would be to keep the database in the spreadsheet, (a task for which it is well suited) and spare yourself deep frustration and unmet deadlines, and invest some time to learn a proper language; one that is easy to learn and efficient at templating HTML.

Python comes to mind. The alternative being taking advantage of the features of other Google products (since you're already doing Sheets) like Google Sites that makes it dead easy to create proper websites, and allows to embed and drive some of its pages from the data in a spreadsheet.

But I want to do it in Sheets! In fact, that's why I am asking here!

Agreed. However, I've been down that road, and would have appreciated that someone warned me that madness awaited, so I felt it would have been irresponsible and unfair to the Asker not to warn him of the perils ahead. (I'm looking at those who know, and didn't raise a voice.)

That being said, I've seen people abuse spreadsheets in manners far more troubling. So:

Here is my shot at good advice on the design of a structured-markup generating spreadsheet

You can see all this advice in action in this Google Spreadsheet, so you can examine the actual formulas at your leisure and see how these simple points will help you avoid creating an unmaintainable monstrosity and keep your sanity for a while more.

Create a master tag table that calculates the attributes for basic tags from the values of two or more independent cells
Take advantage of advanced features in the spreadsheet like named ranges to keep all your formulas simple by referencing the values on the master tag generating table
For the named ranges, choose names that look like the markup they reference so your formulas are readable and maintainable, like say
```
 =CONCATENATE(p_s1, A4, p_close) or
 =CONCATENATE(span_s2, B7, span_close)
```
Complexity will creep up quickly, keep your formulas simple and build the markup progressively from the inside out, taking advantage of using formulas in adjacent cells. for example, in column C surround a plain value with inner <p> tags, in column D, surround the value from C in outer <span> tags, in column E, surround whatever is in D with outer <div> tags and so on. one step at a time, you've got plenty of columns and then some
Avoid at all costs doing two steps or complex markup in a single cell. Always rely on your master tags table, because you know when you're done, someone will suggest that the class for all titles should be something different than what you coded. You'll be glad then that you didn't hardcode those attributes in a formula that wraps around twice on the f(x) bar, did you?

Please do visit the example sheet since that will be a lot clearer that a gazillion screenshots or the dense prose I tend to write when I get all worked up. Hope this really helps!

Google-sheets – Extract table from moneycontrol.com

When URL: http(s)://www.moneycontrol.com/stocks/fno/marketstats/futures/gainers/homebody.php?opttopic=&optinst=allfut&sel_mth=1&sort_order=0

opened in client (browser) following server side rendering of dynamic data are requested by browser:

 - send @   jquery-1.12.0.min.js:4 
 - ajax @   jquery-1.12.0.min.js:4
 - updaterss    @   jquery.webticker.js:39
 - (anonymous)  @   jquery.webticker.js:137
 - many more...

this is why you can not use IMPORTHTML or IMPORTXML to fetch Dynamic data as in this webpage.

How to check if Error: Imported Content is Empty is because of dynamic data?

use a la carte rendering of web page in question like:

=IMPORTXML("http://www.moneycontrol.com/stocks/fno/marketstats/futures/gainers/homebody.php?opttopic=&optinst=allfut&sel_mth=1&sort_order=0", "/")
- "/" will force to fetch everything from URL, and there is nothing useful in this IMPORT XML/HTML for your need.
Using the Chrome browser's INSPECT function displays all the client requests.