Magento – Magento Profiling – Why so many EAV: Magento\Eav\Model\Config::getAttribute

eavflat-catalogmagento2

I'm just reviewing the speed of a store and just wondering if anyone knows more about why I have over 100 EAV: Magento\Eav\Model\Config::getAttribute 0.006978 rows within my profiler.

It's just I do not remember ever seeing this many entries when reviewing profiler on previous occasions. This store does have a flat catalog enabled, however, it was disabled for a bit after attributes reached the limit for using a flat catalog. I thought EAV wouldn't be used if the flat catalog was enabled?

UPDATE

Even CMS pages and empty category pages without any products seem to be using the same number of getAttribute calls so I am unsure where they are coming from as the profiler does not make this clear. This seems to be an issue on the product pages also however even slower. This also seems to be getting worse I will try and save a copy of my profiler output for review.

I have tried disabling all third-party modules and reverting to the luma theme but still there and believe it may be since updating to 2.2.0 or 2.2.1?

Can anyone else try profiler to see if this is normal and what version you are running?
Should I be concerned? Pages Load in about 2-3 seconds noncached on production but within dev, I am noticing huge slowdowns and even worse looking profiler. Magento section of profiler can show as low as 1.5 seconds on dev but take 80 seconds for a response this was always maxed 3-5 seconds. Without profiler on the site speeds up considerably however I'm still concerned at some product pages taking 15 seconds when they used to be under 3s.
How can I stop it / track down where they are coming from?

I have a 2.1.7 that I have checked on and this is not the case here.

Profiler:

This seems to be getting worse so I want to show just to show how bad this profiler looks on an empty category page now (Dev and Production in separate tabs):

https://docs.google.com/spreadsheets/d/1fbw_XlJRpRYue8m9VI31lv8BdwBqZypRahFelMGWCq8/edit?usp=sharing

This, however, does seem to be worse with profiler enabled and page loads are a more normal 3 seconds uncached when this is disabled. I need to try a better profiler to try and figure this out but so far I haven't found a good one that is free.

Update 2

I had another little play with this today and have started by removing a lot of blocks from all catalog_product_view.xml files within the core vendor files. None of these removed any of the attributes calls even tho the page was basically blank.

I then modified the get attribute call to just echo a 1 out and the bulk of these was shown at top of the page. This is suggesting to me that this is happening before the page even begins to render unlike other instances of getAttribute which echoed next to where the attribute was called.

My issue however now only affects product and category pages which is odd wish I knew what caused that to change.

magento->LAYOUT->layout_render->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute->EAV: Magento\Eav\Model\Config::getAttribute

Best Answer

I have debugged this function today: I can see you concern as this function seems big and we can see some load/save function calls within it.

However, after inspecting the code, load and save in fact are not calling the database but rather feed property in the classes that appears to be a 'shared' model.

--> each time the function is called, it will assign the attribute's data onto an array that is a property of the Config model and this property will prevent further calls to be doing the whole lot of code lines again..

The one line that could harm is in the function \Magento\Eav\Model\Entity\Collection\AbstractCollection::addAttributeToSelect

in this function, the line $attrInstance = $this->_eavConfig->getAttribute($this->getEntity()->getType(), $attribute); seems to be what will finally load the data for the attribute. However, if your system happens to be using a cache, this should use the cache storage to read the data and that means it should load at the fastest speed you can get.

Long story short, this function is typically a function in Magento that is doing a lot but after all these years of settling Magento as a premium ecommerce platform, it is quite clear Magento team has not fallen in basic traps of calling greedy functions again and again.

Related Solutions

Magento – Magento 2: Join EAV attribute from product attribute set on a custom flat table

I've managed to get a working solution where you can use the attribute set model ( or the full EAV ) simply by inversing the query, ie. instead of joining on the flat table, i'm in turn joining the flat table on the product object.

public function getDiscountRules($productId)
{
    $productCollection = $this->productCollectionFactory->create();
    $discountRuleTable = $this->productCollection->getConnection()->getTableName('discount_rules');
    $productCollection
        ->getSelect()
        ->join(
            ['dr' => $discountRuleTable],
            'dr.product_id ON e.entity_id'
        );

   $productCollection->addFieldToFilter('entity_id', $productId);
   return $productCollection->getItems();    
}

This simply joins the flat table on the product EAV and still gives you all the information you want, without having to join 4 tables just to join on the attributes for the products.

Magento2 – Are Flat Tables Still Relevant?

Yes, flat tables are still a thing. You can enable flat tables for products and categories from stores->configuration->catalog->storefront.

Why do we need flat tables?

They do offer significant performance boost for large catalog data. One of our client has only 5 categories and around 250 products so it doesn't really matter if you have flat on or off. It really mattered when we switched it on for another store that had 800 categories and 17000 products and 5 multistores. Performance boost becomes apparent when you start noticing what's happening in the database. Here's what the database looks like when you have multiple websites.

The point being joins become very expensive with increase in table size even with indexes. Flat tables REDUCE (not eliminate) JOINs.

When are they created or updated

Flat tables are regenerated when you do full reindex with bin/magento indexer:reindex. They are updated on product or category update. Only relevant rows are updated.

Flat tables are not generated on setup:di:compile

**Which attributes go into flat tables? (and some limitations) **

Short answer is all the attributes that are going to be used in product listing (listing + layered navigation). An attributes store front settings decide whether or not the attribute should be included in flat tables

All attributes are not flattened because of limitations I mention below.

Flat table limitation

Finally there is one limitation for these flat tables which is imposed by mysql. There are row size and column size limits on table. You can read more about it on mysql table limits

You will find this <max_index_count>64</max_index_count> interesting node in config.xml of module-catalog.

Hope that helps.

Update 2

Best Answer

Related Solutions

Magento – Magento 2: Join EAV attribute from product attribute set on a custom flat table

Magento2 – Are Flat Tables Still Relevant?

Related Topic