C# – the most efficient way to manage large collection

ccollectionswpf

I have a rather difficult problem:

I periodically receive a Dictionary<decimal, int> subDict containing 100 records where

  • decimal is a Price
  • and int is Amount

The outcome I need is a dataGrid showing prices with amounts. But I cannot display subList directly, because sublist is often missing some prices:

subDict:

  • Price | Amount
  • 100.0001 | 192
  • 100.0005 | 123
  • 100.0007 | 2
  • 100.0008 | 123

above needs to be displayed as:

  • Price | Amount
  • 100.0001 | 192
  • 100.0002 | 0
  • 100.0003 | 0
  • 100.0004 | 123
  • 100.0005 | 0
  • 100.0006 | 0
  • 100.0007 | 2
  • 100.0008 | 123

I have to store a price with 4 decimal places, however the range of Price can be anything from 0 to 10 000. Final requirement is – once I click on the amount in the DataGrid, I need to be able to get access to all values in the row.

So I decided to ask for some help – can anyone think of a good strategy of dealing with such a problem?

One idea that came to my mind would be to initialize Dicionary<decimal, int> bigDictionary containing every price in the range.

However, if I initialize this dictionary with all the prices, with specified granularity (i.e. step of 0.0001) I end up with 10 000 * 10 000 records. That is rather large list, which takes considerable amount of memory. Of course it is easy to bind it to DataGrid in WPF so this can sort of work, but is in my opinion very inefficient.

The other idea I have would be to save incoming prices to a Dictionary<decimal, int> smallDict but then I would need to have some sort of mechanism to update DataGrid in such a way, that price gaps would be filled on the fly. Also I would somehow need to keep the dataGrid sorted by the price.

The first idea I am able to implement. The second one, I have no idea about so any code would be excellent.

I would be greatfull for any ideas!
Thank you for your time

Best Answer

Considering creating some type of drill-down interface. Rather than immediately trying to display 10,000 * 10,000 records to the user, summarize those 100,000,000 records into 100 ranges. Let the user select the desired range and then "zoom in" and split that range into 100 sub-ranges. Keep repeating until you get to the detail level and show the 100 entries that belong to the parent range.

Assuming your price range is 0.0000 to 9,999.9999 and you want to display 100 records at a time, your 100 top level ranges would be 0 - 99.9999, 100 - 199.9999, Etc. The "width" of your ranges would be:

  1. 100 (Top Level)
  2. 1
  3. .01
  4. .0001 (Detail Level)

This takes care of your display problem. What about the memory-usage issue? Create a view dictionary and bind your grid to it. Every time the user zooms in or out, simply populate this dictionary with the summarized or detail data that correspond to the current range and zoom level. You'll probably need some extra controls to show the current zoom/range to provide a way for the user zoom in and out.

Related Topic