Are you sure all those packets are actually leaving the iPhone?
Maybe Apple imposes bandwidth restrictions on apps, as I told you on stackoverflow you should really ask this question in the Apple Bluetooth mailing list, where there are several helpful Apple engineers that could potentially help you.
Otherwise you really need to specify what chipsets you are using if you want to get something out of these questions.
EDIT: ok, you are using the CSR chipset, then I suggest contacting CSR directly and posting your question here:
https://lists.apple.com/mailman/listinfo/bluetooth-dev
EDIT2: a sniffer could certainly help you see whether those packets are actually being sent by the iphone and not being received by the CSR chipset, or else it is just that the iPhone is never actually sending them over the air
The advertisement packet your beacons send can contain custom data. However the space is very limited, around 31 bytes. About 16bytes of it goes to the 128bit "service UUID" of your beacon. The structure of the packet is defined by the bluetooth spec and it does have some overhead, so I'm not sure how much is actually available for the custom data.
The service UUID is an identifier which your iphone app will search for. The idea that each service has a unique identifier, so if an app wants to find heart rate monitors, it will scan for the heart rate monitor uuid. In iOS you can even do a wildcard scan, and it will return any ble advertisements it can find, regardless of the services it has. It is however recommended by Apple guidelines that you specify the service UUID.
There is no requirement for pairing in order to receive the advertisment packets from the beacon. There is also no theoretical limit that would prevent that the beacons couldn't be found, given an infinite amount of time.
I haven't heard anyone putting 500 advertising BLE beacons in the range of an iPhone to see how many it can detect. With 500 beacons advertising simultaneously there will be a lot of collisions between packets. This could theoretically be calculated with knowing the advertisement interval.
The iPhone scan window (how long the channels are actively listened to) and scan interval (how often the scanning window is entered) cannot be adjusted, isn't documented and cannot be relied to remain the same between versions. This means that there isn't a way to really calculate how the advertisements will be found.
There's some recommendations published by Apple about advertising: https://developer.apple.com/hardwaredrivers/BluetoothDesignGuidelines.pdf
Oh, and by default iOS will filter the results so you get one notification per scan session, per beacon. You can set the "allow duplicates" option in the scan to get multiple notifications per scan session for each beacon if this is needed, but again, apple doesn't recommend it.
If you end up testing this with 500 beacons, or even > 10, I thin a lot of people would be interested in your results. I personally would also appreciate hearing about your experiences.
Best Answer
According to the Bluetooth 4.0 core specification the time period of 'advertising events' are as shown below for the different types of advertising packets:
ADV_IND: General connectable and scannable advertisement packets' time period ranges from 20 ms to 10.24s in steps of 0.625ms.
ADV_DIRECT_IND: Directed advertisement packets' time period is less than or equal to 3.75 ms. This kind of advertisement events can happen consecutively only for 1.28s. This is for a establishing a quick connection (if there is a device listening).
ADV_NONCONN_IND: Non-connectable and non-scannable advertisement packets' time period ranges from 100 ms to 10.24s in steps of 0.625ms.
ADV_SCAN_IND: Scannable advertisement packets' time period ranges from 100 ms to 10.24s in steps of 0.625ms.
So, unless you know the kind of device you are scanning for a good approach would be continuously to scan for around 11 (max) seconds to see if there are any advertising devices around. How often to do this will depend on the amount of battery or power available.
Hope this helps.