You can't tell by visual inspection, that's for sure because some of them are lacquered/painted and even those that aren't all tend to look dark-grey. What you are asking is really tricky to fathom because there are so many characteristics that look the same between two ferrites at one frequency but are vastly different at another. If you are still interested I'll try and say what I'd do (what I'd really do is throw all my unboxed/unmarked ferrites in the trash and buy some more).
I'd consider winding (say) 5 equally spaced turns and putting the coil in a circuit to see what its inductance was - maybe a colpitts oscillator with a few caps that can be switched in and out. Maybe even make a band-pass filter from it and see where it resonates if you have a signal generator.
First type of result this will tell you is the inductance of the wound core. Then using the squared relationship between turns and inductance you can deduce its "effective permeability". This should enable you to narrow down the type of core to a range of possibilities.
You need to be be avoiding "test frequencies" significantly above 100kHz and preferably more like 10kHz - this is to reduce parasitic capacitance giving you errors.
OK so far, you might have determined the approximate "effective permeability" of the core BUT there are plenty of suppliers toting vastly different materials that you'd have to read through to try and identify the part so I'd next consider seeing how the indctance varied with temperature.
You don't need to test over a vast range, maybe just 25ºC to 50ºC would give you a decent shot at trying to uncover the ferrite. Use the oscillator/filter idea mentioned earlier and a controlled temperature - almost certainly the inductance will rise with temperature although there are a small percentage that will stay stable or fall but this will give you another tell-tale characteristic of the ferrite.
So now you have effective permeability and some idea what its temperature characteristic looks like. Scanning through various supplier's websites might narrow down the ferrite to maybe five or ten types.
It's going to be a long process this way and you may never uncover what it is that is sitting in your junk box. I suppose if your effective permeability is low it's likely to be either very temperature stable (i.e. good for filters up to (say) 1MHz) or it could have very low losses up to over 50MHz. The temperature test that indicated hardly any change in inductance across 25ºC might tell you its a material like Ferroxcube's 3D3: -
Also shown is 3C90 for comparison. 3D3 has a flat curve of inductance/permeability against temperature; probably changing something like 5% in a 25ºC change around ambient. 3C90 probably changes about 20%. It also has a much higher permeabilty. I'd recognize these two ferrites from their characteristics!
I think I've definitely convinced myself to throw all unrecognizable ferrites in the bin.
Bottom line - if you have a target circuit try it.
EDIT Also, here's is a question/answer on EE stack exchange that might also be useful or provoke some other ideas.
Schematic 1 offers direct connection to the center tap of the exterior
port windings, and is suitable for PoE connection (which may either
draw power, or supply power, over twisted pair). Schematics 2 and 4
do not have low-resistance connection suitable for PoE. Schematic #3
does technically have the ability to connect to PoE, but only
through an inductor (which may saturate).
Modern designs would usually allow bidirectional use of
the four lanes, and #2 does not. It is not surprising that it may
become obsolete, in favor of #4.
Best Answer
Colours are not an industry standard for ferrites as these recipes are proprietary and there are many variations of different materials.
You might want to contact the supplier for specs and if none, continue with your trial and error methods. The most important aspect is often the power losses at certain temperature ranges in order to be thermally stable.
This major MAGNETICS company (by same name) simplifies their specs down to 4 different materials P,F,R,K
http://c1233384.r84.cf3.rackcdn.com/UK_MAG_ferrite_material_selection_guide_AN.pdf
I do have several books of hundreds of different ferrite samples from TDK, if anyone is interested in paying for shipping them.