Its all about what your doing and how much inventory you can sacrifice.
When deciding if a charm should become a staple of your characters inventory you have to look at alot of things.
Charms are seperated into three sizes: Small, Large, Grand
These charms can have a single mod or sometimes up to three, that being said we can classify smalls and larges more as *utility* charms because we usually get them to add MF/Life/Mana/Resists and Skill Charms for Grands because with the exception of a few good mods GC's that are normally sought after are +1 Skill either plain or with a mod. (Note: There ARE exceptions to this, AR/Life GC's, 15 All Resists Gc's etc.)
Then we have to look at the type of character we are:
1) if we are a PVP player we really dont need much inventory room, and we can fill up on charms to the max, for more life, more resists, more ar, and more skills.
2) If we are a MF character our inventory to charm ratio becomes a bit more important, as we want to have as much room as possible to pick up gear as it drops and at the same time we want to mantain a high count of MF from our inventory.
3) If we are a PVM character not as concerned with MF such as a dclone killer or Uber Trist Killer we dont have to be as concerned with inventory room but we still want *SOME* room to pick up good drops.
That being said since we have a maximum of 40 spaces in our inventory we have to balance our charm to space ratio accordingly, some people prefer up to 16 spaces to be open for drops, (thats 4 2x2 Helms, or 2 2x8 Weapons)
while others can live with as few as 4 or 8. Your maximum count really boils down to how many charms NEED over how much space you want.
Lots of charms can be found with 2 or more desirable mods on the same charm, however these are usually expensive to trade for and rare to find, for example: a 20 life with 5 resists does the work of 2 small charms in one, or even a 4 resists with 18 life is great. (Provided you need the extra resists & life) Any charms you can find that pull double duty can help you trim down the fat from your inventory.
Also look at finding the unique charms such as Annihilus (Small- Resists,Attribs,+1 Skill,+Exp Gained), Hellfire Torch (Large- Resists,Attribs,+3 Skills), and Gheed Fortune (Grand- MF, Extra Gold), these can really help you scale down the number of utility charms you need and free up space for other things.
Keeping things like +1 to str small charms or 4 to dex GC's isnt really advisable because you're just wasting space unless you HAVE to have that 1 str for equipping something until you level some more, go ahead and ditch it.
There are alot of different configurations and they greatly depend on what type of character you are making, and it all boils down to how much room you are willing to sacrifice as to how many Skill GCS/Utility charms you should have.