Something else I would recommend is try to establish a guide where certain gems/herbs/etc. tend to be used with a specific sort of magic. Not that you have to adhere to it 100%, but Alchemists did study things, and group them by their supposed 'magical' attributes. This can also make it a little easier for the game if you have an idea to look for.
For example, with Gems you might use (this is pooled from various sources). Others will be suggestions since I don't think you want to add more gems
Finally, some of these will have overlap.
Agate - focus, concentration, protection
Amber - reversal, change - I would also recommend plants (jade normally would be that), but since amber comes from trees...
Amethyst - Clarity of the mind, influencing others, possibly clairvoyance and healing
Beryl - element air, fights, creativity, intelligence
Coral - if pearl is used for women, maybe coral could be used for men. It has also connotations of immortality,
Diamond - Strength, durability, the element of earth
Emerald - inspires extreme emotion (love, passion, etc.) and reproduction. This could be used to make copies of stuff as well
Garnet - Spheres of luck and pleasure
Malachite - amplification, change, some protection as well
Opal - sympathy, compassion, faith and love, foresight
Pearl - purity, women, and the element of water
Quartz - moon, sexuality, deception, madness, illusion
Ruby - justice, the element of fire, destruction, blood
Turquoise - corruption, putrefaction, necromancy, darkness
The one Gem I left out of the above is Azurite since it isn't commonly mentioned. Maybe it could have control of Magic. Sources I do have say it is good for traveling out of body and channeling. Maybe it could also be used for summoning/banishment.