Okay this is a complete guess, but is it some sort of plant or flower? I'm pretty sure you can use some flowers in baking, flowers are known for their scent and you have to look after them or else they wither.
A riddle, Ground not for thy daily bread, Leave me in my scented bed, Ignore me not, lest I doth wither, My name I leave you to consider. What am i?