Not unless you're doing a voiceover for a flick about a gravel pit.
I've heard a few singers over the years that have a gravelly voice and they were quite successful. Rod Stewart comes to mind as does Kim Carnes.
April Stevens was a singer who sang the song "Teach Me Tiger" around 1959-1960 or so.
I once heard her described as having "a throat full of angora sweaters."
I'd say it is a compliment.
It is neither a compliment nor an insult ... In itself, it is just a statement or opinion.
Yeah, it probably means your voice "rocks"
It's a compliment.