Technically, the soap is more at risk of getting dirtier.
Unless it's some sort of anti-bacterial soap, the main purpose of soap isn't actually to "clean"...
Rather, soap molecules have properties that attach themselves to dirt, grease, oil, etc... And surround them in water. This makes the dirt easier to wash away with water - which is how you get clean when using soap.