Answer:
6
Step-by-step explanation:
The worst scenario is that of a slight possibility, the first 5 balls you took are completely different color. The chance of this happening is tiny, but if you want to be sure, then you should consider about it.
But, after taking out the sixth ball, you can be sure that the ball, now right on your hand, must share a same color with at least one of the balls you took out, since there are only five colors.