The density of ice is less than that of liquid water, because it is floating. No, the water will not overflow.
Work Step by Step
The ice cube displaces a volume of water whose weight is exactly the same as that of the water (liquid or solid) contained in the ice cube. As the ice melts, the water level is unchanged.