Yes, there will be a decrease in brightness of each bulb as another is added.
Work Step by Step
When many lamps are connected in parallel, each feels the same voltage (This is Rule 1 for parallel circuits, on page 444.). The battery must supply more and more current, but because of its internal resistance, this increased current causes the terminal voltage to drop (the colorful technical term is that the voltage "sags"). The result: each bulb experiences reduced voltage as another bulb is added, and each bulb diminishes in brightness as another is added.