Police said at least some 20,000 people had gathered just after the protest's 3:45 pm (14:45 GMT) kick-off time outside the Bundestag, Germany's parliament building.
The protests came after some parties voted alongside the far-right party, Alternative for Germany (AfD), over a non-binding resolution on immigration last week.
Friedrich Merz caused dismay within his party for saying the law was so important he was prepared to pass it with support from the far-right, nativist Alternative for Germany.