The protesters, including pensioners, were pressed up against a wall of steel thrown around the parliament building. Some broke through the barriers and spilled onto the streets, forcing the police to bring in reinforcements and deploy armoured buses to buttress the main parliament gate.
The protest came as results from rural Yamaguchi showed that Tetsunari Iida, an advocate of renewable energy, had lost his bid to become governor. He was defeated by a candidate backed by the opposition Liberal Democratic party (LDP), which promoted nuclear energy during its decades in power, Kyodo news agency reported.
Iida, who wants Japan to give up nuclear power by 2020, had promised to revitalise Yamaguchi's economy with renewable energy projects and opposed a project to build a new nuclear plant in the town of Kaminoseki.
Energy policy has become a major headache for prime minister, Yoshihiko Noda. Weekly protests outside the PM's office have grown in size in recent months, with ordinary workers and mothers with children joining the crowds.
On Sunday, the protesters, holding candles, took their demonstration to parliament. Many had marched past the headquarters of Tokyo Electric Power Co, the company at the heart of Fukushima disaster, the worst nuclear crisis since Chernobyl in 1986.
"We are here to oppose nuclear power, which is simply too dangerous," said Hiroko Yamada, an elderly woman from Saitama prefecture near Tokyo.
"[Noda] isn't listening to us. He only listens to companies and Yonekura," she said, referring to Hiromasa Yonekura, the chairman of Japan's biggest business lobby.
An upset victory by Iida would have put pressure on the government which is considering an energy portfolio to replace a 2010 programme that would have boosted nuclear power's share of electricity supply to more than half by 2030.
Noda, who approved restarting two reactors recently, has said he would decide on a new medium-term energy plan in August, but reports said that decision could be delayed.
Experts have proposed three options: zero nuclear power as soon as possible, a 15% atomic share of electricity by 2030, or 20-25% by the same date compared to almost 30% before the Fukushima disaster.
Under pressure from businesses worried about stable electricity supply, Noda has been thought to be leaning towards the 15% option, which would require all of Japan's 50 reactors to resume operations before gradually closing older units.
The growing anti-nuclear movement, however, may make that choice difficult.
Multiple inquiries into the Fukushima nuclear disaster have pointed to the failure by authorities and energy companies to adopt strict safety measures and disaster response plans.