Olmert, appearing before parliament's influential Foreign Affairs and Defense Committee, also said Israel would continue to negotiate with Palestinian President Mahmoud Abbas even if militants continue to fire rockets at southern Israel from Gaza, panel member Yuval Steinitz said.
But implementation of any peace accord would depend on the attacks stopping, another meeting participant said.
Olmert appeared before the committee to brief it on preparations for the peace summit, which is expected to take place at the end of November in Annapolis, Maryland. The meeting is meant to formally relaunch peace talks, which broke down in violence nearly seven years ago.
Olmert told the committee that the summit will be a one-day gathering, but will be used to launch intense negotiations with the Palestinians. He expressed confidence that he could reach a peace accord with the Palestinians.
"It is not impossible to bridge the gaps," Olmert was quoted as saying by the meeting participant, who spoke on condition of anonymity because the meeting was held behind closed doors.
Steinitz and lawmaker Yossi Beilin said Olmert told panel members that Israel would release more than 400 of the 12,000 Palestinian prisoners it holds ahead of the summit.
Because so many Palestinian families have relatives held in Israeli jails, prisoner releases are a charged issue in Palestinian society, and releases are considered a confidence-building measure.
Even so, the Palestinians have urged Israel free even more prisoners.
Olmert told lawmakers that Israel hopes to wrest from the Palestinians the "recognition that Israel is the country of the Jewish people."
Palestinian negotiator Saeb Erekat dismissed Israel's demand, which would imply the Palestinians drop their demand for the return of Palestinian refugees and their descendants to homes they lost after Israel was established in 1948.
"When they say that they want us to recognize a Jewish state, that is impossible," Erekat told Palestinian radio.