The University and College Union, which represents around 120,000 staff, will allow local branches to make a final decision on imposing a boycott on cooperation with Israeli academics.
"Every member should have the opportunity to have their say," the union's general secretary Sally Hunt told an annual meeting in Bournemouth, southern England.
But Hunt said she did not believe the majority of members would support a boycott and that it would likely be difficult to enforce.
Any future boycott would involve a block on Israeli and British university or college staff working on joint projects or assisting each other in their work, union spokesman Dan Ashley said.
The motion put to the union asked members to note that "Israel's 40-year occupation has seriously damaged the fabric of Palestinian society through annexation, illegal settlement, collective punishment and restriction of movement."
It called on British academics to condemn the "complicity of Israeli academia in the occupation."
Israeli Foreign Ministry spokesman Mark Regev said any boycott would be targeting the only "Middle Eastern country that has fully free and independent academia."
"We find it strange that British academics single out the Jewish state for special discriminatory treatment," Regev said.
Britain's ambassador to Israel, Tom Philips, told Israel Radio Thursday that the British government would not support a boycott.
"We don't support such boycotts," he said. "We think that the best way to achieve peaceful resolution to the problems of the region is to encourage both sides to take the steps necessary for progress through close engagement and dialogue."
Bill Rammell, Britain's higher education minister, said he was disappointed the union had passed a motion "which encourages its members to consider boycotting Israeli academics."
"I profoundly believe this does nothing to promote the Middle East peace process," he said.
However, Ashley said the motion encouraged members to debate the issue, not to vote for a boycott.
In 2005, Britain's 40,000-member Association of University Teachers, voted to boycott Israel's Haifa and Bar-Ilan universities for actions which it said undermined Palestinian rights and academic freedom.