Television video of the fight Wednesday night showed two men struggling in the balcony - one with his shirt pulled off - as several people stood around them and the orchestra performed a medley from the movie "Gigi".
Conductor Keith Lockhart briefly halted the performance while the men were escorted out.
Witnesses said they heard a scream from the balcony, and the sound of chairs falling, then a second scream as the fight escalated.
"The first time there was a scream, Keith looked up that way but he kept going," audience member June MacIndoe told Boston's WHDH-TV. "Then about a minute and a half later ... there was a big scream and you could hear chairs falling over and you could see them up there, fists going."
At that point, Lockhart stopped the performance for a couple of minutes, she said. "He just stood there, you know, quiet."
The concert resumed with Ben Folds later performing.
Boston police spokesman David Estrada said police officers on security detail at the hall escorted the men off the property, and no charges were filed. No injuries were reported, he said.
A Boston Symphony spokeswoman did not return a call for comment.