The three were hanged Thursday, according to a justice ministry official who spoke on condition of anonymity citing policy. He refused to disclose the names of the executed.
Kyodo News agency identified the three as Hifumi Takezawa, 69, Yoshio Iwamoto, 63, and Kozo Segawa, 60. The three had been convicted of multiple murders, and Iwamoto and Segawa were also convicted of robberies, according to Amnesty International Japan.
Japan is one of the few industrialized nations that retains the death penalty. The government is routinely criticized by human rights activists for the extreme secrecy surrounding the executions, which are conducted by hanging.
"We strongly condemn the executions," Amnesty International Japan said in a statement. "We hope Japan will ... take a step in the near future toward abolishing the death penalty, which is an extreme form of human rights violation."
Four men were executed on Christmas Day under Justice Minister Jinen Nagase. In April, three more were executed.
Thursday's executions bring the number under Nagase to 10 - the highest since 1993, when executions resumed after a moratorium of more than three years, according to Amnesty International Japan.
Executions are often carried out when parliament is not in session or on Fridays. Critics say that is meant to avoid fueling public debates on the death penalty, which has been abolished in many developed countries except for Japan and the United States.
Thursday's executions came as parliament was in recess. Prime Minister Shinzo Abe is expected to reshuffle his Cabinet next week.
It is believed that 104 inmates are currently on death row in Japan.