Police in Hiroshima prefecture (state) are working with U.S. military officials to investigate the alleged incident, which the woman reported to police, top government spokesman Nobutaka Machimura told reporters.
"It would be unforgiveable if this turns out to be true," Machimura said.
Hiroshima prefectural police spokesman Chikara Nakata confirmed that they were looking into "the incident that has been reported in the media," but refused to discuss any other details.
The woman allegedly met the men, said to be Marines from the nearby Iwakuni Marine Corps Air Station, at a restaurant or bar in Hiroshima early Saturday morning, local media reports said, citing unnamed local police officials.
The men allegedly took the woman outside to a car and drove her to a parking lot around 2 kilometers (1.2 miles) away where they raped her, according to the reports.
The U.S. Embassy said in a statement that it was "aware of the reported incident" and, together with U.S. Forces Japan, was cooperating fully with Japanese authorities. "We take reports of this nature very seriously," it added.
USFJ spokesman Master Sergeant Terence Peck made identical remarks.
Neither officials nor the media reports named any of the people involved in the alleged incident.
The U.S. has about 50,000 troops based in Japan under a bilateral security treaty. Many Japanese complain of crime, pollution and noise associated with the bases.
The rape of a schoolgirl in Okinawa, which has the largest U.S. military presence, by three American servicemen sparked large-scale protests in 1995.
Hiroshima is 687 kilometers (429 miles) southwest of Tokyo.
The discovery of the submarine has unveiled a few "inconsistencies." For example, how can one explain the fact that the sub was found where it needed to be searched for from the start?
When on a state visit to Singapore, Russian President Vladimir Putin promised to revisit the discussion of the 1956 Declaration between the USSR and Japan regarding the issue of the peace treaty with Japan
The TurkStream, which runs along the bottom of the Black Sea from Russia's Anapa to Turkey, will consist of two lines, each with a capacity of 15.75 billion cubic meters of gas a year