Tokyo filed the complaint with the Hamburg-based International Tribunal for the Law of the Sea, Kyodo News agency reported, citing unnamed Japanese Foreign Ministry officials.
Foreign Ministry officials could not immediately comment on the report late Friday.
A Russian border guard vessel captured the Hoshin Maru 88 with 17 crew members on June 3 and the boat was taken to a port in Petropavlovsk-Kamchatskiy on the Kamchatka Peninsula, the ministry has said.
The border guard had been investigating the fishing boat over discrepancies between the actual catch and declared amount, according to the ministry.
Earlier Friday, Foreign Minister Taro Aso said that Japan has pressed for the release of the crew, along with another boat, the Tomi Maru No. 53, held since last November in accordance with international law.
"At this point we have not received a response, so we are at the point of filing a complaint in keeping with that law," Aso told a regular news conference.
Russia has seized several Japanese fishing boats over alleged poaching since last year, mainly around a string of islands that the two countries have been disputing for some 60 years.
The disputed islands, called the southern Kurils by Russia and the Northern Territories by Japan, was seized by the Soviets near the end of World War II.
The row has prevented the two countries from signing a peace treaty formally ending World War II hostilities.