Carrying photos of the slain policeman and banners calling for local institutions to fight crime in the province, the protesters walked in central Pristina and stopped outside the city's court to protest against the justice system.
The policeman, Triumf Riza, was gunned down last week in a parking lot in Pristina, meters (yards) away from an area frequented by youths. The suspected assailant later handed himself in to the police, who have not said anything about the motive.
Kosovo's justice system has frequently come under attack, described as prone to corruption and lenient on criminals and armed gangs.
Reflecting the discontent, an ethnic Albanian protester accused the local authorities of not standing up to criminals in the province of some 2 million people.
"People are being murdered and there are no convictions," said Kadri Jetullahu, 34. "I'm extremely unhappy with the work of our institutions."
Marching alongside the protesters, the province's Prime Minister Agim Ceku said he was walking in a show of "determination to fight crime."
"We have obstacles in justice," said Ceku. "But, we have the will of people and government to fight it."
The top United Nations official in the province, Joachim Ruecker, condemned the killing as a "heinous crime and a terrible tragedy."
"His death has united Kosovo's people against crime like no other event that I have witnessed here," Ruecker said in a statement.
Kosovo is formally a part of Serbia, but has been under U.N. and NATO control since 1999 after the alliance bombed Serb forces and ended the troops' brutal crackdown on separatist ethnic Albanians.