Rusmajli denied any impropriety but said he was leaving the post to facilitate a judicial investigation into the allegations.
A spokesman for the government had no immediate comment on the resignation or on whether it had been accepted.
Rusmajli, 42, had been in the post since April.
His resignation follows claims by the head of Albania's prison service, Saimir Shehri, that Rusmajli was promoting his brother's construction company for a bid to build a new detention facility in the Albanian city of Durres.
Shehri resigned his position Tuesday after making the accusations.
Albania's judiciary has been rattled by recent allegations of corruption, including claims that convicted criminals had secured illegal early release from prison.
Earlier this month, parliament voted to oust the country's top prosecutor for allegedly failing to fight organized crime gangs. The decision must be ratified by President Bamir Topi in order to take effect.