"I'm just relieved," said Young, an 18-year-old who won Junior Wimbledon last month. "This is my 12th match and the first one that I've won."
In another men's match, former Australian Open champion Thomas Johansson of Sweden beat Kristof Vliegen of Belgium 6-2, 6-1.
In the women's draw, American Vania King ousted Italian Tathiana Garbin 6-4, 6-4.
Young broke Delic to go up 2-0 in the second set, then had to survive break points on his next three service games.
"I said to myself, 'I'm winning this one,"' Young said.
When Young broke again at 2-2 in the third set, Delic fired a ball over the roof of the Connecticut Tennis Center in frustration.
Young won the Australian Open junior title in 2005, and earlier this year won a challenger tournament in Aptos, California.
"This was the big goal, winning in the first round (of an ATP event) for the first time," he said.
He will face the tournament's top seed, Nikolay Davydenko in the second round. Young has been granted a wild card to play in the U.S. Open next week.
Later Sunday, No. 5-seeded Patty Schnyder was to face Francesca Schiavone of Italy.