Bob King, the U.S. special envoy for North Korean human rights, was due to travel Friday from Tokyo to Pyongyang to request a pardon and amnesty for Kenneth Bae, and return the next day.
Bae was sentenced in April to 15 years of hard labor by the authoritarian state, which accused him of subversion.
Department spokeswoman Marie Harf said the U.S. is "surprised and disappointed by North Korea's decision" and remains gravely concerned about Bae's health.
King intends to return to Washington from Tokyo on Saturday.
It's a further setback to U.S.-North Korean relations, which already are severely strained by concerns over Pyongyang's nuclear program.
"We have sought clarification from the DPRK about its decision and have made every effort so that Ambassador King's trip could continue as planned or take place at a later date," Harf said in a statement, referring to the country's formal title of the Democratic People's Republic of Korea.
"We remain gravely concerned about Mr. Bae's health and we continue to urge the DPRK authorities to grant Mr. Bae special amnesty and immediate release on humanitarian grounds," Harf said.
The North Korean diplomatic mission at the United Nations in New York declined to comment Friday.
Bae, a 45-year-old tour operator and Christian missionary, was arrested last November and accused of committing "hostile acts" against North Korea. He suffers from multiple health problems and was recently hospitalized.
On Wednesday in Tokyo, King had cautioned that Washington had received no guarantees from Pyongyang that Bae would be freed.
The visit by King would have been the first public trip to North Korea by an administration official in more than two years and could have provided an opening for an improvement in relations.
North Korea has previously used detained Americans as bargaining chips in its standoff with the U.S. over its nuclear and missile programs. Multination aid-for-disarmament talks have been on hold since 2009, and efforts by Washington to negotiate a freeze in the North's nuclear program in exchange for food aid collapsed 18 months ago.
Bae is at least the sixth American detained in North Korea since 2009. The others were eventually allowed to leave without serving their terms, with some releases coming after prominent Americans, including former Presidents Bill Clinton and Jimmy Carter, visited North Korea.
The last one to be freed was Eddie Jun. Jun, a Korean-American from California, was arrested for alleged unauthorized missionary work during several business trips to the country. He was brought back to the U.S. when King last visited Pyongyang in May 2011.
Alexandra Olson at the United Nations contributed to this report.
Copyright 2013 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.