Annan's words reinforced the findings released earlier Thursday of a panel of five UN human rights investigators, who condemned the facility as a place of torture and rights abuses that should be closed.
'I cannot agree with everything in the report, but the basic point is that one cannot detain individuals in perpetuity,' Annan told reporters after a lunch meeting.
'Sooner or later there will be a need to close Guantanamo. It will be up to the government to decide. Hopefully, it will be done as soon as possible.'
The United States has detained at least 520 suspects in the war on terrorism since 2002 at its Guantanamo naval facility, without bringing charges or holding trials. Military tribunals were started in October. The practise has been assailed by US and international human rights groups.
In the 54-page report, which will be submitted for formal consideration next month by the UN Human Rights Commission in Geneva, the five investigators called for Guantanamo to be closed 'without further delay' and the transfer of the detainees to 'pretrial detention facilities on United States territory.'
Until then, the US government should revoke 'all special interrogation techniques authorized by the Department of Defence' and refrain from practices of 'torture or cruel, inhuman or degrading treatment or punishment,' they wrote.
Washington says that the report is flawed and one-sided because the investigators refused to visit the facility, located on a corner of Cuba still in U.S. government hands.
The report is a 'discredit to the U.N. when a team like this goes about rushing to report something when they haven't even looked into the facts,' said White House spokesman Scott McClellan. 'All they have done is look at the allegations.'
The UN rapporteurs were given permission by the Pentagon to visit Guantanamo in November, but not to interview detainees. The UN team refused to go because it would have contravened UN principles of human rights investigations.
The investigators charged that the United States has been acting as judge, prosecutor and defence counsel at the US naval base. Most of the detainees were captured in Afghanistan after the US-led coalition ousted the Taliban regime in the wake of the 2001 terrorist attacks against New York and Washington.
Washington has labelled the prisoners 'enemy combatants,' arguing they have no right to trial as prisoners captured in wartime, instead to be detained until the end of combat, but do no deserve the legal status as prisoners of war because they were not serving in a recognized army.
Annan said that 'charges have to be brought' against detainees, who must be 'given a chance to explain themselves. ... It is common under any legal system.'
The report detailed interrogation techniques that met the definition of torture under international conventions, including submitting detainees to conditions and positions of powerlessness to extract information. Force-feeding of hunger strikers was also cited.
The investigators particularly denounced the use of excessive violence, citing photographs that show how detainees were shackled, chained, hooded and forced to wear earphones and goggles. The source of the photos was not clear.
'They also showed beating, kicking, punching, but also stripping and forced shaving' of detainees who resisted, the report said. 'It is of particular concern that some of these violations have even been authorized by the authorities.'
The investigators also mentioned 'plentiful evidence' that prisoners suffered serious mental health problems. <!--page-->
John Bellinger, a US State Department official, told Cable News Network (CNN) that it was 'quite remarkable' that the investigators took the complaints 'without any dose of skepticism.' McClellan called the report a 'rehash of allegations that have been made by lawyers representing some of these detainees.'
'We know that these are dangerous terrorists that are being kept at Guantanamo Bay. They are people that are determined to harm innocent civilians or harm innocent Americans,' he said.
Only the International Committee of the Red Cross, which does not divulge its findings except to the government in question, has been allowed to meet prisoners. Several years ago, however, ICRC concern about psychological devastation to younger prisoners became public.
McClellan said that the US military treats detainees 'humanely, as directed by the president of the United States.'
Revelations of how the United States has treated prisoners in various situations since September 11, 2001 world have damaged the U.S. image abroad and plagued President George W. Bush's government.
Last year, Bush's nominee for attorney general, Alberto Gonzales, spent days before the Senate explaining his involvement in issuing legal opinions that torture only occurred when it causes pain 'equivalent in intensity to the pain accompanying serious physical injury, such as organ failure, impairment of bodily function or even death.'