A string of athletes - including golfer Rory McIlroy and tennis ace Milos Raonic - have pulled out of the Games due to the mosquito-borne virus.
It can cause birth defects in babies, including tots being born with tiny heads.
Dr Allen, 49, said: “The worry is that with the Rio Olympics less than a month away many athletes and travelling fans could be exposed and might carry this disease to other parts of the world.
“We are now seeing an unusually high number of babies being born with small heads in this region.
“It is now recognised as a serious complication of the Zika virus disease.
“And it is also reported that it can be transmitted from a carrier to a partner through sex.”
The first female-to-male transmission of the disease through unprotected sex was seen in the US last week. A woman in her 20s who returned home from a Zika-hit area infected her partner.
More than 1,600 cases have been confirmed in Brazil and it has spread to Central America and the Caribbean. There is no known cure for Zika, which has affected thousands of mums and babies in South America.
Dr Allen, joined a team of medics on a 1200-mile expedition across South America, which has seen more than 35,000 cases of Zika, earlier this month.
His group was catching mosquitoes and assessing mothers and children in the run-up to the Games which start on August 5.
Researchers at the University of Sao Paulo claim the Olympics would result in no more than 16 additional cases of the disease.
But the World Health Organisation (WHO) declared Zika’s renewed spread a global health emergency when it started becoming clear that the mosquito-borne virus was causing severe birth defects last November.
It has now reached 39 countries in Latin America and the Caribbean and is also spreading in south east Asia and the Pacific.
More than 200 scientists asked for the Games to be moved or postponed over Zika last month.
In an open letter to WHO, the scientists argued new scientific findings made it “unethical” for the Games to go ahead.
But Olympic chiefs stressed the risk of catching the virus in August remains low.
