Earlier this month senior judges overturned the original decision, describing it as ‘perverse’, and ordered the case to be re-heard by the first-tier tribunal.
It means Spence could still win his case in a future re-hearing.
His previous legal challenges were allowed by junior judges despite Spence’s lawyers indicating he is ‘primarily sexually attracted to adults’, which judges said showed he has ‘at least a residual or secondary sexual attraction to children’.
The child sex offence took place in the United States in 2008, when he was 25 years old.
He was convicted of ‘lewd and lascivious conduct with a victim aged under 16’ in a nightclub in Saint Lucie, Florida.
Spence told the Home Office: ‘I started partying with a female and she was underage.
‘I can’t take it back, but I have learned from it.
‘It was just one night in a bar. I should have just asked questions instead of partying.’
Shadow home secretary Chris Philp, who this week attacked judges’ ‘tyranny’ over the immigration system, described the latest case as ‘deeply perverse’
He was jailed for three years and then deported from the US to Jamaica after serving half his sentence.
Judges in the first appeal found Spence ‘has been sexually attracted to children and has pursued relationships with children in the past’.
They added: ‘The appellant’s attempts to downplay his sexual involvement with children and his lack of candour about the nature of his offending in my view raise doubts over whether he has truly addressed his sexual attraction to children.’
But they noted there had been no further sexual convictions and there was ‘no evidential basis’ to show he was still pursuing sex with children, and granted his human rights appeal.
His wife and daughter are both British citizens and have always lived in the UK.
Spence has never lived in Britain, although the family have maintained contact by telephone and met during holidays abroad, court documents showed.
In the Court of Appeal, judges noted that Spence’s relationship with his now wife began when she, too, was a child.
Giving the leading judgment, Lord Justice Lewis said: ‘The fact that he is primarily attracted to adults must mean that the first-tier tribunal considered that he has, at least, a residual or secondary sexual attraction to children.’
The junior tribunal’s decision to grant Spence’s case was ‘frankly perverse’, he added.
The judges granted the Home Secretary’s appeal and ordered a fresh hearing by the first-tier tribunal.
Mr Philp said: ‘This is yet another deeply perverse example of unelected judges trying to override elected ministers to allow a Jamaican child sex offender into the UK – after the US had rightly kicked him out.
‘The tyranny of the judiciary continues.
‘We’ve seen foreign murderers, rapists, drug dealers and paedophiles allowed to stay in the UK by judges using interpretations of the ECHR that defy the will of Parliament and defy common sense.
‘That’s why, as I explained this week, we must leave the ECHR, abolish the immigration tribunal and end the power of the courts over immigration.
‘Deciding who gets to enter or remain in the UK should be in the hands of our elected parliament – not unelected judges.’
A Home Office spokesman said: ‘We will do everything in our power to continue contesting this case.
‘We strongly believe this individual’s presence in the UK is not conducive to the public good.
‘We will not allow foreign criminals to exploit our laws.
‘The Home Secretary has been clear that anyone settling in the UK must have a clean criminal record.’