In May 1536 Wyatt was imprisoned in the Tower of London for allegedly committing adultery with Anne Boleyn. He was released from the Tower later that year, thanks to his friendship or his father's friendship with Thomas Cromwell, and he returned to his duties. During his stay in the Tower he may have witnessed not only the execution of Anne Boleyn (19 May 1536) from his cell window but also the prior executions of the five men with whom she was accused of adultery. Wyatt is known to have written a poem inspired by the experience, which, though it stays clear of declaring the executions groundless, expresses grief and shock.
In the 1530s, he wrote poetry in the Devonshire MS declaring his love for a woman; employing the basic acrostic formula: the first letter of each line spells out SHELTUN. A reply is written underneath it, signed by Mary Shelton, rejecting him. Mary, Anne Boleyn's first cousin, had been the mistress of Henry VIII between February and August 1535.
Around the year 1537, he took Elizabeth Darrell as his mistress. Elizabeth bore Wyatt three sons, Henry (who died in early infancy), Francis (born in 1540 and took the surname of Darrell), and Edward, who was later executed for his part in the Wyatt's Rebellion of 1554, led by his legitimate half-brother, Sir Thomas Wyatt the Younger. Wyatt left Elizabeth properties in Dorset.
By 1540 he was again in favour, as evident by the fact that he was granted the site and many of the manorial estates of the dissolved Boxley Abbey. However, in 1541 he was charged again with treason and the charges were again lifted—though only thanks to the intervention of Henry's fifth wife, Queen Catherine Howard, and upon the condition of reconciling with his wife. He was granted a full pardon and restored once again to his duties as ambassador. After the execution of Catherine Howard, there were rumours that Wyatt's wife, Elizabeth, was a possibility for wife number six, despite the fact that she was still married to Wyatt. He became ill not long after, and died on 11 October 1542 around the age of 39, while staying with his friend Sir John Horsey at Clifton Maybank House in Dorset. He is buried in nearby Sherborne Abbey.