No, I don't believe that there is any evidence at all for this and not even for Elizabeth and Henry VIII being involved.
Historians think that this myth of Henry and Elizabeth Boleyn having an affair stems from a confusion between Elizabeth Blount (Bessie Blount), a known mistress of the King, and Elizabeth Boleyn as their names sound quite similar. When Henry VIII heard rumours that he had slept with Elizabeth Boleyn (Elizabeth Howard, Mary and Anne’s mother), he said “Never with the mother”. This alleged relationship was also never mentioned in the dispensation that Henry applied for to make his marriage to Anne legal or when he got his marriage to Anne annulled in 1536 - there was only mention of his relationship with Mary Boleyn.
Elizabeth Boleyn (née Howard) was born in c1480, making her around 11 years older than the King. Although she was known for her beauty and was a lady-in-waiting to both Elizabeth of York and Catherine of Aragon, I cannot see there being any reason to believe the myth of an affair between Elizabeth and Henry.
