Mayne states that George Gordon Byron was born on 22 January 1788 in a house on 24 Holles Street in London. However, R.C. Dallas in his Recollections states that Byron was born in Dover. He was the son of Captain John "Mad Jack" Byron and his second wife, the former Catherine Gordon (d. 1811), a descendant of Cardinal Beaton and heiress of the Gight estate in Aberdeenshire, Scotland. Byron's father had previously seduced the married Marchioness of Caermarthen and, after she divorced her husband, he married her. His treatment of her was described as "brutal and vicious", and she died after having given birth to two daughters, only one of whom survived: Byron's half-sister, Augusta. In order to claim his second wife's estate in Scotland, Byron's father took the additional surname "Gordon", becoming "John Byron Gordon", and he was occasionally styled "John Byron Gordon of Gight". Byron himself used this surname for a time and was registered at school in Aberdeen as "George Byron Gordon". At the age of 10, he inherited the English Barony of Byron of Rochdale, becoming "Lord Byron", and eventually dropped the double surname.
Byron's paternal grandparents were Vice-Admiral the Hon. John "Foulweather Jack" Byron, and Sophia Trevanion. Vice Admiral John Byron had circumnavigated the globe, and was the younger brother of the 5th Baron Byron, known as "the Wicked Lord".
He was christened, at St Marylebone Parish Church, "George Gordon Byron" after his maternal grandfather George Gordon of Gight, a descendant of James I of Scotland, who had committed suicide in 1779.
"Mad Jack" Byron married his second wife for the same reason that he married his first: her fortune. Byron's mother had to sell her land and title to pay her new husband's debts, and in the space of two years the large estate, worth some £23,500, had been squandered, leaving the former heiress with an annual income in trust of only £150. In a move to avoid his creditors, Catherine accompanied her profligate husband to France in 1786, but returned to England at the end of 1787 in order to give birth to her son on English soil. He was born on 22 January in lodgings at Holles Street in London.
Catherine moved back to Aberdeenshire in 1790, where Byron spent his childhood. His father soon joined them in their lodgings in Queen Street, but the couple quickly separated. Catherine regularly experienced mood swings and bouts of melancholy, which could be partly explained by her husband's continuing to borrow money from her. As a result, she fell even further into debt to support his demands. It was one of these importunate loans that allowed him to travel to Valenciennes, France, where he died in 1791.
When Byron's great-uncle, the "wicked" Lord Byron, died on 21 May 1798, the 10-year-old boy became the 6th Baron Byron of Rochdale and inherited the ancestral home, Newstead Abbey, in Nottinghamshire. His mother proudly took him to England, but the Abbey was in an embarrassing state of disrepair and, rather than live there, decided to lease it to Lord Grey de Ruthyn, among others, during Byron's adolescence.
Described as "a woman without judgment or self-command", Catherine either spoiled and indulged her son or vexed him with her capricious stubbornness. Her drinking disgusted him, and he often mocked her for being short and corpulent, which made it difficult for her to catch him to discipline him. She once retaliated and, in a fit of temper, referred to him as "a lame brat".
Upon the death of Byron's mother-in-law Judith Noel, the Hon. Lady Milbanke, in 1822, her will required that he change his surname to "Noel" in order for him to inherit half of her estate. He obtained a Royal Warrant allowing him to "take and use the surname of Noel only". The Royal Warrant also allowed him to "subscribe the said surname of Noel before all titles of honour", and from that point he signed himself "Noel Byron" (the usual signature of a peer being merely the peerage, in this case simply "Byron"). It is speculated that this was so that his initials would read "N.B.", mimicking those of his hero, Napoleon Bonaparte. Lady Byron eventually succeeded to the Barony of Wentworth, becoming "Lady Wentworth".