Charles Hanbury-Williams was the youngest son of Major John Hanbury of Pontypool. He was educated at Eton as a contemporary of Fox and Pitt. Following a grand tour of Europe he settled on an inherited estate in England. His early career focused on politics, society and literature - indeed his poetry was very well-received within a social circle that included Horace Walpole. His diplomatic career began when he was in his late thirties.

In 1750 he was ordered to the Prussian court in Berlin at this time relations between Britain and Prussia were highly strained. His role was to discover the King's propensity for war with his neighbours, and to examine the state of his army. He was greeted coolly upon his arrival, and compounded the problem through his own arrogance, to the point that he was snubbed by the Prussian King and his court. His time in Berlin ended with his peremptory recall to Britain, and his diplomatic career was only saved with the help of his friends in the Government.