Originally Hermione Youlanda Ruby Clinton-Baddeley, she was born in Broseley, Shropshire, England. A descendent of British Revolutionary War general Sir Henry Clinton, she and her older sister (the actress Angela Baddeley of Upstairs, Downstairs fame) moved in elevated social circles, Hermione's first husband being the Hon. David Pax Tennant, a descendant of William the Conqueror and elder brother of Stephen Tennant. Hermione was known for standout supporting performances in such films as Mary Poppins (as Ellen, the maidservant), The Belles of St. Trinian's, The Unsinkable Molly Brown, Passport to Pimlico, The Pickwick Papers, Tom Brown's Schooldays and A Christmas Carol, although she first began making films back in the 1920s. She was a principal character in Brighton Rock (1947).

Her television roles brought her increased visibility; besides many guest appearances she became known to American TV audiences for her roles in Little House on the Prairie and Maude. She was also a sought-after voice-over actress (The Aristocats, The Secret of NIMH).

She continued to work until shortly before her death at 79, in Los Angeles, California, of a stroke.