"Brownies" is an old term referring to elf-like creatures who lived in tree trunks.
When I was a boy my mother game me a book on Brownies. It was full of illustrations of these elves in elaborate tree trunk houses. It's adoption as a mascot by the then Browns owner followed.
The Browns, BTW, were named after their original coach HOFer Paul Brown.