E. Cobham Brewer 18101897. Dictionary of Phrase and Fable. 1898.
(Anglo-Saxon, pening or penig). For many hundred years the unit of money currency, hence pening-monegre (a money-changer). There were two coins so named, one called the greater = the fifth part of a shilling, and the other called the less = the 12th part of a shilling.
My penny of observation (Loves Labours Lost, iii. 1). My pennyworth of wit; my natural observation or motherwit. Probably there is some pun or confusion between penetration and penny of observation or pennorth of wit.
A penny for your thoughts. See Heywoods Dialogue, pt. ii. 4. (See PENNYWORTH.)