The Wallace Collection was definitely a highlight of this London trip, and I can't believe I have never heard of it before!
The Collection is housed at Hertford House in Manchester Square, which is the former townhouse of the Seymour family, Marquesses of Hertford. It's a little off the beaten path, just north of Oxford St, but very a walkable distance from the shops.
The house is beautiful and full of some very well known pieces of art. And... it's free to go in!