Well, you tend to undermine your argument with the illustrations in the book you cite, lots of bright colours there. And I'd say that the owner of the 3666 would have been wealthy enough to dress his servants in the colours of the klickies in the set. Clothing was how you displayed your wealth and, especially after the Black Death, wealth shifted downwards to a wider and, therefore, a more colourfully dressed population to some degree, hence the Sumptuary Laws in England.
As for the house element of the 3666 set, over time castles became more domesticated, as it were, as society became more settled so a mix of domestic and military architecture came about. It may be unusual to have a half-timbered structure so prominent in a castle, but you could certainly find them within the castle enclave.
And let's remember, Playmobil's a toy.