You're right both. The pop-up window showing more smileys needs more time to load for every new smiley being added. Of course it will pop-up faster if used regularly and therefore the smileys still being saved in the cache of your browser.
As far as I'm concerned I never use that "more"-window to select a smiley by clicking on it anyway but just automatically type the shortcut while typing my post.
Perhaps this means, I'm too long at Klickywelt board for knowing all the shortcuts of my favorites by heart...
But, as Sylvia says: for everybody using the click-selection it's obviously much easier to find one from a small list of smileys than from a bunch of hundreds of them.