Although I haven't noticed that per say, it is a possibility when you point it out. It all has to do with who the guests are really. I noticed there have been a lot of new guests who don't entirely fit the format.
I agree about certain "too interesting" guests. There is a fine line between "interesting" and boring trivia nonsense. Rory McGrath was the absolute worst! I remember Stephen had to threaten him because he wouldn't stop chiming in with the latin names of all sorts of animals.