I like how it is explained in Idiocracy, stupid people have a lot of kids they can't care for, so they become stupid kids. These stupid kids go out and knock up their stupid high school lovers and quickly produce more stupid kids. Smart people have maybe one or two, who are well taught and well mannered and go on to have one or two themselves at a more appropriate age. Not only does this takes more time for the genes to be passed along, but it's to a generation no bigger than the one before it.

With a ratio like that, smart people are becoming extinct.