Not just any form of sexual activity, but genital orgasm, the only form of sexual activity Freud considered normal (He was as much of a prude as those people who regard as normal only sexual activity between a man and a woman who are married). A desire to engage in an activity because we find it pleasurable in itself, whether or not it culminates in genital orgasm, he considered perverse. That’s why he called the infant polymorphous perverse.
Those Victorians who regarded children as sexually ignorant/innocent were closer to the truth than Freud was, because children explore their bodies in the same spirit of curiosity they explore everything. The world is new to them, and they take pleasure in all of it. That pleasure is not specifically sexual, much less specifically genital. It’s only when we become adults that we lose interest in the world and decide that Freud was right; genital sex is the only pleasure in life. Children know better.