If you removed every perverted thing in this series it would still be great. You can think about it like the perverted stuff is a running gag. The setup with a serious protagonist and girls around him is bog standard but the thing is that its so well done with great backgrounds, character design and good and funny dialog it does not feel like every other harem show. The first episodes have a issue with too fast pacing and the more serious chapters of the manga all have at least one pervy scene crammed in there when that is unnecessary. I'm wondering if the mangakadidn't want to make an Ecchi but had his original draft rejected. I saw some Pokémon fanart he made and that explained a lot of were he got his art style from.