The manga series has been out a long while before the anime? As someone reading this series and watching the show, both are top tier series. People must remember in show adaptions, the development can diverge to either follow the Manga or Light Novel. This manga is a brilliant realistic take on sword combat which is on track to becoming one of my favorites. On the other hand, the show itself is really good even if it's pacing is fairly different from what most were expecting. Of course, the problem with the show is the same issue with most Isekai light novel/manga adaptions. They SKIP a lot ofthe original scenes which are priceless while focusing on main story things. That's why on every single episode premiere of the show, you always see comments on literally site saying the show is lackluster.

The show was unable to replicate the epicness here, as the goal was to appeal to universal watchers of a broader audience.

Do you need to read the manga to watch the anime?
Honestly no, but the manga has several valuable/important scenes the show was forced to skip due to time contraints.

This is the same issue with RE:MONSTER, the manga itself is a legendary Isekai masterpiece between LN and manga. Sadly since the show killed the pacing with skipping, it will forever be known as another MID Isekai series