Page History

...

While there is no harm in keeping extra, unmodified component directories in your theme, it can be beneficial to remove component directories which are unchanged. 

The main advance advantage to keeping your theme simple/small is that it can make future upgrades easier.  Generally speaking, the fewer components you have in your theme, the less likely your theme will need modification in a future upgrade (as generally your theme might may require updates if one of the components in your theme had it references underwent structural/larger major changes). 

You can remove a Component directory as follows:

...