Well, you can have mysql for learning purposes even if the page you'll be hosting in the server won't use it.
As being a database server, doesn't mean that you need to use it for web development, you can use it to store/receive data from other sources as well, but, isn't mandatory of course, if you aren't going to use it at all there's no point in installing it.
As for cli use, the mysql bundled client es good enough to manage, to me however it's best to use gui tool when you have to work with very big databases or large result sets, not that you can't do it in the client but using gui tools makes it easy to not get lost