The mysql-client package allows you to connect to a MySQL server. It will give you the "mysql" command-line program.
The mysql-server package allows you to run a MySQL server which can host multiple databases and process queries on those databases. It will give you the MySQL daemon, a.k.a. mysqld.
The "MySQL" package probably includes both of the above.
If you just need to connect to a remote server and run queries, install just mysql-client. If you need to host a database, install the client and server.
The mysql server package will install the mysql database server which you can interact with using a mysql client. You can use the mysql client to send commands to any mysql server; on a remote computer or your own.
The mysql server is used to persist the data and provide a query interface for it (SQL). The mysql clients purpose is to allow you to use that query interface.
The client package also comes with utilities that allows you to easily backup/restore data and administer the server.
