You can use "maximum entropy" arguments to convince yourself or others that a Gaussian distributional assumption is the best (e.g. if the only thing you know is

You can use "maximum entropy" arguments to convince yourself or others that a Gaussian distributional assumption is the best (e.g. if the only thing you know is that your distribution has support over ]-inf,+inf[ and that its variance is bounded, the maximum entropy distribution is a Gaussian) and, in such a case, the sample arithmetic mean will give you the maximum likelihood estimate of the population "location" parameter (and, yes, along with variance, it will contain all required information to perfectly summarize your sample, since these are sufficient statistics, under a Gaussian assumption). TL;DR: If you can safely assume additiveness/normality, then... yes, mostly. Otherwise, not necessarily.