The group members view in the sidebar is mod/groups/views/default/groups/sidebar/members.php. You could override this view with a modified version (http://docs.elgg.org/wiki/Overriding_a_view). If you want only admins to see it then capsule the code within an if-clause:

if (elgg_is_admin_logged_in()) {

<original code>

}

Then normal users / group members won't see the member's list in the sidebar anymore.