Access permissions for Annotations...

Is there a reason that annotations have a seperate access level to the entity that owns them? It's just if you edit an entity and change it's access level then any existing annotations may not display. Is there a way to change this behaviour or a specific reason for it?

If a user makes an entity public yet none of the comments go public, it seems a bit backwards.