Rearrange sidebar

I'd like to prioritize the default elements of the sidebar , but cannot find where this is defined. I know for a plugin you can extend the sidebar view and give priority, but all default sidebar elements have prio 500, so a plugin is either on top of all, or below the default elements.