1 - Check $CONFIG->wwwroot in your /elgg-config/settings.php

2 - Check RewriteBase parameter in your .htaccess

3 - 'Flush the caches' via Administration is recommended

The configurations look OK.

Can I delete the cache manually (looks like a single file with timestamp as name) or would that break some functionalities?

Can I delete the cache manually (looks like a single file with timestamp as name) or would that break some functionalities?

It's not a single file but directory located at /your-elgg-data-path/caches

You can temporarily disable simple and system caching in Site Settings. Then run Upgrade.
This should completely remove all the old cache.

But I don't think the cache is the reason.

Uncaught Error: elgg/Ajax cannot be used with external URLs

It looks like the CORS errors.

We use these headers in some project:

Header add Access-Control-Allow-Origin "*" 
Header add Access-Control-Allow-Headers "origin, x-requested-with, content-type" 
Header add Access-Control-Allow-Methods "PUT, GET, POST, DELETE, OPTIONS" 
Header add Access-Control-Allow-Credentials "true"

Please Google to learn more about CORS issues