You shouldn't be running into memory issues though, I would guess that it is either a server issue, or you have something running into an infinite loop

How would I even begin to go about tracking down the issue?

deactivate all the plugins and then reactivate them one at a time or something?

That's pretty much it unless you have any more meaningful error logs.  If a clean installation still runs in to those memory errors then it's a hosting problem.