Set up a localhost LAMP solution. XAMPP/MAMP are fine. I recommend placing a symlink to htdocs in your home dir. E.g. on OSX/XAMPP: cd ~; ln -s /Applications/XAMPP/xamppfiles/htdocs

Create a github account, set up git, and register your SSH key with github.

Fork the Elgg/Elgg repo into your account.

Create a separate Elgg install for each major Elgg branch you want to work on. This is because you can't downgrade an Elgg installation once installed/upgraded. Here's an example for 1.11:

cd ~/htdocs
git clone git://github.com/Elgg/Elgg.git elgg111
cd elgg111
git checkout 1.11
git remote add fork git@github.com:yourGithubUsername/Elgg.git 

The Elgg 1.11 codebase is now in ~/htdocs/elgg111. Continue to install Elgg as normal, giving it its own data directory and database for 1.11.

You should end up with a working Elgg 1.11 install at http://localhost/elgg111/

Now you have all you need to start bug fixing!