You only need to do this if you have not installed PortableApps.com.
Download XAMPP portable by scrolling down a bit on Sourceforge and then clicking on the latest version in the 5 series (not 7). Download one of the zip files and unzip it to your portable drive. Go to your drive and into xampp or xampplite and run setup_xampp.bat which sets up the Apache Web server to use the correct drive letter. If the drive letter changes (when you use the external drive on a different computer) Apache will no longer work so run that batch file again to fix it.
To test the server run xampp-control.exe which you should find in the XAMPP folder. Once it starts click on the first two Start buttons. If Apache and MySQL go green the server is working. Press the two Stop buttons and then Quit.