Download XAMPP by clicking on More downloads under XAMPP for Windows. Find the latest version. Download the zip or 7z file (not an installer) and unzip it to your portable drive (7-zip is quicker than Windows). Go to your drive and into xampp and run setup_xampp.bat which sets up the Apache Web server to use the correct drive letter. If the drive letter changes (when you use the external drive on a different computer) Apache will no longer work so run that batch file again to fix it.
To test the server run xampp-control.exe which you should find in the XAMPP folder. Once it starts click on the first two Start buttons. If Apache and MySQL go green the server is working. Press the two Stop buttons and then Quit.