How to: Archive a phpBB forum using Wget and preserve styling

NOTE: This post is based on https://ibboard.co.uk/Programming/how-to-archive-phpbb.html but adds some extra steps which I found necessary to make it work best.

Recently, I took it upon myself to archive the MultiWii forums as part of the Drone Pi project I’m currently working on (more details and instructions coming soon!). I did this because the whole site keeps going up and down and I’m not sure how long the forum will remain available.

NOTE: This tutorial assumes you’re using the default “prosilver” theme. It may work with other themes as well, but I cannot verify/guarantee this.

Step 1: Getting Wget

Wget is usually available out of the box on Linux. If not, install it using your package manager.

These steps assume you’re running Linux, but the scripts provided may also work on macOS or on Windows under Cygwin (http://www.cygwin.org).

Step 2: Downloading the useful pages

I used the command from the IBBoard tutorial for this:

wget -m -p -np -R "*sid=*,ucp.php*,memberlist.php*,*mode=viewprofile*,*view=print*,viewonline.php*,search.php*,posting.php*" https://forums.example.com

Please be aware this it might take a very long time to run this command – be ready to leave it going overnight (or in my case, several days). It going slowly is no bad thing because it stops the web server from being overwhelmed.

Note: Replace forums.example.com with the web address of the actual forum you want to archive.

This command ensures that only useful pages are downloaded, and only downloads pages from the forum section of the site – if there are other sections they won’t be downloaded, saving time and bandwidth. You may want to make a backup copy in case any of the next steps go wrong.

Step 3: Moving phpBB archived files into place

You’ll now have a complete copy of the forum, but you can’t use it in its current form. You’ll need to download the archived forum scripts from IBBoard (https://ibboard.co.uk/Programming/archive-phpbb.zip) and do the following.

  • Enter the folder where you downloaded all the pages (we’ll call this the download folder).
  • Make a new subfolder called “archive” and move all the top-level .php (such as viewtopic.php* and viewforum.php*) files in there.
  • Extract the archived forum scripts into the download folder, accepting any questions about overwriting files.

Step 4: Making the CSS work

At this stage your copy is nearly ready, but none of the styling will work, which makes it difficult and annoying to use the archived copy. To fix this, first download the script I wrote to fix the stylesheets and javascript files from here.

Then do the following:

  • Enter the download folder where you archived the forum.
  • Hunt around in the “styles” and “assets” folders for any .css and .js files.
    • When you find them, you’ll notice they have ?s in the file names. You need to change the file names to remove the ?s and all the characters afterwards.
    • ^ so “font-awesome.min.css?assets_version=18” becomes “font-awesome.min.css”
    • Do this for all the .css and .js files.
  • Now put my script in the download folder and run it using the terminal
bash phpbb_fix_css_js.sh

NOTE: Depending on how big the forum you archived was, this might take a very long time to run – around an hour at most.

Step 5: Create symbolic links so the topics can be accessed by ID

Run the following script in the terminal from IBBoard:

for file in viewtopic.php\?f\=*\&t\=*; do
	if [ ! -L "$file" ]; then
		f=$(echo "$file" | grep -oP "(?<=f=)[0-9]+")
		grep -oP '(?<=#p)[0-9]+' "$file" | while read post; do
			ln -s "$file" viewtopic.php\?f\=$f\&p\=$post
			ln -s "$file" viewtopic.php\?p\=$post
		 done
	fi
done

Step 6: All Done!

Your phpBB forum archive is now complete, and the pages should all look exactly the same as they did before. To use it, you’ll need to put these files on a webserver or run a webserver on your local PC (this is what I’m doing as my archive isn’t public – at least not yet).

Summary

This has been a longer and fairly complicated post, but there are very few tutorials for this around, so I thought it was worth writing one. I hope you find it useful if you ever find yourself needing to archive a phpBB forum.

That’s all for now, but stay tuned for some Pi Drone-related posts coming soon!

Tagged , . Bookmark the permalink.

About Hamish McIntyre-Bhatty

I'm currently studying for a Computing and IT degree with the Open University, and am a software developer as well. I enjoy coding in Python, C++ (still learning), and Java. Having written 4 open-source programs (hosted on launchpad.net), set up my own website, and started volunteering at Wimborne Model Town to work on their river control system, I still find the time to enjoy cycling, acting, photography, and playing bass guitar. I go climbing every now and then as well. More recently, I wrote my first commercial program, Disk Verifier, and created a PSID Unlocker GUI for Parted Magic. I also create tutorial and other informative videos for Parted Magic LLC.

Leave a Reply

Your email address will not be published. Required fields are marked *