Our sister site urban75 hosts one of the UK’s biggest online communities, and has just switched from the popular vBulletin forum software to the new XenForo system.
With over 10.3 million posts and 40,000 users notched up, this was not a decision to be made lightly, but growing dissatisfaction with both vBulletin’s software and their company ethos made the switch a matter of priority.
Here, the site’s technical officer explains in some detail how the move was made, and the technical issues encountered:
Migrating what?
First a bit about the urban75 boards. Based in Brixton, London, UK, urban75 is one of the busiest, non-profit community sites around, featuring huge photography sections, bulletin boards, live chat, drug information, football, direct action info and more. The boards are entirely non-profit and carry no advertising. The site is primarily funded by donations from board users (there are no subscriptions).
As a result, we’re not running on particularly flashy hardware, upgrading the server using spare equipment our hosting provider has available.
vBulletin Days and Nightmares
We have been running vBulletin for almost a decade and it served us well for much of that time.
In August 2010 we upgraded to vBulletin 4. That upgrade was, frankly, a nightmare.
I spent many, many hours trying to optimise the database, tracing slow queries and following bug reports. I also ended up installing Sphinx search, even though vBulletin seemed very slow at handling the results it got back from Sphinx.
At first there were many complaints, some about things which we could fix, and others about things which eventually people just got used to.
xenForo is announced
When xenForo was announced, several of the moderators on the boards went over to the community on xenforo.com and were impressed by what they saw.
I speculatively purchased a xenForo license when they first became available, thinking I could use it myself even if it turned out to be unsuitable for Urban75.
Following the purchase of the license, I installed a test copy on our little test server, and invited the moderators over for a look.
I think it’s fair to say everyone was blown away by the functionality and there was soon a consensus that we should look to move the main Urban75 boards to xenForo.
Testing Times
We weren’t going to go live with a pre-1.0 version so we waited. We invited a number of users over from the main boards to take a look at xenForo and feedback their thoughts. That lasted a couple of weeks with lots of positive feedback.
When 1.0 was released I started further investigations into the migration, checking what functionality was missing and what add-ons would be useful.
Using Paul M’s vBulletin 4 Importer I ran through an import from a year-old backup of the main boards, importing around 9 million posts without a hitch. On our test server the import took just over 24 hours.
There was no way I was going to use the xenForo built-in search on that size of message base so I installed the Sphinx executable and mlx’s Sphinx Search Engine add-on, followed by a few hours configuring and indexing using Sphinx.
Running some test searches showed results from the xenForo boards on the test server coming back much faster than they did on the vBulletin live server. Both were using Sphinx so that indicated something seriously bad was going on with vBulletin displaying search results.
I tried a few other extras, including Darkimmortal’s vB4 URL Redirection add-on, the Tapatalk and ForumRunner mobile add-ons, and the Stop Forum Spam and Multiple Account Detector add-ons.
Preparing for the Migration
Work got in the way of planning a full migration for a while, but eventually I put a stake in the ground, choosing the weekend of the anniversary of our vBulletin 4 upgrade.
With the help of other moderators, we got a couple of styles sorted out, one very similar to the default xenForo style and another variation without avatars which had been a particular request.
In preparation, I installed the xenForo software on the live server under a different directory from the existing vBulletin, and did some of the config under a single Administrator user.
Here We Go!
The actual night of the migration arrived and I closed the vBulletin site at 23:59 on 5th August 2011. We have a small phpBB installation which we open for times when the main boards are down and I put a link to that on the “Site Closed” message.
First things first, a full MySQL dump of the vBulletin database and the embryonic xenForo database, just in case.
Then the first step of the migration, using the latest version of the vBulletin 4 importer.
The user importer fell over quite quickly, complaining about “Duplicate entry for key 2″ for a user named “Ęllie”. Scratched by head for a few minutes before gently renaming the user and re-starting the import. A few minutes later I hit another user with the same issue, this time called “‘Mått”. Same fix for that one and the rest of the users imported fine.
I started the largest part of the import, copying the Threads and Posts at 01:55 on Saturday morning. By 02:30 it was obvious this was going to be a very long job so I went to bed hoping that the import might speed up while I was asleep. Rising at 06:30 I found that it hadn’t got any faster and that my original estimate of a much faster import on the live system was sadly wrong.
With me monitoring progress through the day, and updating the users via the phpBB boards and the “Site Closed” message, the posts finally finished importing at about 21:30. The importer then runs through a cache update for users and threads.
A dash through configuring Friendly URLs, moving the Sphinx config to index the xenForo boards instead of the vBulletin ones, checking basic user permissions and configuring a few more add-ons took me until 23:30 when we finally re-opened the boards.
So 23.5 hours start to finish for 10.3 million posts and I understand that we’re now the largest UK site running xenForo.
Aftermath
It took me a little while afterwards to get the vB4 Redirection working properly, and we’ve had a slow-query issue with another add-on which the author is hoping to fix. My thanks to Kier for helping track down where that query was coming from despite my still-frazzled support request.
The users seem generally pleased with the results, though there are a number of complaints about relatively minor pieces of functionality missing, dislike of change, dislike of the colours along with some genuine issues with useful functions missing that were previously available.
Some of these are already addressed in xenForo 1.1, other might be later, and some may never get fixed.
From a tech admin point of view, xenForo is much lighter on the hardware, we’ve hit more than 1,000 users online already which is close to the highest we every had under vBulletin and the only issue has been with the add-on I mentioned earlier.
My thanks to all the folks at xenForo for coming up with what has to be the neatest bit of forum software around. Also thanks to the authors of all the add-ons we’re using, and to all the xenForo community members for sharing their experiences and supporting such a great product.
It’s been a long journey, but we’ve reached our first xenForo milestone. May there be many more in the future!
Board Stats as of 8/8/2011
Discussions: 278,346
Messages: 10,346,335
Members: 40,492
Test System
CPU: One core of 2.133GHz Xeon X3210
RAM: 1.5GB
OS: Ubuntu Linux 10.10 virtual machine running on vSphere 4.1 host
Disks: Virtual disk stored on 3ware-controlled RAID1 pair of 250GB SATA drives
Live System
CPU: Intel Quad Core Q9300 @ 2.50GHz
RAM: 4GB
OS: FreeBSD 7.2
Disks: Adaptec-controlled RAID1 pair of 150GB WD Raptor SATA drives
I quite like it, it is a lot more functional than vbulletin.
I’m surprised at the low physical requirements; I guess bandwidth is another matter.
Welcome to xenForo, from another London-based community, South East Central. Ours being pretty damn tiny compared to urban75, it was really interesting (and frankly cold sweat-inducing) to read about your import process! It’s nice to know we’re in good company using xF and all the best with future enhancements.
No chance of going back to vB, then? 🙁