CZ:Proposal for cost-reducing hosting plan
Because our hosting cost of $320 per month is much greater than our small community can consistently sustain through monthly donations, we have explored other possibilities for a lower cost hosting plan. Darren Duncan volunteered to carry out an exploration to determine whether that would be possible with our current host, Steadfast Networks. Greg Mullane, John Stephenson, Matt Innis, and Dan Nessett contributed. Anthony.Sebastian 19:29, 9 August 2013 (UTC)
The following is Darren’s proposal for $100/month hosting:
Darren's proposal
To drastically lower Citizendium's hosting costs to something closer to what it can manage to pay for on a monthly basis, this is the short term plan. Some of the following bullet points can be done in a different order.
- Have Steadfast Networks provision a new dedicated server for us, see Steadfast dedicated servers, and see specifically the least expensive option on that page: Dual Core Atom D525 @ 1.8Ghz; 4 GB RAM; 2 × 500 GB storage; $99.95 per month. I don't know what preinstalled OS options there are, but I recommend using the latest CentOS (Linux), unless Greg has a better idea. Steadfast is whom we currently host with but we're currently using 3 boxes with a total cost of over $300/month, and they will all be replaced.
- Install and configure on that server the latest versions of the Mediawiki software and PostgreSQL database software (9.2.4+). Setup in DNS/etc a temporary domain name for this server, such as new.citizendium.org. Also setup any necessary mail servers so that Mediawiki can send emails such as for supporting part of the new user authentication process.
- Perform a full backup of the current PostgreSQL database used by the current en.citizendium.org production Citizendium MediaWiki software, that has all the articles and user data and so on for the encyclopedia itself. Load a copy of that backup onto the new server and make sure that the new server works correctly, with appropriate testing and tweaking as necessary. It should be possible to do editing or create articles, creating users, and so on. The new server should act just like the current one aside from any consequences of newer versions of the software.
- Shut down the current production Citizendium MediaWiki so that the Pg database can no longer be altered, and do likewise on the new server. On current production, make a new complete backup. On the new box, wipe the database used for testing. On the new box, load that latest production backup, so the old and new servers have matching databases. Then change the DNS settings so that the new box has the domain en.citizendium.org and the old box has old.citizendium.org. Then enable Mediawiki on the new box only so that the encyclopedia is live again. Put up some static web page on the old server, or automatic redirect, so people with slow DNS servers going to the old box can find the new one.
- Setup some place on the Citizendium wiki to hold future discussion threads, where this would be used instead of the current forum.citizendium.org Simple Machines forum.
- Shut down the Simple Machines forum and make a full backup of its MySQL database. Add a static page pointing to the wiki.
- Have appropriate compressed archive tarballs of the entire en.citizendium.org wiki database and the entire forum.citizendium.org database taken and backed up to multiple / at least 3 physical machines, probably my personal computer, and Greg's, and several other people's. So if something happens to any of the Steadfast servers, now or later, we at least have these archives to recover from. This is likely several gigabytes of data.
- Due to time constraints and related complexity, the old forum.citizendium.org discussions won't appear in the wiki right away, but they will be saved in archives so they can be put up later when some tech (probably me) has the time. But you will have the ability meanwhile to have new discussions. Cost cutting can't wait for this migration.
- Shut down all the email lists and email boxes hosted on our servers. Where any are needed, create replacements on Google mail or groups. If necessary we can setup our DNS so that Google actually hosts those under our domain, though I don't know if that costs extra. Hosting our own mail and dealing with all the spam consumes a huge amount of our current server resources and that really isn't something we should keep spending money on.
- Also like #7, have tarballs of all the archived email discussions that our mail server had kept and back them up to the 3+ physical machines as well, so these can be imported/migrated later as useful.
- Once we're confident we have a complete archive of everything important from Citizendium on several of our own machines, and that the wiki is running on the new server without trouble, decommission the 3 old servers so Steadfast can reuse or decommission them as they see fit, and we're no longer paying for those.
- Soon as we get the chance, sometime later, import the old Simple Machines or email or whatever archives into some place where people can see them. Exclude anything that is supposed to remain private or privileged.
- I want to make clear that the backups of everything to multiple physical machines needs to be exhaustive, including any privileged information such as user accounts and such, and so trustworthy people need to be in charge of said machines, what will keep anything confidential that needs to be. Now is not the time to be trying to work out what is or isn't important, aside from that we can probably just delete all the reams of logs related to email spam and such, which are taking up most of our server disk space.
- Assuming that we pay for servers aligned on month boundaries, we should have the new server provisioned by the end of July, and have the old ones shut off by the end of August, after which we shouldn't have more than the $100/month on our bill. This requires Greg signing off on or suitably altering the plan, and any other stakeholders signing off as well.
Darren's amendments to proposal and explanations
I overlooked the blog. The core point here is that everything we host ourselves on the 3 Steadfast servers will be disappearing except for the wiki. If we were hosting a blog too or whatever, then that's going too.
More generally, all content we have in any forum on the servers is meant to be thoroughly backed up and archived to several non-Steadfast machines, and then only the wiki would be reinstated on the new hardware.
I know we have the wiki, the Simple Machines forum, and email; I don't know what if anything else we were hosting. Greg or others know better.
Just keep all the wikis, that should be easy. They're all driven by the same software, and are probably in the same database cluster for that matter; the most trivial complete PostgreSQL backup method would include them all anyway. So backing up the wiki in the simplest way backs them all up.
Strictly speaking, with appropriate software config, we could run the existing forums too. We would have to install more software and configure everything to share the 4GB of RAM. The worst that would likely happen is performance might suffer a bit, but as long as our usage is low it is fine. If usage picks up that this becomes a problem, then donations might pick up then too.
That all being said, we will want some kind of automatic backup solution to some other hardware.
But really, lets try and make this work with the $100/month and not preemptively try to spend more for problems that may not exist.
Greg knows better, but I suggest say limiting MySQL to just 100MB of RAM or so, it just needs to run the forums, and give PostgreSQL most of the RAM, say 2-3GB for the wikis, and the rest for the OS and web server, or something.
Greg, was there or was there not some technical limitation such that the forums had to run on a different machine than the wiki, something about network domain exclusivity, or does (hopefully) this problem not exist?
Comments of initial collaborators
John Stephenson wrote: All sounds good to me. I certainly think it's possible to set up a very basic forum on the main wiki. The only thing we would lose is the non-Citizen comments board. I suppose they could comment on the blog or on our social networking pages, or via a general discussion mailing list.
I know that there is a 'secret' Constabulary wiki somewhere, the Editorial Council wiki (ec.), and the test wiki (test.). Not sure if there are any others...
Matt Innis confirm what John wrote about our wikis.
Greg: That is correct. And as far as I know, there are no other wikis other than those mentioned here, secret or otherwise. The plan looks fine to me. We need to think about backups since we will only have a single box, but that's not a pressing concern.
Dan Nessett: "Since you asked, here is what I recommend. Get rid of all Citizendium applications other than the wiki and look into using one Citizendium server that can handle its load. Greg should be able to figure out what you need. I recommended this last year, but it was not popular. However, Citizendium is no longer the project it once was and I think a wiki could handle all its current needs. Forum traffic is really a trickle and you could set up a talk page on the wiki to take its place. You could move any email accounts you wish to retain to gmail. Greg could set up the DNS server to forward legacy email addresses to the new gmail accounts. The mailman lists, bugzilla, the software repository really are no longer needed."