User:Dan Nessett/Technical/How to set up a CZ clone on Ubuntu: Difference between revisions
Jump to navigation
Jump to search
imported>Dan Nessett (→Notes) |
imported>Dan Nessett No edit summary |
||
| Line 25: | Line 25: | ||
|} | |} | ||
Using importDump.php in /maintenance I populated a version of CZ as a local development environment. The Statistics special page showed in excess of 129,000 pages. The import reported populating 116,400 pages (looking at the pages table, the exact number is 116,486). This checks out, since the daily dump of CZ does not include histories. There are approximately 12,700 live articles, each of which would have a history page. Noting, 116,500 + 12,700 = 129,200, it appears all content pages were loaded. However, it took in excess of 3 1/2 days (about 80 hours) to import the content. This suggests looking at more efficient import strategies (e.g., using mwdumper or converting to SQL with xml2sql and importing directly into the database). | * Using importDump.php in /maintenance I populated a version of CZ as a local development environment. The Statistics special page showed in excess of 129,000 pages. The import reported populating 116,400 pages (looking at the pages table, the exact number is 116,486). This checks out, since the daily dump of CZ does not include histories. There are approximately 12,700 live articles, each of which would have a history page. Noting, 116,500 + 12,700 = 129,200, it appears all content pages were loaded. However, it took in excess of 3 1/2 days (about 80 hours) to import the content. This suggests looking at more efficient import strategies (e.g., using mwdumper or converting to SQL with xml2sql and importing directly into the database). | ||
I now need to run maintenance/runJobs.php. The statistics page shows 272,975 queued jobs, so running all queued jobs is going to take a while. [[User:Dan Nessett|Dan Nessett]] 22:39, 23 November 2009 (UTC) | : I now need to run maintenance/runJobs.php. The statistics page shows 272,975 queued jobs, so running all queued jobs is going to take a while. [[User:Dan Nessett|Dan Nessett]] 22:39, 23 November 2009 (UTC) | ||
* Had trouble getting texvc to work: | |||
:* Need to ensure images directory has both a /math and /tmp subdirectory with read/write access and the images directory is accessible to the apache2 server (I simply chmod 777 both of them)/ | |||
Revision as of 12:22, 25 November 2009
Notes
- Had to modify createAndPromote.php to check password for validity before creating user. Otherwise, if the password is invalid a "ghost" user is created in mwuser table.
- Had to create a dummy user with user_id of 0, so XML dump import would work.
- Need to set Xdebug variables for both apache2/php and php-cli versions of php.ini
- Had to set xdebug.max_nesting_level=200 in /etc/php5/cli/php.ini so dump import wouldn't croak.
- Some useful information on MW XML dumps: http://www.mail-archive.com/wikitech-l@lists.wikimedia.org/msg01712.html, http://www.gossamer-threads.com/lists/wiki/wikitech/180598, http://meta.wikimedia.org/wiki/Data_dumps, http://meta.wikimedia.org/wiki/Xml2sql
- Used cloc to count lines of PHP code in CZ:
Directory Files Blank Lines Comments PHP code statements CZ phase 3 1005 56590 69544 460125 CZ includes 321 14769 33313 97375 CZ extensions 142 3769 6742 27350 CZ includes+extensions 463 18583 40055 124725
- Using importDump.php in /maintenance I populated a version of CZ as a local development environment. The Statistics special page showed in excess of 129,000 pages. The import reported populating 116,400 pages (looking at the pages table, the exact number is 116,486). This checks out, since the daily dump of CZ does not include histories. There are approximately 12,700 live articles, each of which would have a history page. Noting, 116,500 + 12,700 = 129,200, it appears all content pages were loaded. However, it took in excess of 3 1/2 days (about 80 hours) to import the content. This suggests looking at more efficient import strategies (e.g., using mwdumper or converting to SQL with xml2sql and importing directly into the database).
- I now need to run maintenance/runJobs.php. The statistics page shows 272,975 queued jobs, so running all queued jobs is going to take a while. Dan Nessett 22:39, 23 November 2009 (UTC)
- Had trouble getting texvc to work:
- Need to ensure images directory has both a /math and /tmp subdirectory with read/write access and the images directory is accessible to the apache2 server (I simply chmod 777 both of them)/