Yesterday, I submitted a proposal to Talis under their Incubator fund. If successful, I would have the pleasure of working with Paul Stainthorp, E-Resources Librarian at the University of Lincoln, and Casey Bisson,  Information Architect at Plymouth State University. The bid is to develop an idea which I’ve posted about before, based on Casey’s work on Scriblio and our adventures with WordPress MU, in particular, JISCPress.

Anyway, rather than re-iterating the bid here. You can read it in full by clicking here.

Comments are very welcome. Thanks.

Related posts

• Scriblio, Triplify and XMPP PubSub
• WordPress: Beyond Blogging session at IWMW10
• WordPress: Beyond Blogging!!
• Feeding WordPress with EPrints: A Social Repo?
• Jailbreaking WordPress with Web hooks

Over the weekend, there was yet another awesome WordCamp over in Boston. I am still bummed we were unable to go. But! We followed along at home, and there was some great news.

WPcares was launched during the event to raise money for Haiti. The original goal was $1,000. Last I checked, it was over $3,000 and this will be matched in donations from

• StudioPress
• RocketGenius
• iThemes
• Headway Themes
• WebDev Studios

Wow. This is why I love the community SO FREAKING MUCH.

The WordPress Foundation was also launched.

The point of the foundation is to ensure free access, in perpetuity, to the projects we support. People and businesses may come and go, so it is important to ensure that the source code for these projects will survive beyond the current contributor base, that we may create a stable platform for web publishing for generations to come. As part of this mission, the Foundation will be responsible for protecting the WordPress, WordCamp, and related trademarks. A 501(c)3 non-profit organization, the WordPress Foundation will also pursue a charter to educate the public about WordPress and related open source software.

Please visit their site for more information.

And, visitors to site may have noticed the clean design over there. It is the new 2010 theme, which will be the new default theme for this year.

Update: there will be some important terminology changes. Dougal gives a rundown over here between sites and networks.

I’ll go into this in depth at a later date, when trunk gets more finalized.

Random Posts

• April 21, 2009 — 2.7.1 released, and an overview (6)
• July 13, 2009 — WordPressMU 2.8.1 now out, features and a couple bugs (4)
• September 30, 2007 — Where to find error logs (3)
• July 19, 2009 — Buddypress Blog Author Link (6)
• July 31, 2008 — Introducing Ron (1)

© andrea for WPMU Tutorials, 2010. | Permalink | No comment | Add to del.icio.us
Post tags:

Need real genuine helpful support? MU Support.

Feed enhanced by Better Feed from Ozh

Last week, in the post where I announced my new WPMU plugin Shared Blogroll, I briefly mentioned the use case that I had in mind, which was of course education-focused. Yesterday and today I did some more work on the plugin itself (MORE AJAX 4EVA) and some of the supporting players. I thought I’d spell it out a little more so that others could implement what is probably the coolest system for shared blogrolls on earth, or at least on WPMU, which amounts to pretty much the same thing.

The plugins

• Shared Blogroll. This creates a widget that allows bloggers to embed a link category from any other blog on the system on their own blog.
• Add Users Sidebar Widget. This plugin is already pretty great, as well it should be seeing as it’s written by good-looking Canadians. I made some edits to the plugin to make it a bit more flexible.
  1. First I removed the RSS box from the Add User widget. For most of my users, it’ll confuse; for all of them, it’ll be redundant, since they’ll all be using Wordpress blogs and thus will have feeds that are very auto-discoverable.
  2. As the plugin stands in the repository, links are automatically put into the blog’s default link category (or, if they’re using FeedWordPress, the FWP link category). For most users, that’s something unhelpful like “Links” or “Uncategorized”. I altered the plugin so that the blog owner can choose which link category the added links will be put into. As I’ll discuss below, I think that this will add a lot of flexibility to the way that the plugins can be used together. You can download the text of my modified version of Add Users Sidebar Widget here: add-link-sidebar-widget.txt. Save as .php before uploading – and make sure that you uninstall previous versions of the plugin before uploading, or you might get whitescreened to death.
• Optional step. The default WP Links widget is set up so that you can’t select to display an empty category of links. As you’ll see below, I want to have instructors create a new category for their blogrolls, and I figured it’d be pretty confusing if they couldn’t create a sidebar widget corresponding to a link category until at least one student had posted a link (thus populating the category). So I wrote a little plugin that replaces the default Links widget with one that allows you to pick an empty links category. Drop this into mu-plugins: link_widget_allow_empty_cat. Be warned, though, that this plugin deregisters the default Links widget, which might have the effect of removing every links widget on your installation of WPMU. (I haven’t tested it.) For me, it’s OK, because I don’t have any active users yet. As an alternative, you could edit the core of WP (making sure to back up, take note of where you made the edit for future upgrades, blah blah blah) in the following way. In wp-includes/default-widgets.php, line 139 reads

  $link_cats = get_terms( 'link_category');

  Replace it with

  $link_cats = get_terms( 'link_category', 'hide_empty=0');

  That should leave all of your pre-existing links widgets in place.

The process

The immediate goal of this setup is to allow a class blogroll to be shared effortlessly between members of a class. (Though I can imagine lots of other very cool uses.) Here are the directions I’m going to give to instructors and students on how to leverage these two plugins to make it happen.

Instructors

• Create a new Link Category for your class blogroll – let’s say you call it “ENG 110 Blogroll”.
• Add a Links widget to the sidebar, making sure to select “ENG 110 Blogroll” from the category dropdown.
• Add an Add Links widget to the sidebar. Again, be sure to select “ENG 110″ blogroll as the target category for added links.
• Give your blog’s ID number to your students and give them the following directions.

Students

• After creating a blog, visit the professor’s blog and enter your site’s URL into the Add Link sidebar widget.
• Add a Shared Blogroll widget to your own blog’s sidebar. Enter the professor’s blog ID number as the source blog, select “ENG 110 Blogroll” from the dropdown list, and Save.

I hope that the instructors and students get into this blogroll sharing thing. It distributes what would otherwise be a huge amount of work for the instructor, or *worse yet*, for me.

Related posts:

1. New WPMU plugin: Shared Blogroll
2. Blog-specific email plugin for WPMU users
3. Streamlining Group Blogs

OK, so this will probably be the final installment of my “pimp your WPMu databases” series Part one was moving from WPMuDEV’s Multi-DB package to SharDB; part two dealt with optimizing tables across a number of databases in MySQL, and the final installation will share some SQL code, once again thanks to Gowtham, for deleting specific tables across various blogs and databases.

Ok, so here is the setup, when we started UMW Blogs we used the spam filter Spam Karma 2. It was a personal favorite of mine, and while a lot more work than Akismet, it did a pretty good job of shutting out spam. Spam karma 2 has since died, and we moved to Akismet well over a year ago now. However, for at least the first 500 or 600 blogs (maybe more) we had Spam karma 2 installed, which inserted two tables into each blog kinda like this:
wp_21_sk2_logs
wp_21_sk2_spams

Where 21 was the blog ID, so across numerous blogs and databases there are a number of blogs with two fields like those above (just a different blog ID number, say wp_581_sk2_logs) filled with spam. It would make sense to drop these tables out as part of an optimization of our databases, but the only question I had was how?

Well I went to Gowtham, the oracle, once again (I hope this well of riches doesn’t go dry for overuse ) and he sent me the following script for deleting all tables with _sk2_ in the name. Now I imagine very few folks have this specific issue with Spam Karma 2, so the details of this script may seem irrelevant. But if you think about it, if there is a plugin that inserts tables on numerous blogs spread across various databases, and an uninstall of that plugin doesn’t delete the tables, then such a script would be quite useful to you. Just change the list of databases from my specific setup to yours and instead of the following bit of code:
$sql_q = "SHOW TABLES LIKE '%_sk2_%'";

Put a unique series of characters between the ‘%_uniquecharacters_%’ to delete a series of tables across blogs and databases.

How freaking cool is this? I tested the script on UMW Blogs, and it worked beautifully, UMW Blogs databases just lost another half a gig of spam fat, and I’m am happy to say it’s never been healthier. No heart disease in the future of this WPMu install—though we can never rule out plugin cancer

Here is a text version of the script.

And here it is sans the opening and closing PHP calls:

# PHP script to find all tables, whose name contains
# the string '_sk2', in a given set of MySQL databases
# and delete them. This is a 2-step process.
# Run this script as
#
# php mysql_skarma.php > mysql_skarma.sql
#
# Then, run the following command in a terminal
#
# mysql -u USERNAME -p < mysql_skarma.sql
#
# and enter PASSWORD when prompted.
#
# First written: Gowtham, Mon, 25 Jan 2010 08:01:48 -0500
# Last modified: Gowtham, Mon, 25 Jan 2010 09:20:22 -0500

# Close the connection
mysql_close($connect);

And to build on my last post about Moving from Multi-DB to SharDB on WPMu, here is a rundown on some of the work done to further optimize the multiple-databases for our WPMu at UMW Blogs.

First off, we have a plugin install called WordPress MU Sitewide Tag Pages that basically republishes every public post from around UMW Blogs into one blog, the tags blog, which is located at http://tags.umwblogs.org. This plugin has a ton of uses for us, not least of which is sitewide tags and categories, as well as feeds off those sitewide tags and categories for syndication and republishing into course sites. Another function of this blog could be a searchable archive of all the public work done. So, needless to say, this blog is important enough that we have let it run now for almost two years without pruning. And when I checked the wp_posts table of this blog in the database there was what seemed to be 99,066 posts.

And that one table was weighing in at a hefty 425 MBs, which mean it was out-of-control, and I imagine some of our recent performance and down time issues might be related, so I decided to cull this table. So after backing it up, I tweeted looking for advice on how I might delete all posts from the tags blog before a certain day, say August 15, 2009—the week before the Fall semester started this year.

And in no time at all, Michigan’s finest, Gowtham (and sgowtham on Twitter) sent me the following bit of code:

DELETE FROM wp_posts WHERE post_date < "2009-08-15";

Along with the note, “please be sure to replace the right table name in place of wp_posts though.” Which would like like this for the tags blog with the ID 1291:

DELETE FROM wp_1291_posts WHERE post_date < "2009-08-15";

So I just copy that bit of SQL code into the PHPMyAdmin editor that lets you run SQL queries for a specific database like so…

And within a minute 66,000 posts were removed, and the table was now 142 MBs.

Now, let’s stop here for a second. How sick is that? Within minutes I got a solution to my issue and deleted 66,000 posts and potentially saved UMW Blogs a couple of more crashes this coming week, from a friend on twitter who just knew what to do. How much richer am I because of this relationship? How much richer is all of UMW? It’s kinda wild to think about that. And more than that, it doesn’t stop there….

When I removed those 66,000 posts the size of the table was still showing 425 MBs, but there was now a value in overhead column of the table that had a value of 140 MBs. I had o idea how to get rd of this, so asked again in Twitter, and D’Arcy Norman tweeted me back this:

He was right, optimizing the table brought the 425 Mbs down to a slender 140 MBs, and the overhead value was flushed clean. So this got me thinking, hmmmm, what if I could run a script and optimize every table in every database? I mean this site has been running non-stop for almost three years, and has been a work horse. How about giving UMW Blogs an “oil change”—to use Gowtham’s analogy which works beautifully. So, when I tweeted about that optimization question, Gowtham once again sent me a message, and after I gave him the naming conventions of my 18 databases on UWM Blogs, wham, I had a PHP script to optimize every table in UMW Blogs.

People, are you getting the point here? Gowtham is an Engineering Physics grad, a hell of a photographer too, but he knows his shit. And he was willing to share that will me in all of about a few minutes the first time. But for the optimization script, he actually wrote it up and sent it to me via email. I don;t know how long that took him, but it would have taken me hours and hours of research and learning to figure out I could do it. And hours more to beg someone who knew at UMW to write it for me (if that person existed). Well, this process was a matter of minutes.

And, Gowtham not only sent me the SQL Optimization script, but was more than cool with me sharing it, so here is the code to the script as a txt file.

And here it is copied below sans opening and closing PHP calls.

# PHP script to optimize each table in a given set
# of MySQL databases and improve the overall performance.
# First written: Gowtham, Mon Feb 27 09:33:01 EST 2006
# Last modified: Gowtham, Sun, 24 Jan 2010 13:01:58 -0500

# Close the connection
mysql_close($connect);

I put in the drop database sign-in information, uploaded it to UMW Blogs via FTP, and went to the script at http://umwblogs.org/mysql_optimize.php, and a few minutes later all of the tables in all 18 of the databases were optimized.

Is this not awesome, UMW Blogs has gotten the oil change it has been in desparate need of—and I don’t recommend waiting every 4,000 users and blogs, every 2,000 may be wiser. So, thanks to Gowtham, UMW Blogs is purring like a cat, and I really hope it stays that way.

And just to show you I have no shame, something else I saw when the databases were optimizing is that random blogs still have the Spam Karma tables in them, I would think about 500 or 600 blog tables filled with old spam. The tables are labeled wp_blogid#_sk2_logs and wp_blogid#_sk2_spams. For example: wp_21_sk2_logs and wp_21_sk2_spams. So riddle be this Batman of Gowtham, how would you find and drop all tables with _sk2_ in them across 18 databases?

I love the internet, and I can’t thank you enough Gowtham for your selfless support, people like you make this space both fun and wonder-filled.

In the early morning hours yesterday we moved UMW Blogs from multi-db (which is a multi-database setup provided through WPMuDev Premium at a cost) to the ever so free SharDB multi-db setup, and I’m happy to say it worked. We’ve had some “unscheduled downtime” recently at UMW Blogs, and between the ever-growing traffic and a syndication/tags blog that’s all too full with 99,000 posts—it was high time to start cleaning database house. This post will take you through the switch from multi-db to SharDB, which is surprisingly simple. And the following post will be about optimizing a multiple database setup that depends heavily on syndication like UMW Blogs does.

Multi-DB –> SharDB
So, our first step was to move to Ron Rennick’s (of wpmututotials and @wpmuguru fame) SharDB setup because UMW is no longer a subscriber of WPMuDEV Premium, and given upgrades to multi-db are no longer available to us—-we were eager to be rid of it. What’s more, I’m hoping this move to SharDB (accompanied by the database optimization and tags blog clean up) will bring UMW Blogs back to a snappiness we haven’t known for a couple of months now. So here’s to hoping….

When I downloaded SharDB I expected a lot more work than there actually was. The move is rather simple, and save a hiccup or two on my end, it went smooth as silk. The first thing to do is configure the db-settings.php file. UMW Blogs is running on 16 databases (well actually 18, but that is besides the point) on one dedicated server, we don’t have several servers or multiple data centers so I think our setup was rather simple. And while we probably should consider getting another server or two some time soon, as Joss Winn suggested, for right now we’re easy.

[What follows is pretty ugly ]

The UMW Blogs db-settings.php configs look like this:
// If you have multiple datacenters you can come up with your own datacenter
// detection logic (php_uname?). This helps ensure the web servers try to
// connect to the nearest database servers first, then distant ones.
define( 'DATACENTER', '' );

/* That's all, stop editing! Happy blogging. */

I wasn’t entirely certain about all the details in the config file, so before I did the upgrade I posted a few questions on the WPMu Forums here, and Ron himself bailed me (thanks Ron!). Once I had the db-settings file squared away I uploaded it to the main directory of UMW Blogs. After that, I added two lines to the wp-config.php file:

//SharDB Settings
define(‘WPMU’, ’1′);
require_once(‘db-settings.php’);

And finally, I uploaded the dp.php file to the wp-content directory, and the shardb-admin.php file to the mu-plugins directory. (Keep in mind I had already deleted the db-config.php and db.php files that were running multi-db setup.)

Once I had everything up, I went to the homepage of UMW Blogs and got this error:

Connected to localhost:3306 but unable to select database ‘umwblogs_umwblogs’ while querying table ‘wp_domain_mapping’ (global_r)Connected to localhost:3306 but unable to select database ‘umwblogs_umwblogs’ while querying table ‘wp_site’ (global_r)Connected to localhost:3306 but unable to select database ‘umwblogs_umwblogs’ while querying table ‘wp_site’ (global_r)Connected to localhost:3306 but unable to select database ‘umwblogs_umwblogs’ while querying table ‘wp_site’ (global_r)Connected to localhost:3306 but unable to select database ‘umwblogs_umwblogs’ while querying table ‘wp_site’ (global_r)Connected to localhost:3306 but unable to select database ‘umwblogs_umwblogs’ while querying table ‘wp_site’ (global_r)No WPMU site defined on this host. If you are the owner of this site, please check Debugging WPMU for further assistance.

Turns out, multi-db does not require you to change the database name in the wp-config for the mutliple databases. so our database settings were still set to the original single database we started UMW Blogs on: umwblogs_umwblogs. Once I figure this out, I changed the db_name setting to point to our global database:
define('DB_NAME', 'umwblogs_wpmu_global');

Now this aneming convention is particualr to our setup, we have our tables named umwblogs_wpmu_global, umwblogs_wpmu_0, umwblogs_wpmu_1, umwblogs_wpmu_2, etc., up through 9, and then a through f. So our database names look like this:

umwblogs_wpmu_global
umwblogs_wpmu_0
umwblogs_wpmu_1
umwblogs_wpmu_2
umwblogs_wpmu_3
umwblogs_wpmu_4
umwblogs_wpmu_5
umwblogs_wpmu_6
umwblogs_wpmu_7
umwblogs_wpmu_8
umwblogs_wpmu_9
umwblogs_wpmu_a
umwblogs_wpmu_b
umwblogs_wpmu_c
umwblogs_wpmu_d
umwblogs_wpmu_e
umwblogs_wpmu_f
umwblogs_wpmu_vip1

Once I changed the db_name setting in wp-config, SharDB seemed to work fine on every other blog but the main blog: umwblogs.org. After a second of panic, I realized that SharDB was looking for the home blog (or Blog ID 1) in the database _wpmu_c, but I had it in the wpmu_vip1 database. Once I exported the tables from the wpmu_vip1 database into the wpmu_c database, everything worked perfectly. Put the question for me remains, why didn’t SharDB pick up the vip1 database? I had it noted in my db-settings file as follows:

// VIP DB's
if ($enable_home_db === true) {
add_vip_blog(1,'vip1'); // home blog
}

Additionally, when I was doing something in the backend, I came across this error:

Connected to localhost:3306 but unable to select database ‘umwblogs_wpmu_global’ while querying table ‘wp_sitemeta’ (global_w)

I can’t for the life of me remember where I was precisely now, but I’m sure I’ll find it again soon. I was wondering if this might have something to do with a number of global tables we added to the db-config.php file for getting BuddyPress to work with multi-db? Not sure here, kind of fishing for some help, but I know with multi-db we had issues adding global tables for BuddyPress to the global database (same goes for the domain mapping plugin table), and one way to ensure those tables were created in the global database was through the following list of code I stole from D’Arcy, and he breaks the whole thing down better than I ever could here. Now I imagine SharDB doesn;t have this issue, but just wanted to get a sense if anyone else playing with the move from MultiDb to SharDB had any ideas.

// Usage: add_global_table(TABLE_NAME)
// EX: add_global_table(‘something’);
add_global_table(‘umwblogs_wpmu_global’);
// BuddyPress
add_global_table(‘bp_activity_sitewide’);
add_global_table(‘bp_activity_user_activity’);
add_global_table(‘bp_activity_user_activity_cached’);
add_global_table(‘bp_friends’);
add_global_table(‘bp_groups’);
add_global_table(‘bp_groups_groupmeta’);
add_global_table(‘bp_groups_members’);
add_global_table(‘bp_groups_wire’);
add_global_table(‘bp_messages_messages’);
add_global_table(‘bp_messages_notices’);
add_global_table(‘bp_messages_recipients’);
add_global_table(‘bp_messages_threads’);
add_global_table(‘bp_notifications’);
add_global_table(‘bp_user_blogs’);
add_global_table(‘bp_user_blogs_blogmeta’);
add_global_table(‘bp_user_blogs_comments’);
add_global_table(‘bp_user_blogs_posts’);
add_global_table(‘bp_xprofile_data’);
add_global_table(‘bp_xprofile_fields’);
add_global_table(‘bp_xprofile_groups’);
add_global_table(‘bp_xprofile_wire’);
add_global_table(‘wp_domain_mapping’);

Other than that SharDB is up and running on UMW Blogs, an given it may be one of the most unstable WPMu sites in the world give we have over 100 plugins and 130 themes, it has been solid as a rock thus far, so fine work Ron.. But to make sure we don’t push our luck too much, now it’s time for the database optimization drill, post forthcoming.

I updated SharDB (Download) for WordPress MU 2.9.1.1 and added support for the comment meta table.

• December 29, 2009 — Scaling WordPress MU with multiple databases (31)
• September 24, 2009 — Changing the domain (17)
• January 21, 2010 — Testers, start your engines (6)
• January 14, 2010 — 2.9.1 released, last before merge (6)
• January 6, 2010 — Merge News (1)

© ahome for WPMU Tutorials, 2010. | Permalink | 4 comments | Add to del.icio.us
Post tags: database, multiple, shard, wordpressmu, wpmu

Need real genuine helpful support? MU Support.

Feed enhanced by Better Feed from Ozh

The bulk of the merge has been done in WordPress core. At least with regards to moving over the files. Now the fun part begins: bug gardening, testing and tweaking.

Here’s what you can do to help not only the dev team and others, but also yourself:

- set up a local install of WPMU
- import some data, plugins & themes from your current setup
(in other words, create a working copy somewhere)

Now, upgrade it to the latest WordPress trunk. The zip is located at the bottom of this page. To manually upgrade, just overwrite the MU files with the files located in the zip.

When you find bugs, please file them over at the WordPress trac, and pick the term multisite in the component field dropdown, so the other devs know is it MU related. Some terminology is changing, so this refers to the multiple blog capabilities. Your login to file a ticket is the same as your forum username and password.

Did you get the part where I said to do this on a dev version and not a live site? Good.

• January 14, 2010 — 2.9.1 released, last before merge (6)
• January 6, 2010 — Merge News (1)
• January 5, 2010 — The merge begins (21)
• December 27, 2009 — Merge news and Buddypress awesomeness (7)
• December 9, 2009 — Twitter Digest (4)

© andrea for WPMU Tutorials, 2010. | Permalink | 6 comments | Add to del.icio.us
Post tags: development, merge, trac, trunk, wordpress, wordpressmu, wpmu

Need real genuine helpful support? MU Support.

Feed enhanced by Better Feed from Ozh

In setting up WPMU sites for classes, I often wind up using the Text widget to add a bunch of important links – login, Dashboard, Add Post, Add Page, etc… to each site. Manually. I finally decided to save some time and just write a plugin that provides a generic widget to give the links on any site that uses it.

If you’re not logged in, it provides a link to login:

If you are logged in, it gives you some handy links:

I’ll eventually add a way to customize the widget (display Add Page? display Add Post? something else?) but for now, it’s an easy fire-and-forget widget for the most common links used by people in a course blogging site.

Get your copy over at the WordPress plugin repository: Important Links Widget

I just wrote a widget that lets a blog admin pull a link category from any blog on the same WPMU installation. It is pretty much the bomb, and you are going to love using it so much that you may have to take downers in order to get yourself under control.

Check it out.

Side note: I’m going to start creating separate pages (as opposed to posts) for each of my software releases. That way all comments/support regarding a given plugin will be in the same place, with a pretty URL. That’s why I’m turning off comments on this post.

Related posts:

1. Blog-specific email plugin for WPMU users
2. New BuddyPress plugin: Invite Anyone
3. New BuddyPress plugin: BP Include Non-Member Comments