BuddyPress at Scale

We are using BuddyPress on NYT (largest recipe website of Turkey) and I want to share some ideas and experience about scaling BuddyPress. (We might be second largest BuddyPress user)

1) Scaling WordPress

First of all we need to know scaling WordPress because (as you know) BuddyPress is a WordPress plugin. I don’t want to go into detail, you can reach so many helpful resources about it.

• https://www.scalewp.io/#content
• https://www.quora.com/How-well-does-WordPress-scale
• https://premium.wpmudev.org/blog/scaling-wordpress-wpmu-buddypress-like-edublogs

We use these technologies in our infrastructure – stackshare.io/nefisyemektarifleri-com/nefisyemektarifleri

2) Storage

Who remembers old multisite problems like – https://codex.wordpress.org/How_to_Scale_WPMS

The ext2/3 filesystems on Linux only allow 32,000 files or directories in a directory. Thankfully ext4 haven’t this limitation. (I mean in theory) You shouldn’t put hundreds of thousands objects into same directory, (that’s insane)

BuddyPress’ default upload structure is not good for large scale but don’t worry you can change structure via proper filters.

We use daily based structure and sync to s3. When varnish couldn’t reach image from storage server, it fetches from s3.

Btw, BuddyPress-VIP-Go can give some ideas. Thanks Paul.

Pro tip: When you add new attachments to post which published “23/09/2015” WordPress puts new uploads to “…/uploads/2015/09/foo-bar.jpg” (if you organize uploads into month- and year-based folders)  when your uploads directory contains millions of objects this approach is not efficient. (rsync process began to take longer and eats your CPU)

Thankfully, we have `upload_dir` filter. We can organize uploads daily basis and sync fresh uploads only.

3) Notifications

We have a lot of custom events. For example, when your recipe added more than x time to users cookbook we add new notification.

We have developed robust notification system over BuddyPress. It supports, push notification(we use Parse), web push notification and email notification.

Some of our events are really heavy, for example when video of recipe publish on website, we send notification who added that recipe to their cookbook. That means, thousands of records into notification table.

We are using RabbitMQ for queueing things and we use custom WP-CLI commands as worker which are running on server’s screen.

4) Core hacks (Yikes!)

I confess, we are doing quick and dirty solution sometimes, one of them is `bp_notifications_get_all_notifications_for_user` function, it caches all notifications for user and we added hard limit there. (My Boss reported this problem on trac as well.)

So shortly;

Can Buddypress support hundreds of thousand of users?

– Yes, obviously.

I want to be next facebook, can BuddyPress do that?

– trust me, you won’t be next facebook 🙂

I need X feature which is not exist in BuddyPress core?

– Simple, write a plugin for BP, for example one of our core feature is “following” and it’s just a BP plugin. (Thanks r-a-y)

If you have question, leave me comment below, I would like to hear questions from you 🙂

Cheers,