Two years reviewing plugins on WordPress.org

Two years ago, a very interesting journey began.

I joined the WordPress.org plugin review team and, well, it’s an impressive amount of work since then, but after two years and still tired, I feel happy and satisfied with all that we have achieved.

The “Internal Script”

The “internal script” is a tool from the plugins team that helps us in code reviews and that we have inherited from Mika.

It is the largest piece of software I have ever created in my life, and it has been created based on an iterative process that is described in the following image.

Since then, it has not stopped evolving.

These are some of the achievements we have made with this tool:

• Having more than 200 automated checks on a static code analyzer based on nikic/PHP-Parser that allows us to look for security and compatibility issues in plugins.
• Automatically generate a response email with all found issues of which there is a high degree of certainty (the team reviews it eliminating possible false positives).
• Perform static analysis of code taking into account the context. For example, the tool is able to recreate a previously declared string or determine from where the execution of a function or method is called. Thus, without executing the code, it can determine which values will be in which places, with more or less certainty (which is also taken into account).
• Multithreading system to parallelize the execution of the different checks. Currently, a it launches 16 threads, which allows the process to complete within 5-10 seconds in most cases (25-30 seconds without threads).

The software is divided into large blocks of checks that have been created naturally according to the needs and cover the main issues we see in the plugin review.

For example there is a check to analyze the plugin data (readme file and plugin headers) that searches and gives information about dozens of possible problems, from the most basic and simple to more complex things, for example:

• That the description is empty or identical to the description in the sample readme file.
• That the plugin mentions a plugin with which it integrates and has not included it in the “Requires Plugins” parameter of the plugin’s headers.
• That you have entered a non-existent WordPress version in the “Tested up to” parameter.

We keep evolving it, the best thing is that we have the largest testing ground, more than 200 new plugins every week from different authors and we find all kinds of cases in the code we review. The plugin authors will never cease to amaze me.

We are taking more and more things into account to be faster, more effective and more accurate in code reviews.

The team

The plugin review team is made up of volunteers, some sponsored and some not, who dedicate their time to help others improve their plugins and manage the directory according to the guidelines.

I have always worked alone, or almost alone, and for projects that end, and I like to be part of something with continuity and in which there are more people with the same goals. The plugins team, mathematically, would not work without being a team. So far there are people who have joined and people who have left and we know that without the contribution of each one it would simply not be possible.

I came to the team knowing the basics, learned little by little, and once I reached a point of some confidence I documented the team’s processes and helped onboard new people.

Numbers

Do you like statistics? Come on, let’s crunch some numbers about my work on the team.

Emails

What impresses me the most: I am the person who has sent the most emails within the WordPress.org system in these two years.

I am closely followed by 4 great volunteers, two of them from the plugins team as well (it’s logical, we manage a lot of emails).

I have handled 10985 cases by email and sent 18511 emails in 8896 conversations.

Of the almost 9000 conversations, more than 3000 have been for general questions and issues, 723 have been for guideline violation cases, and the rest related to plugin reviews.

Plugins

As for the number of plugins reviewed, in the last two years:

• I have initiated about 4200* new plugin revisions (out of a total of 14984 new plugins).
• I have approved 2156 plugins (out of a total of 7077 approved).

* For some elements we have estimated data due to inaccuracies in the data and the change of systems with which we have taken data.

These data have been improving over time, for example in the last nine months, for which we do have accurate data:

• I have initiated about 2145 revisions (out of 7369 new plugins, in the last 9 months we have received as many plugins as in the previous 15 months).
• Each plugin requires more than one review before being approved, I have performed 7014 plugin reviews.
• I have approved 1223 plugins (out of a total of 4091 approved).

Hours

And how long has all this taken?

1948 hours (July 2023 to present).

These are the hours per month spent since I started counting them.

I do not have specific data on what each hour has been dedicated to, but in general if we divide it into three large blocks and make an absurd average, for each hour I have done:

• Tool development: 0.3 commits and changes in 65 lines of code.
• Plugin review: 2.15 plugin reviews initiated, 7.5 additional reviews and 1.1 plugins approved.
• Directory management: 1.56 doubts solved by email, 0.37 violations of guidelines managed, 4.73 messages on Slack, 0.037 team meetings, 0.04 publications in make, 0.03 documentation articles created.

Doing these statistics it has struck me that in the last 9 months I have reviewed as many plugins as in the previous 15 months… and I am spending more or less the same amount of time.

This data confirms something that I am very happy about, that we are becoming more efficient.

SiteGround

Everything that has happened with my participation in the team would not have been possible without the support of SiteGround, which from the beginning wanted to support me and continues to do so today.

Yes, their support is economical, but they go beyond that. They have helped me when I have had doubts about how to improve the team, they helped me to give my first talk in English in front of a large audience, they treat me as one of their people and have invited me to their offices to meet and get to know each other.

And the only thing they’ve asked of me is to keep doing what I do at WordPress.org.

Over time I have gotten to know them better and, more than a company, I feel them as a family.

Beneath the layers of concrete and glass of their impressive offices, and behind their hundreds or thousands of servers, there is a spirit that is shared: a kind, resilient and visionary spirit. I think that’s what makes SiteGround SiteGround. Knowing that such a relevant organization in the sector operates with that human spirit and that passion to do better, honestly, fills me with pride and makes me feel very comfortable with them.

Life

I’m still the same but with outbursts of ideas that I need to turn into code before they go out of my head and a master in argumentation in English via email.

In this time, after the last update everything has balanced a bit, it is true that I still dedicate a lot of free time to continue contributing to the team, I guess by transforming my free time into something not so free (although I enjoy it), but I am doing my things and watering the plants again, although honestly it still feels complicated to go several days on vacation (I have done it, but it happens at the cost of bearing a greater workload several days before and after the holidays).

Anyway, we will get on with life and work as best we can.

What we have achieved

Overall, we have met the main objective which is to help plugin creators improve their plugins in terms of security, compatibility and compliance with guidelines:

• We have formed a new team with people from different places and with different availabilities that is making this work in parallel.
• We have improved the internal tool so that we are able to see many more issues and we are faster and more specific in providing solutions.
• We have created a public tool that helps plugin creators review their code and see many of the issues we see.
• We are reviewing twice as many plugins as we did a year ago and are maintaining a consistent review time.
  ⚠️ This is likely to change during the week of WordCamp Europe, as much of the team will be there.

What does the future hold? Hopefully a lot of success for the team. That will mean a lot of success for plugin creators, plugin users, WordPress and the internet in general. We’ll see.