Categories
Coding PHP

WordPress and it’s Broken Core

When I develop with PHP (7.2+) I develop with strict mode and xdebug enabled. I do this on purpose because notices and warnings are both important to your quality of code. Constantly I am running into one issue with WordPress core that is quite frankly annoying.

This bug is also affecting WPEngine users a lot.

Warning: Parameter 1 to wp_default_scripts() expected to be a reference, value given in /[..]/your-directory/wp-includes/class-wp-hook.php on line 286
Warning: Parameter 1 to wp_default_packages() expected to be a reference, value given in /[..]/your-directory/wp-includes/class-wp-hook.php on line 286
Warning: Parameter 1 to wp_default_styles() expected to be a reference, value given in /[..]/your-directory/wp-includes/class-wp-hook.php on line 286

The Error

The Error is pretty self evident that somewhere in the core wp_default_scripts, wp_default_packages, and wp_default_styles are all being called incorrectly. But where? and why?

Diving into class-wp-hook.php we’ll find that this is error is emitted from inside the apply_filters code. Unfortunately for us, the apply_filters code uses a lot of magic in the form of call_user_func_array – a built in PHP function that calls a user’s function with an array of arguments. Blammo. That should already be setting off your alarms. If wp_default_scripts expects a reference passing an array of values is definitely going to blow it up.

The Investigations

You’ve probably found yourself here by googling the above error message(s). You’ve probably already sifted through a few forum posts that tell you to “ignore errors” or “fix your theme” or “we have no idea!”

I’m going to start with ignoring warnings is a terrible suggestion. It always has been. Fix your code, increase your code quality. But what happens when your code isn’t the problem?

Unfortunately, a bug in core code takes time to fix. So you can report it (it’s a 5 month old bug, but the problems show up on the internet long before that). You can wait. And you can hack your core files to get rid of the error yourself.

Disclaimer: hacking your core files will void your warranty.

The Actual Problem

Since I’ve been tracking down this problem, I’ve discovered that it’s not super simple. The problem is how parts of WordPress core handle the wp_scripts object. If you write themes you’re probably aware of wp_equeue_script (and it’s friends). If you look at wp_enqueue_script you’ll notice the first thing it does is obtains a global object of $wp_scripts (I also despise global variables, but that’s another issue) and if this object doesn’t exist, it creates a new one. Easy enough.

The part that breaks is when the filter gets run for WordPress’ default filters. In there it adds add_action( 'wp_default_scripts', 'wp_default_scripts' ); and as one would expect, it tries it’s hardest to set up the default scripts. But that’s when it breaks. wp_default_scripts expects the reference to the global $wp_scripts to be passed. It’s not. It’s passed as a value and that breaks.

It would be a lot easier (and likely better) if it wasn’t passing the instance wp scripts at all and used the global wp_scripts() function, the same way wp_enqueue_script does. It would be even better if WP_Scripts was an actual singleton since that’s what they’re attempting to do with the global variable. But what the heck do I know?

The Hack

To fix this, jump into your core files and find class-wp-hook.php, roll on down to line apply_filters and make the function look like this:

public function apply_filters($value, $args)
{
    if (!$this->callbacks) {
        return $value;
    }

    $nesting_level = $this->nesting_level++;

    $this->iterations[$nesting_level] = array_keys($this->callbacks);
    $num_args = count($args);

    do {
        $this->current_priority[$nesting_level] = $priority = current($this->iterations[$nesting_level]);

        foreach ($this->callbacks[$priority] as $the_) {
            if (!$this->doing_action) {
                $args[0] = $value;
            }

            // Avoid the array_slice if possible.
            if ($the_['accepted_args'] == 0) {
                $value = call_user_func_array($the_['function'], array());
            } elseif ($the_['accepted_args'] >= $num_args) {
                $special_cases = array('wp_default_scripts', 'wp_default_packages', 'wp_default_styles');
                if (in_array($the_['function'], $special_cases)) {
                    $arg = $args[0];
                    $args = array(&$arg);
                }
                $value = call_user_func_array($the_['function'], $args);
            } else {
                $value = call_user_func_array($the_['function'], array_slice($args, 0, (int)$the_['accepted_args']));
            }
        }
    } while (false !== next($this->iterations[$nesting_level]));

    unset($this->iterations[$nesting_level]);
    unset($this->current_priority[$nesting_level]);

    $this->nesting_level--;

    return $value;
}

This will get rid of the warnings by passing the arguments to the default functions as references. Of course, if you upgrade PHP this hack will get overwritten and you’ll have to do it again if the fix isn’t actually put in the core.

Hope that helps you out and thanks for visiting!

Categories
Coding Games PHP

The Triviosity Rewrite

The Rewind

In 2010 I put a game online. Triviosity, a daily trivia game with occasional specially crafted challenges, has been online for over 8 year. It’s a game that people play on a regular basis and it’s always held a piece of my heart. When I originally wrote it I had full intent to make it a mobile app. An iOS app specifically. The original was built with Zend Framework 1, had a clunky API back-end with intent for the mobile app, and quite frankly it was rock solid. It ran without issues. Unfortunately the iOS app never happened, and as other frameworks and tools blossomed, ZF1 became outdated and not worth the time to build upon it.

Fast Forward

On thanksgiving weekend in October 2017 I decided it was time to rewrite my old game. It was time to switch up the technology and actually make the mobile app. At first glance I thought it was going to be a very daunting task. Switching from ZF1 to Laravel 5.5 seemed like it was going to be a huge undertaking. But it wasn’t. My data models from the original game were so awesome that porting the “beef” of it (I like to work with fat models) only took me a few days.

I sat on that small victory for months. Enter the void of hesitation. Was it too easy? Was I going about it all wrong? Was I doing this the right way? Is anyone new even going to play this rewrite?

If there’s one thing I absolutely detest it’s doing something over. I had to do this, but I hated doing this. There was a necessary evil at play – I want to build the new mobile apps. But I need to redo the old game to get there.

Then I got rolling. I don’t know what changed. I got about 90% complete then I stopped.

Who’s going to play this thing? Why am I doing this? Question, doubt, rinse, repeat.

Then the tidal wave hit. Frack it. “I’m putting this live.” It’s “good enough” and I can fix the missing parts. As a matter of fact, once I put it live, my desire to finish it and make it better went up exponentially.

On July 30, 2018 the unscheduled launch of the new Triviosity happened. A few hundred error emails later, and a bunch of random fixes, the dust had settled. And it’s alive.

Performance

There are two things that take precedence in my work. Security and Performance. Without going into too much detail about security (I can’t give away all my secrets) the new API includes request signing, encryption, and wonderful third party oAuth based authentication.

For performance, It was critical that I make this as fast as I possibly can while still leveraging my old (affordable) web servers. After launching you can see a substantial drop in load times and render times. (This isn’t just the home page either, it’s all pages.)

look at those performances
super graphs

The kicker was simple. Use the tool-set you have properly. As I mentioned, this was being built with Laravel 5.5 (and upgraded to 5.6 before my actual launch). As such, there’s one hell of a generally under-utilized tool-set available to me to use. So here’s a simplified rundown:

  1. CacheCache all the things you can. Cache your config files, routes, database queries. There are pages (score listings) that have a substantial amount of queries of data that doesn’t change often. Caching this turned render time from 500ms to 100ms.
  2. Compile / Minify CSS and JS – Laravel has a webpack config that utilizes a basic Laravel Mix out of the box. It has many nifty plugins to do everything you need. I only really needed the sass compiler, minifier, and JS minifier. This task is often overlooked and it yields some of the greatest performance boosts. The production build tool is A+.
  3. Offload to the Queue! – Using an asynchronous queue to handle things that don’t need to happen for the end user to continue is also a huge boost to performance. Things that can happen in the background: big database updates, sending emails, generating images – all done without the user feeling the slightest bit of a slowdown.
  4. Use the F’ing Framework – Far too often (I’m guilty of this too) I see code that “uses” a framework, but then it doesn’t actually use the framework. Laravel and it’s dependencies have been built up over the years to do a lot of the repetitious work. If I told you the code for the game controller was only 230 lines, you probably wouldn’t believe me. I leverage the framework’s models and view renderer to do the majority of the work. For reference, the old game play code was almost 1000 lines. Less code, same game, more efficient.
  5. Ditch The Cruft – I don’t use any external third party JS on this site with the exception of Google Analytics and Google AdSense (non blocking). Facebook, Google, and Twitter authentication is completely via server side redirects. The share buttons are just good old fashioned native links (which, as a bonus, launch the native apps on most mobile devices).

This is how you get a double A’s on speed tests.

The Future

Now that I have successfully released the best performing site of my life, I can finally work on the native mobile apps. I have no direct timeline for these. I’ll be building them as 100% native apps (Swift for iOS and Kotlin for Android) and not using some goofy cross-platform build tool so that will have some affect on the time frame. But they’ll be fast. They’ll be secure. They’ll be just as awesome and performance ready as the site itself.

Now with an even more rock-solid foundation I can build on this. We can all expect more curated challenges with great features.

TL;DR

It took me about 10 months to complete the rewrite. But in reality only 40 days. It kicks ass. It’s fast. It’s great. The mobile versions are coming soon.

Now go play some Triviosity. And let me know if you break something.

Categories
Linux PHP Ubuntu Zend Framework

The Switch: Apache + Mod_PHP to Nginx + PHP-FPM

File this under “another thing I should’ve done ages ago.”

I decided that I should explore the world of Nginx as a web server since many people have been telling me it’s good. And all I can say is holy shit, it’s good. The setup was simple and after a few idiotic mistakes on my part, it was up and running.

At first I was skeptical as to how fast it would be and with my first couple of benchmarks, nginx was definitely faster.. but not by much. With just a simple php file on a very low resource machine (Ubuntu 11.10, on a 256MB VM at rackspace which I use for playing around) I used ‘ab’ to test 1000 requests with 10 concurrent:

Nginx:
Concurrency Level:      10
Time taken for tests:   0.473 seconds
Complete requests:      1000
Total transferred:      191000 bytes
HTML transferred:       26000 bytes
Requests per second:    2112.79 [#/sec] (mean)
Time per request:       4.733 [ms] (mean)
Time per request:       0.473 [ms] (mean, across all concurrent requests)
Transfer rate:          394.09 [Kbytes/sec] received

Apache:
Concurrency Level:      10
Time taken for tests:   0.533 seconds
Complete requests:      1000
Total transferred:      245000 bytes
HTML transferred:       26000 bytes
Requests per second:    1877.53 [#/sec] (mean)
Time per request:       5.326 [ms] (mean)
Time per request:       0.533 [ms] (mean, across all concurrent requests)
Transfer rate:          449.21 [Kbytes/sec] received

As you can see from the initial benchmark, there’s not much difference, but it is noticeable. And if you throw even more at it I’m pretty sure the gap will be bigger.  One thing that stood out most to me is the extra amount of data that Apache sends.

After I setup a zend framework application, I ran the benchmarks again. Same 10 concurrent, 1000 requests:

Nginx:
Concurrency Level:      10
Time taken for tests:   15.892 seconds
Complete requests:      1000
Total transferred:      3735000 bytes
HTML transferred:       3577000 bytes
Requests per second:    62.92 [#/sec] (mean)
Time per request:       158.922 [ms] (mean)
Time per request:       15.892 [ms] (mean, across all concurrent requests)
Transfer rate:          229.51 [Kbytes/sec] received

Apache:
Concurrency Level:      10
Time taken for tests:   17.724 seconds
Complete requests:      1000
Total transferred:      3791000 bytes
HTML transferred:       3577000 bytes
Requests per second:    56.42 [#/sec] (mean)
Time per request:       177.242 [ms] (mean)
Time per request:       17.724 [ms] (mean, across all concurrent requests)
Transfer rate:          208.88 [Kbytes/sec] received

Again, the difference is there. Nginx is clearly faster. It’s clearly winning. But I’m still just benchmarking with settings that I know Apache can handle on the low resource box. And this of course is all about resources and effectively using them. So I pumped it up. Time to do ab -c 100 -n 10000, ten thousand requests with one hundred concurrent and the results are amazing:

nginx:
Concurrency Level:      100
Time taken for tests:   122.030 seconds
Complete requests:      10000
Total transferred:      37350000 bytes
HTML transferred:       35770000 bytes
Requests per second:    81.95 [#/sec] (mean)
Time per request:       1220.301 [ms] (mean)
Time per request:       12.203 [ms] (mean, across all concurrent requests)
Transfer rate:          298.90 [Kbytes/sec] received

Apache:
CRASHED after 485 requests.
apr_poll: The timeout specified has expired (70007)
Total of 485 requests completed
load average: 83.73, 30.80, 11.43

The server load under apache went into a state of pure cluster-fuck. Apache could not contain itself with 100 concurrent connections on a box with such low resources, whereas Nginx handled it with EASE. The requests per second were slightly slower at 81.96 when doing 100 concurrent connections, but that request count is still amazing compared to apache crashing.

I’m sorry Apache+mod_php, you lose. Now it’s time to migrate all my stuff.

Categories
PHP Randomness Zend Framework

Just a Log

Applications, especially ones that run over and over again with zero persistancy (of the application itself) like a web app needs logging. It’s important to be able to log different types of messages to different places and fortunately Zend_Log is so extensible that it can log to almost anything you can think of.

My typically logging setup consists of multiple log writers configured in Zend_Log to allow me to control where and how certain messages get logged.

Broken down into logging levels I usually have ‘debug’, ‘info’, ‘notice’ all sent to a single file and not enabled in production.
Warnings and Errors are usually considered runtime-issues that are not “omg! it’s broken!” issues, so they get logged to a file and are enabled on production. The last three levels, ‘alert’, ‘crit’, ’emerg’ are all considered to be top priority – aka – your application is failing. And as such these get logged to a file and emailed immediately to a person of importance.

In all cases, in my development environment, all log messages utilize Zend’s Firebug support.

Fortunately Zend_Log supports all this complexity with a very simple set of configuration options.

It’s all done with a few lines of settings in your application.ini file and voila, magic.

Here’s how to bootstrap your logger and set it to a Zend_Registry key for easy use:


protected function _initRegisterLogger() {
    $this->bootstrap('Log');
    $logger = $this->getResource('Log');
    Zend_Registry::set('Zend_Log', $logger);
}

And here’s a sample application.ini – with omitted portions so you can just see the logging items.


[production]
; the operand param doesn't allow for y > 4 & y < 7 so we need multiple writers.
resources.log.production.writerName = "Stream"
resources.log.production.writerParams.stream = APPLICATION_PATH "/logs/production-critical.log"
resources.log.production.filterName = "Priority"
resources.log.production.filterParams.priority = 3
resources.log.production.filterParams.operand  = "<"
resources.log.production1.writerName = "Stream"
resources.log.production1.writerParams.stream = APPLICATION_PATH "/logs/production-errors.log"
resources.log.production1.FilterName = "Priority"
resources.log.production1.filterParams.priority = 4
resources.log.production1.filterParams.operand  = "="
resources.log.production2.writerName = "Stream"
resources.log.production2.writerParams.stream = APPLICATION_PATH "/logs/production-warning.log"
resources.log.production2.filterName = "Priority"
resources.log.production2.filterParams.priority = 5
resources.log.production2.filterParams.operand  = "="

[testing : production]
resources.log.testing.writerName = "Stream"
resources.log.testing.writerParams.stream = APPLICATION_PATH "/logs/testing-notices.log"
resources.log.testing.filterName = "Priority"
resources.log.testing.filterParams.priority = 5
resources.log.testing.filterParams.operand  = "="
resources.log.testing1.writerParams.stream = APPLICATION_PATH "/logs/testing-info.log"
resources.log.testing1.filterName = "Priority"
resources.log.testing1.filterParams.priority = 6
resources.log.testing1.filterParams.operand  = "="

[development : testing]
; this logs all messages
resources.log.testing.writerName = "Stream"
resources.log.testing.writerParams.stream = APPLICATION_PATH "/logs/debug.log"
resources.log.testing1.filterName = "Priority"
resources.log.testing1.filterParams.priority = 7
resources.log.testing1.filterParams.operand  = "="
; no filter on firebug, logs everything.
resources.log.firebug.writerName = "Firebug"
Categories
Coding PHP Zend Framework

Queueing With Zend Queue and MemcacheQ

I was bored last night so I thought I’d enhance one of my applications with a little bit of Queueing. (And for the record both Queueing and Queuing are valid spellings depending on your locale)

The concept is fairly straight forward. Normally when a web application runs, it runs in a linear pattern. It starts, it does stuff, it does more stuff, then it finishes with a web page. That’s pretty much it. The problem is, the user has to wait for it to do stuff and more stuff. And when they have to wait for more and more stuff, it gets slow. If some of that stuff can be offloaded because it has no affect on what the user sees right now – why not? That’s where a Queue comes in to play. Send a command to a queue that something else can process for you.

So enter Zend Queue. A feature of Zend Framework that will let you use various queue systems to your advantage. I chose MemcacheQ because it was simple to install and I knew that I’d need not do anything to my configurations to utilize it. MemcacheQ is a light-weight, fast, persistant queueing system (It’ll remember what it has when it gets shut down) based on Memcache’s protocol. So accessing the data in the queue is a piece of cake.

I tried using Zend_Queue_Adapter_Memcacheq in Zend Framework 1.10.8 – but it failed. It doesn’t seem to work properly with MemcacheQ 0.2.0 which is the latest version. That lead me to write my own little adapter – based on the existing one – to fix the incompatibilities. I also added automatic serialization to the messages sent to the queue. This allows me to send useful items, instead of just strings. I implemented the queue system in a way that I can now do queued inserts and updates on database rows wherever I see the need.  It’s also setup in a way that if sending a message to the queue fails, it will do the desired action right away (slower for the user, but no loss of data.)  After that I was set.

I send my messages to my queue as described in Zend Framework’s documentation ($queue->send($bundle)); My bundle is a small array containing a few useful items. One: a class name, two: a method to call, three: parameters for the method. And that’s it. Now I’m queueing magical commands.  I need to process the queue.

I then wrote a queue processor. This processor is a very small command line tool that utilizes my existing Zend Framework applications configuration – so it has all the access it needs to the application database and code base. The queue processor pulls in a small set of bundles from the queue – executes them – and waits for a short period of time and then processes more.  In the event that a command fails, the queue processor will re-queue the item and wait to try again.  This is to prevent data loss.  If for some reason the database is unavailable the updates and inserts will persist in the queue until they can be executed properly.

I hope I’ve explained it in enough detail that you can understand the process. If not, leave a comment and I’ll try and clarify it. I’m not posting any code just yet because it’s not “pretty.”

So how well did it do? Judge for yourself. You can tell when I enabled it.

Categories
Coding PHP Zend Framework

Tell Your Redirector How to Do It’s Job

There’s a big difference between a redirect that uses a 301 and a 302 code on a website. If you don’t know the difference, you should find out.
301 says “This stuff has moved permanently and you shouldn’t come here anymore.”
302 says “You just need to go here, but this place will still answer you later.”

The appropriate use isn’t too complicated if you understand those two little statements. Within a web application there are many redirects that take place. If you use the 301 in these situations web browsers might get confused and your application won’t work properly.

By default the Zend Framework action helper aptly called “redirector” defaults to using 302’s; which is fine for most cases. But if you happen to be in a refactoring mode and rearranging your code and url structure it’s super important to use 302’s or your search engine rankings can fall right off the table.

Telling the _redirect helper to use a 301 code is really quite easy:

$this->_helper->redirector->setCode(301);
$this->_redirect('/newplace');

Unfortunately it’s a bit strange, because at first you think you can just use ‘$this->_redirect(‘destination’, code); but you can’t and that sucks.  If you have multiple case redirects and some might be 301’s or 302’s you should make sure you set the code back before redirecting.