the most awesome guy ever.

The Blog of Darryl E. Clarke

  Random musings from a jaded coder who just needs a hug.

Archive for the ‘Zend Framework’ Category

The Switch: Apache + Mod_PHP to Nginx + PHP-FPM

Thursday, December 22nd, 2011

File this under “another thing I should’ve done ages ago.”

I decided that I should explore the world of Nginx as a web server since many people have been telling me it’s good. And all I can say is holy shit, it’s good. The setup was simple and after a few idiotic mistakes on my part, it was up and running.

At first I was skeptical as to how fast it would be and with my first couple of benchmarks, nginx was definitely faster.. but not by much. With just a simple php file on a very low resource machine (Ubuntu 11.10, on a 256MB VM at rackspace which I use for playing around) I used ‘ab’ to test 1000 requests with 10 concurrent:

Nginx:
Concurrency Level:      10
Time taken for tests:   0.473 seconds
Complete requests:      1000
Total transferred:      191000 bytes
HTML transferred:       26000 bytes
Requests per second:    2112.79 [#/sec] (mean)
Time per request:       4.733 [ms] (mean)
Time per request:       0.473 [ms] (mean, across all concurrent requests)
Transfer rate:          394.09 [Kbytes/sec] received

Apache:
Concurrency Level:      10
Time taken for tests:   0.533 seconds
Complete requests:      1000
Total transferred:      245000 bytes
HTML transferred:       26000 bytes
Requests per second:    1877.53 [#/sec] (mean)
Time per request:       5.326 [ms] (mean)
Time per request:       0.533 [ms] (mean, across all concurrent requests)
Transfer rate:          449.21 [Kbytes/sec] received

As you can see from the initial benchmark, there’s not much difference, but it is noticeable. And if you throw even more at it I’m pretty sure the gap will be bigger.  One thing that stood out most to me is the extra amount of data that Apache sends.

After I setup a zend framework application, I ran the benchmarks again. Same 10 concurrent, 1000 requests:

Nginx:
Concurrency Level:      10
Time taken for tests:   15.892 seconds
Complete requests:      1000
Total transferred:      3735000 bytes
HTML transferred:       3577000 bytes
Requests per second:    62.92 [#/sec] (mean)
Time per request:       158.922 [ms] (mean)
Time per request:       15.892 [ms] (mean, across all concurrent requests)
Transfer rate:          229.51 [Kbytes/sec] received

Apache:
Concurrency Level:      10
Time taken for tests:   17.724 seconds
Complete requests:      1000
Total transferred:      3791000 bytes
HTML transferred:       3577000 bytes
Requests per second:    56.42 [#/sec] (mean)
Time per request:       177.242 [ms] (mean)
Time per request:       17.724 [ms] (mean, across all concurrent requests)
Transfer rate:          208.88 [Kbytes/sec] received

Again, the difference is there. Nginx is clearly faster. It’s clearly winning. But I’m still just benchmarking with settings that I know Apache can handle on the low resource box. And this of course is all about resources and effectively using them. So I pumped it up. Time to do ab -c 100 -n 10000, ten thousand requests with one hundred concurrent and the results are amazing:

nginx:
Concurrency Level:      100
Time taken for tests:   122.030 seconds
Complete requests:      10000
Total transferred:      37350000 bytes
HTML transferred:       35770000 bytes
Requests per second:    81.95 [#/sec] (mean)
Time per request:       1220.301 [ms] (mean)
Time per request:       12.203 [ms] (mean, across all concurrent requests)
Transfer rate:          298.90 [Kbytes/sec] received

Apache:
CRASHED after 485 requests.
apr_poll: The timeout specified has expired (70007)
Total of 485 requests completed
load average: 83.73, 30.80, 11.43

The server load under apache went into a state of pure cluster-fuck. Apache could not contain itself with 100 concurrent connections on a box with such low resources, whereas Nginx handled it with EASE. The requests per second were slightly slower at 81.96 when doing 100 concurrent connections, but that request count is still amazing compared to apache crashing.

I’m sorry Apache+mod_php, you lose. Now it’s time to migrate all my stuff.

Tags: , , ,
Posted in Linux, PHP, Ubuntu, Zend Framework

Just a Log

Saturday, November 27th, 2010

Applications, especially ones that run over and over again with zero persistancy (of the application itself) like a web app needs logging. It’s important to be able to log different types of messages to different places and fortunately Zend_Log is so extensible that it can log to almost anything you can think of.

My typically logging setup consists of multiple log writers configured in Zend_Log to allow me to control where and how certain messages get logged.

Broken down into logging levels I usually have ‘debug’, ‘info’, ‘notice’ all sent to a single file and not enabled in production.
Warnings and Errors are usually considered runtime-issues that are not “omg! it’s broken!” issues, so they get logged to a file and are enabled on production. The last three levels, ‘alert’, ‘crit’, ’emerg’ are all considered to be top priority – aka – your application is failing. And as such these get logged to a file and emailed immediately to a person of importance.

In all cases, in my development environment, all log messages utilize Zend’s Firebug support.

Fortunately Zend_Log supports all this complexity with a very simple set of configuration options.

It’s all done with a few lines of settings in your application.ini file and voila, magic.

Here’s how to bootstrap your logger and set it to a Zend_Registry key for easy use:


protected function _initRegisterLogger() {
    $this->bootstrap('Log');
    $logger = $this->getResource('Log');
    Zend_Registry::set('Zend_Log', $logger);
}

And here’s a sample application.ini – with omitted portions so you can just see the logging items.


[production]
; the operand param doesn't allow for y > 4 & y < 7 so we need multiple writers.
resources.log.production.writerName = "Stream"
resources.log.production.writerParams.stream = APPLICATION_PATH "/logs/production-critical.log"
resources.log.production.filterName = "Priority"
resources.log.production.filterParams.priority = 3
resources.log.production.filterParams.operand  = "<"
resources.log.production1.writerName = "Stream"
resources.log.production1.writerParams.stream = APPLICATION_PATH "/logs/production-errors.log"
resources.log.production1.FilterName = "Priority"
resources.log.production1.filterParams.priority = 4
resources.log.production1.filterParams.operand  = "="
resources.log.production2.writerName = "Stream"
resources.log.production2.writerParams.stream = APPLICATION_PATH "/logs/production-warning.log"
resources.log.production2.filterName = "Priority"
resources.log.production2.filterParams.priority = 5
resources.log.production2.filterParams.operand  = "="

[testing : production]
resources.log.testing.writerName = "Stream"
resources.log.testing.writerParams.stream = APPLICATION_PATH "/logs/testing-notices.log"
resources.log.testing.filterName = "Priority"
resources.log.testing.filterParams.priority = 5
resources.log.testing.filterParams.operand  = "="
resources.log.testing1.writerParams.stream = APPLICATION_PATH "/logs/testing-info.log"
resources.log.testing1.filterName = "Priority"
resources.log.testing1.filterParams.priority = 6
resources.log.testing1.filterParams.operand  = "="

[development : testing]
; this logs all messages
resources.log.testing.writerName = "Stream"
resources.log.testing.writerParams.stream = APPLICATION_PATH "/logs/debug.log"
resources.log.testing1.filterName = "Priority"
resources.log.testing1.filterParams.priority = 7
resources.log.testing1.filterParams.operand  = "="
; no filter on firebug, logs everything.
resources.log.firebug.writerName = "Firebug"

Tags: , , , , , ,
Posted in PHP, Randomness, Zend Framework

Queueing With Zend Queue and MemcacheQ

Thursday, November 18th, 2010

I was bored last night so I thought I’d enhance one of my applications with a little bit of Queueing. (And for the record both Queueing and Queuing are valid spellings depending on your locale)

The concept is fairly straight forward. Normally when a web application runs, it runs in a linear pattern. It starts, it does stuff, it does more stuff, then it finishes with a web page. That’s pretty much it. The problem is, the user has to wait for it to do stuff and more stuff. And when they have to wait for more and more stuff, it gets slow. If some of that stuff can be offloaded because it has no affect on what the user sees right now – why not? That’s where a Queue comes in to play. Send a command to a queue that something else can process for you.

So enter Zend Queue. A feature of Zend Framework that will let you use various queue systems to your advantage. I chose MemcacheQ because it was simple to install and I knew that I’d need not do anything to my configurations to utilize it. MemcacheQ is a light-weight, fast, persistant queueing system (It’ll remember what it has when it gets shut down) based on Memcache’s protocol. So accessing the data in the queue is a piece of cake.

I tried using Zend_Queue_Adapter_Memcacheq in Zend Framework 1.10.8 – but it failed. It doesn’t seem to work properly with MemcacheQ 0.2.0 which is the latest version. That lead me to write my own little adapter – based on the existing one – to fix the incompatibilities. I also added automatic serialization to the messages sent to the queue. This allows me to send useful items, instead of just strings. I implemented the queue system in a way that I can now do queued inserts and updates on database rows wherever I see the need.  It’s also setup in a way that if sending a message to the queue fails, it will do the desired action right away (slower for the user, but no loss of data.)  After that I was set.

I send my messages to my queue as described in Zend Framework’s documentation ($queue->send($bundle)); My bundle is a small array containing a few useful items. One: a class name, two: a method to call, three: parameters for the method. And that’s it. Now I’m queueing magical commands.  I need to process the queue.

I then wrote a queue processor. This processor is a very small command line tool that utilizes my existing Zend Framework applications configuration – so it has all the access it needs to the application database and code base. The queue processor pulls in a small set of bundles from the queue – executes them – and waits for a short period of time and then processes more.  In the event that a command fails, the queue processor will re-queue the item and wait to try again.  This is to prevent data loss.  If for some reason the database is unavailable the updates and inserts will persist in the queue until they can be executed properly.

I hope I’ve explained it in enough detail that you can understand the process. If not, leave a comment and I’ll try and clarify it. I’m not posting any code just yet because it’s not “pretty.”

So how well did it do? Judge for yourself. You can tell when I enabled it.

Tags: , , , ,
Posted in Coding, PHP, Zend Framework

Tell Your Redirector How to Do It’s Job

Sunday, October 17th, 2010

There’s a big difference between a redirect that uses a 301 and a 302 code on a website. If you don’t know the difference, you should find out.
301 says “This stuff has moved permanently and you shouldn’t come here anymore.”
302 says “You just need to go here, but this place will still answer you later.”

The appropriate use isn’t too complicated if you understand those two little statements. Within a web application there are many redirects that take place. If you use the 301 in these situations web browsers might get confused and your application won’t work properly.

By default the Zend Framework action helper aptly called “redirector” defaults to using 302’s; which is fine for most cases. But if you happen to be in a refactoring mode and rearranging your code and url structure it’s super important to use 302’s or your search engine rankings can fall right off the table.

Telling the _redirect helper to use a 301 code is really quite easy:

$this->_helper->redirector->setCode(301);
$this->_redirect('/newplace');

Unfortunately it’s a bit strange, because at first you think you can just use ‘$this->_redirect(‘destination’, code); but you can’t and that sucks.  If you have multiple case redirects and some might be 301’s or 302’s you should make sure you set the code back before redirecting.

Tags: , , , , ,
Posted in Coding, PHP, Zend Framework

Database Profiling with Zend Db Profiler Firebug

Wednesday, October 6th, 2010

I don’t know why I haven’t been using this for ages. But Zend’s DB Profiler is fun. Only to be used in development so you can see what your database is doing and how fast/slow it’s doing it.

Enable it in your application’s config.ini (or .xml)

[development]
db.profiler.enabled = true
db.profiler.class   = "Zend_Db_Profiler_Firebug"

And as long as you’re using the DB Factory to create your connection, everything that happens will be automatically profiled and spewed out via Firebug’s awesome logging.  It shows you time spent, number of queries, the queries themselves and parameters used.  It’s awesome.  Why haven’t I been using this before?

You can also enable it directly on your database object if you don’t want it in your config; for example I have block of code that runs only when debugging is enabled (development environment) and as such I might want to enable the profiler there:

$dbAdapter->setProfiler(
       array('enabled' => true, 'class' => "Zend_Db_Profiler_Firebug"));

Yeah, it’s awesome.

Tags: , , , , ,
Posted in PHP, Zend Framework

The Quickest Way To Tweet via Zend Framework’s Twitter Service

Sunday, September 26th, 2010

This is a quick and dirty method.  It assumes a couple of things.

  1. You’ve already created your ‘app’ on twitter.
  2. You’ve already gotten your personal OAuth tokens from twitter.

First you need to setup your tokens;

$token = new Zend_Oauth_Token_Access();
$token->setToken("YOUR_PERSONAL_TOKEN")
      ->setTokenSecret("YOUR_PERSONAL_SECRET");

Second, you need to setup the Zend_Service_Twitter using your access token as well as the application tokens provided by twitter.

$options = array(
 'accessToken'    => $token,
 'callbackUrl'    => 'http://' .$_SERVER['HTTP_HOST']. '/',
 'consumerKey'    => "YOUR_APPLICATION_KEY",
 'consumerSecret' => "YOUR_APPLICATION_SECRET");

 $twitter = new Zend_Service_Twitter($options);
 $response = $twitter->status->update($message);

And that’s it.

If you want to do other things like authenticate multiple users, you’ll have to do a few other steps. Twitter was nice enough to make it “easy” for a single user to create their own app and post without any goofy intermediate authentication steps.

Tags: , ,
Posted in Coding, PHP, Zend Framework