The Linux Bloke

I run a number of web sites with WordPress, and have recently upgraded many of them to WordPress 3.0.1. It seems that every time I have to upgrade WordPress that some of the many and varied plugins are simply broken.

Why is this? I mean, in this day and age of well-defined APIs, OOP, proper object factorization, unit testing, and the like, you’d think this would rarely happen, right?

But, alas, the real problem is that most WordPress plugins are simply horribly written. Period. Way too much dependence on global variables, no use of PHP’s OOP capabilities (even though WordPress definitely supports OOP), and just poor code organization all around.

So, whenever WordPress changes something, lots of plugins simply fail to work properly. It’s so prevalent it drives me mad.

Proper software design principles applies whether you are programming in C++, Ruby, Perl, Java, or PHP. We have those design principles in place for a reason. Popular applications are never static, and should never be expected to be either. That’s why we do certain things in a certain way, folks!

Of all the languages I know, I consider PHP to be the absolute worst, basically the Basic of the 21st century. Because just like Basic, PHP allows to get away with many ills. It allows you to write very sloppy code very quickly, and actually get it to work “good enough” to throw into production!!!!

But when you are talking frameworks and plugins, there is no room for being sloppy. When you are talking millions of users that must rely on your plugins, there are simply no place for taking shortcuts. You do the job right, or not at all. You adhere to the well-established, sound design principles that we have worked out over the past 30 years or so, or you go back to “school” to get a clue.

But I really fault PHP for allowing such evils in the first place. Ruby and Python strongly encourages you to do the right thing when you write code. Ruby on Rails is sweet in this regard.

In my experience (and I have 30 years of it!), it takes just as long to write good code as it does to write bad code. With bad code, you spend much more time debugging it, and “fixing” the bugs probably entails writing more sloppy code to work around the existing sloppiness. So time wise, you’re a penny wise, pound foolish.

And then comes maintainability. With poorly written code, forget it. The time it takes to maintain it blossoms exponentially over time as the underpinnings shift and evolve over time. With Open Source development, what usually happens over time is one of the following:

1. The code is abandoned and everyone stops using it.
2. The code is re-written from the ground up (and using proper design principles finally!!!!)
3. The code has become critical to many applications, but no one truly wants to maintain it because it’s so horrible, so it “limps along” with the barest minimum effort applied just to keep it — somewhat — running.

All of which could’ve been avoided if the code was written properly up front. That would free developers to work on more cool stuff, giving us even more functionality, and also allow the underlying frameworks to also grow and expand without worry of breaking all the plugins and themes out there.

So get a clue you bad PHP code slingers out there! It’s not hard at all writing good code, and is actually quite enjoyable. Spend less time playing video games and more time to educate yourself. It’ll look good on your resume and improve your bottom line. And make those who use your code happier. Why? Because your code won’t call attention to itself by not working, and your name is far less likely to become an expletive.

It’s up to you. Only You can write Good Code. If not you, then who else?

It is no joke that computer hardware has advanced by leaps and bounds over the past decade. 10 years ago, multicore systems were expensive and high-end; today, your grandmother may have one and probably have no clue what she has!

Alas, application software has not kept pace. The Linux OS has done a fair job at being able to leverage some of the power multicore systems offer us, but applications running on them have not. The same can be said more or less for Windows, but it’s been a long while since I did anything systems-level with Windows. But the same issues do apply, however.

We are today with the multicore situation where we were in the 80′s and the 90′s with the multithreaded issues. Back then, CPUs grew support for multithreaded programming, but software — including some OSes — were slow to adopt. The Macintosh, when it was first released in 1984, would only support “cooperative” multitasking when the underlying 68000 was perfect for preemption, which was leveraged by the Amiga that came out a year later in 1985.

Even I wrote a preemptive multitasking OS back in 1980, when I was 18 years old! I had a hard time understanding why Apple and Microsoft were having such a hard time with this 5 years later. The first preemptive OS released by Microsoft was Windows NT, that came out in the late 80′s, shortly after the time Commodore closed its doors.

I to this day am still mystified why big corporations like Apple and Microsoft couldn’t pull off until much later what an 18 year-old such as myself could do in a couple of months!

Today, we are in a similar situation with applications — especially those applications that should be able to leverage multicore power such as Databases and Games and Web Servers. Actually, Web Servers such as Apache doesn’t fair too bad in this area from what I can tell. Can’t speak to IIS, since I’ve never used it. The MySQL Database struggles to be able to leverage the multicore, but the developers there seem to only take incremental steps. For a while MySQL’s performance used to actually degrade on multicore platforms! Now it’s been optimized for at least 8 cores, but you still may not see the expected gain on 16 or 32-core systems.

Why would this be the case? Surely, MySQL and other applications are written to be *multithreaded*. Ah, there lies the rub. Without getting deep into the details of system resource allocation, spin-locking, and the like, I wish to discuss this issue from a “high-level” perspective. I’m sure some of you not-so-tech-savvy tech managers would appreciate that!

And, by the way, it’s nothing wrong with not understanding all the nitty-gritty details of semaphores and synchronization issues if you are a tech manager. At least, as long as your people do. But then, that’s part of the problem. Some do; many don’t. And those that don’t may not be as forthcoming as you’d like for fear of being fired or ridiculed or being given a lower status, etc. That’s the way it is.

From the bird’s point of view, it’s not too hard to understand at all. The basic difference in multithreaded programming and multicore programming is this:

If you are doing multithreaded programming for single-core systems, you gain nothing for trying to do a lot of processing simultaneously in multiple threads. The goal of multithreaded programming is to, instead, keep the idle time of the CPU as low as possible whilst doing the most work possible where the work will be done in serial fashion anyway. If you actually do attempt to do real processing in multiple threads, in most cases your performance will actually degrade faster than if you did it using serial programming. Why? Because the processor takes time to context-switch between tasks, and the more it has to do that, the more overhead you’ll incur.

On a multicore system, on the other hand, your goals are quite different — you actually do want to spread the work out among the cores so that it DOES executes simultaneously, because your performance gain should be directly related to how many cores you have. So, a quad core system should be able to get the work done 4 times faster than a single core system. A 16-core platform should be 4-times faster than a quad-core system, and 16-times faster than a single core.

Ah, but as always, there’s a catch. Hence my use of the words, “should be”, rather than “will be”.

Making efficient use of multiple cores in highly non-trivial. For starters, you may not be able to break the tasks down to a parallelizable form. Or if you can, there may still be dependencies between the tasks where one would have to wait on another for information, or wait for a common resource — such as the hard drives or network cards — to become available.

In a dynamic situation, such as a database server, the issues can become even more convoluted as you deal with the order many resources — such as rows in a table — are locked by many tasks running on many cores.

If you are having to deal with legacy code not designed for multicore systems, as was the case with the MySQL codebase, the issues becomes even hairier.

Also, most languages in popular use, such as C++, Java, Python and Ruby, have little to no facillities for multicore or distributed programming.  Interpreted scripting languages like Python don’t even handle multithreading very well, at least Python pre 3.0. Ruby has issues in this regard as well.

The common wisdom with some is to run your program in multiple processes, which does work for those situations that doesn’t require a lot of state dependencies or resource sharing. That approach, when it works, is nice, because it scales well with the number of cores you have — and it also scales well in a cluster/cloud computing scenario.

But if those simultaneously running systems DO require a lot of sharing of data, resources, and other dependencies, the scale factor is severely restricted. It may call for a complete rework of the algorithms involved or some clever system hacks, or both.

I don’t think there is any common wisdom that can be applied in all cases. And when you are dealing with time-to-market constraints, budgetary realities, and the like, you may be forced to take a sub-optimal path.

So it may be safe to say that it may be a little while before we see software truly leverage the true power of multicore systems. We will see it here and there where the effort can be applied and the understanding is present, but the rest will entail a slow evolutionary process.

MIT offers a course on multicore programming. Can’t say how good or bad it is, but it’s MIT. How can you go wrong?

Let’s say you need to do a website that must support multiple languages for cultures as diverse as Japan, France, Russia, Saudi Arabia, and Brazil, as well as the US. This can be quite a daunting task, with all kinds of unexpected gotchas.

The ideal character set of choice is, of course, UTF8. Alas, you will note that most of the systems you’ll need to use defaults to LATIN1, including MySQL. If your site is written in PHP, that also by default is set to LATIN1.

I find it quite puzzling that in this day and age of globalization that many of the tools don’t default to UTF8. And there are major issues with this, because everything in the chain of delivery must either be set to UTF8 or can handle UTF8 or you’ll see bizarreness when you attempt to display the characters of some languages. You will probably see a series of question marks (“??? ??? ?????”) instead of the actual words. Sometimes you may see a series of squares. Or maybe it looks like total garbage.

To debug charset issues, you must be certain that everything in the delivery chain is set for UTF8. I can’t stres this enough.

For example, on one project, the MySQL database was properly set to UTF8, but we kept seeing LATIN1 creep in from somewhere. The site was driven by PHP, and we made sure PHP was set to UTF8, but there were still issues. It turned out that PDO/mysqli was still defaulting to LATIN1, which was revealed by looking at the results of the following query issued through PHP:

SHOW VARIABLES LIKE “character%”;

Which should result in:

+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | utf8                       |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+

But instead we saw:

+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | latin1                     |
| character_set_connection | latin1                     |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | latin1                     |
| character_set_server     | utf8                       |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+

Clearly demonstrating that there was a connection issue. However, we were able to, as a quick fix, issue the following query on that connection:

SET NAMES ‘utf8′;

Which fixed the problem, though requiring us to run that query on every new connection. I am sure there is a better approach, but we didn’t have time to find it.

But to give you an idea, this is the chain we had to check for UTF8:

• MySQL Server
• MySQL Driver/PDO Wrapper
• PHP
• Browser

If you are interacting with MySQL through the command-line client, then make sure you launch it thusly:

mysql –default-character-set=utf8

Or have the appropriate settings in the [client] section of my.cnf.

The character set headaches are not just limited to MySQL, but any interacting systems, web services, etc. Carefully checking the chain to ensure that every part of that chain defaults to UTF8 is essential to saving the day for the world of localized globalization!