Creating complex shortcodes in WordPress

I occasionally dabble in custom WordPress themes on my wife’s behalf and in furtherance of her vast eBook empire. As part of her current theme, she wanted some shortcodes to do neat things like having a multi-column section of a blog post. That way, she can write:

[column]First column here![/column]
[column]Second column here![/column]

instead of the actual mess of HTML and CSS required to make such magic happen. Plus, if we ever decide to change the markup we’re using to generate columns, we can just change the implementation of the shortcode instead of going and updating a whole bunch of blog posts. Cool feature!

Defining a shortcode

In your theme’s functions.php, you need to do (at minimum) two things:

  1. Create a function that replaces the shortcode with its actual content.
  2. Register your shortcode with WordPress.

This part is straightforward. Say we wanted to create a [contentbox] shortcode to automatically wrapped its contents into an <aside>. That could look something like:

We could use it this way:

[contentbox align="right"]Here's a nice right-aligned piece of content![/contentbox]

When included in a post, that would become:

<aside class='content-box right'>Here's a nice right-aligned piece of content!</aside>

(CSS to make it actually look like a box and aligned right is left as an exercise for the reader.)

The evil that lies within

There’s a problem. WordPress has an ancient, vile, and notorious function in place which is designed to make users’ lives easier and developers’ lives miserable. This function is called wpautop and I hate it passionately.

The idea behind wpautop is simple: convert all double linebreaks into paragraph elements, and tack on <br> tags to whatever’s left. This lets an average user type text into the editor and have it converted to HTML even without using WordPress’ visual editor. The problem is that wpautop doesn’t know anything about shortcodes, so if our user splits our shortcode into several lines (such as the [columns] example at the top of this post), wpautop comes through and sprays <br> tags everywhere. Incidentally, <br> stands for “break”, which is exactly what wpautop is designed to do to nice complex shortcodes.

This problem has been known for ages – it was reported at least 9 years ago as of this writing, was still allegedly being discussed by WordPress devs 7 months ago, and there is no fix. Ridiculous.

Rescuing the shortcodes

There are at least four ways to deal with the problem of wpautop murdering our shortcode content:

Option 1: Change do_shortcode filter priority

While loading a page, WordPress executes a long list of “filters”, which are just functions that operate on the post’s content. wpautop is one such filter, as is do_shortcode. The latter is what ends up calling our own custom shortcode functions, replacing the shortcodes with actual content in the post. In WordPress core, wpautop runs at priority 10 and do_shortcode runs after that, at 11. So one way to thwart wpautop is to change do_shortcode‘s priority to 9, meaning our shortcodes will be resolved first and hopefully wpautop won’t mangle the output too badly.

This isn’t a great solution because it can potentially break all kinds of other themes or plugins (which might rely on WordPress’ default filter order) in subtle ways. Until today, I was using this approach for my wife’s site theme, but then we started switching her shopping cart to WooCommerce and discovered that my hack broke the text on the “proceed to checkout” button (because wpautop vomited a <br> randomly into the button text).

So if you take this route, expect occasional oddities with other plugins. This is not a well-behaved solution.

Option 2: Disable wpautop entirely

I love this option because I hate wpautop. However, this will likely break even more themes/plugins than option 1, and also users may not get the expected results if they’re composing posts in the text editor without using a sensible text parser like Markdown. Still, if you want to do it, just put these two lines into functions.php:

remove_filter( ‘the_content’, ‘wpautop’ );
remove_filter( ‘the_excerpt’, ‘wpautop’ );

Option 3: Disable wpautop on a post-by-post basis

You can install a plugin to disable wpautop only on certain posts–in our case, the ones with shortcodes on it.

I like how the plugin description explains its existence by saying “Back in the day, when the wpautop filter was really sucky…” What, you mean like yesterday?

Option 4: Hide the shortcodes behind a false bookcase

Like squirreling away guns or hiding Jews from the Nazis, this approach pretends that shortcodes don’t exist until after wpautop runs. It’s the most complicated, but it’s probably also the best because it lets wpautop commit whatever debauchery it wants on the post and then puts our precious shortcodes back in place unmolested. There are two steps:

  1. Register a filter before wpautop to search the post content for any shortcodes and replace them with random strings.
  2. Register a filter after wpautop but before do_shortcode to switch out the random strings for the original shortcode text.

Easier said than done, because finding complex shortcodes is no walk in the park. Fortunately, I’ve done the hard work for you!

When included in your functions.php file, the above code will:

  1. Search for any shortcodes in your post/page content.
  2. Once a shortcode opening tag is found, attempt to locate its corresponding closing tag.
  3. Replace the entire shortcode contents in the post with a random string.
  4. After wpautop has run, replace the random strings with the original shortcode markup.

After that, do_shortcode should run and evaluate your shortcodes normally.

On Usability and Folding Laundry

The importance of feeling the consequences of your own work can scarcely be overestimated.

I sometimes fold laundry in our household. Well, okay, I occasionally fold laundry in our household. That is to say, at least twice I have folded laundry in our household. And on one of these occasions, my long-suffering wife asked me a seemingly random question.

How do you hang hand towels?
Hand towels. Like if you’re hanging a towel on the rack, how do you do it?
(picking up a towel I’d just folded) The normal way, threading the narrow end through the towel holder. I’d just unfold the towel once and…oh.

Turns out I was completely incompetent at folding hand towels, because I’d been folding them in the wrong direction. Thus, anytime someone wanted to use one of the towels I’d folded, they had to first unfold it all the way and then re-fold it.

Apart from making me feel like an imbecile, this little experience reminded me of a lesson I’ve seen in action many times. Had I been the one hanging hand towels in addition to the one folding them, the problem would have been immediately obvious. But unless we actually use the end result we’re producing (or have excellent user feedback mechanisms in place), we’re flying blind.

This is especially true in the software world. If you’re not using your own software, then I’ll take long odds that it’s got significant usability problems. I have worked with developers who blame everything on the user, vindicating their own blinkered view of their software product. “gah, these users. If they’d only use the software correctly then we wouldn’t have to make these silly changes.” I cringe every time this happens, and I’ve heard variations on it more than once in my career. If you’re blaming your users for problems they find in your software, it’s time to find another career. The whole point of software development is solving people’s problems. If you’re not interested in doing that, then what are you doing here?

The need for real-world feedback on software is one of the reasons “hallway usability testing” is so valuable. The idea is that it doesn’t take a special kind of user to find usability problems in your application. You can grab just about any random human being who is not the guy who wrote the program, sit him down in front of the computer, ask him to perform a task using your software, and immediately start discovering usability problems. This kind of usability testing is embarrassingly easy to perform, and the majority of development shops (and developers) don’t do it.

The fact that this is true makes it all the more embarrassing that such vast quantities of software are so difficult to use. Take time tracking software, for instance. I have never worked for a company whose time tracking software was a pleasure to use. In all cases, it would have been easier to just enter time in an Excel spreadsheet. (Come to think of it, I believe the first company I worked for actually did track time this way.) I can think of no reason why this should be true, that all time tracking software stinks. There is nothing inherently difficult about writing software to record how much time you’ve worked. But developers, by and large, do not put a premium on software that’s eminently usable, so long as it can be made to work. “Does the software allow users to enter time and generate a report for managers? Great. Meets the requirements. Done.” In fairness, the blame does not land solely at developers’ feet; often companies are unwilling or unable to pay for the time required to really polish software. But although the law of diminishing returns is definitely in effect here, it does not take a huge investment to make some big gains in software usability.

As a developer, start by forcing yourself to use what you’re writing (and use it in unexpected or unusual ways). Also, put it in front of a disinterested party and make them use it. Watch (or record) – but don’t give them any hints. It will be painful, but it’s pain you need to feel. Take the results, pick the top three worst problems the user ran into, fix them, and do it again. Iterate.

As a user, start being willing to pay for software that’s elegant and easy to use. These attributes are admirable goals in and of themselves; we should be encouraging them to the extent we are able.

The Myth of the Unemployed Programmer

A software developer should never truly be “unemployed”. Many professions require a certain capital investment or infrastructure to perform work–for example, if I am a cabinetmaker, I require woodworking tools, wood, stains, finish, and probably a shop to store it all (plus the works-in-progress). On the other hand, a software developer in the modern world merely requires a computer and Internet connection–all the other tools can be had for free. Even deploying a web application requires very little capital investment in a world where $5/month servers exist.

I have mused upon this concept in the past, but never with the sense of immediacy engendered by the events of the past week. As of last Wednesday, I suddenly became an “unemployed” software developer when I was laid off. This, of course, prompted many non-software-related activities (my résumé needed a rewrite and my online presence–including this blog–was sadly neglected), but after the initial flurry settled down, I was faced with the decision universal to the unemployed: how shall I spend my time?

The options are not infinite (financial concerns for most of us preclude a spontaneous sailing trip to the Bahamas), but they are vast. I could implement one of the many software ideas which perpetually rattle about in a developer’s brain. I could write new themes for my wife’s blog, or her other one, or her other one. I could try to catch up with my wife in website count. I could delve into any number of modern developments in technology, most of which are fascinating and directly applicable to my career. I could write a book, because as my wife observes, once you exclude teen romance and vampires (and vampiric teen romance) there is a certain dearth of quality literature for the juvenile boy. (Though, if I did write a book, I’m not sure you’d ever know about it. On the one hand, I feel like a man ought to write under his own name and take responsibility for it, but on the other hand, I consider it merely common courtesy to preserve his friends and acquaintances from ever feeling obliged read whatever drivel he produces.) I could even finish that basement remodeling project I started eleven months ago, but let’s not get crazy here.

None of these (home improvement excepted) require anything significantly more substantial than time and sterling intellect, both of which I now have in abundance. What a strange thing!

Having dwelt upon the matter at intervals in the preceding days, I have resolved that three things shall happen:

  • This blog shall receive some sorely needed attention. (There is an argument to be made for “quality over quantity”, but 22 months between posts is pushing it.)
  • My WordPress theme development skills shall be enhanced in the furtherance of my wife’s online empire.
  • I shall enhance my professional skills by writing an amazing web application, guaranteed to please. More on that later, but as a hint to those who have known me long, it might improve morale. Or at least keep track of it.

The bottom line is that there is no such thing as true unemployment for the modern creative, and in my case, it is time for some housekeeping. (As an aside, if you’d like to hire me, that’s fine too. I won’t take it personally when you attempt to deprive me of all this glorious free time.)