Matthew Bull

IWMW 2008

Off to Aberdeen today! Hopefully this year’s conference bag will be even better than last year’s…

symfony 1.1 and 1.2

symfony 1.1 went live just a month or so ago, and in the relatively short history of symfony is somewhat of a revolution. In fact, the shift in a minor version number belies the effort and heartache that seems to have gone into the newer version, and one wonders at the implied major revamp that symfony 2.0 would someday involve. It saw the departure of a key member of the core team, as well as some major changes to the architecture of the framework as outlined in the diagram below.

The great thing about the platform is that being decoupled the classes can be used in isolation, so you can use some features of symfony and not have to use the whole system. In other words, you can build your own custom framework which uses just the bits of symfony that you want.

All this is great stuff, and should help symfony establish a place as the leading PHP MVC framework.

symfony 1.2 is released

But no sooner had 1.1 gone live than we heard that 1.2 is not just in development, but is actually imminent. The current release date is October, which is not very far off. According to the symfony blog, the main (and absolute must-have) feature of 1.2 is a completely revised admin generator system. Other features include bundling Doctrine with the framework and supporting it fully in the same way that Propel currently is. I’m sure a lot of people like using Doctrine, but for me Propel is just fine, and the differences in performance (at least where Propel 1.3 is concerned) are moot (see this blog for a good comparison) 

Worth upgrading to 1.1?

So should I wait for 1.2? I have a couple of live apps running under 1.0. They work just fine as they are, and I see no reason to update those to 1.1 just yet, only to go through the same process again in October. I think I’ll start using 1.1 in a dev environment for getting used to the new architecture, new command line system, etc. Upgrading to Propel 1.3 is also going to be a good idea for the performance gains.

I’m left wondering when symfony 1.3 will be out. Christmas?! Actually, my guess is that now that some of the major architectural changes have been made, things should start to settle down a little. Anyway, it’s great to see such activity in the symfony core development team: evidence that symfony is not just alive and well, but maturing very fast.

symfony’s sfGuard plugin and LDAP

A while ago I needed to hook up symfony’s excellent sfGuard plugin to some LDAP functionality. There are a couple of issues with the plugin and the readme which I think need fixing. In particular, there is no support for checking both LDAP and standard sfGuard passwords. This is absolutely essential (eg an admin user or guest users who aren’t in LDAP), and I’m somewhat amazed that there’s no provision for this. Moreover, the plugin structure generally makes it seemingly impossible (or if it is possible it’s just too horrible to contemplate) to write your own checkPassword() which does do both sorts of checking.

Anyway this post is partly an attempt for me to get my head round something which I’ve either totally misunderstood, or just isn’t well documented.

Confusing README

The README file that comes with the plugin is great, but it’s a little confusing for explaining how you might override its own way of checking passwords with something that’s LDAP-based. It’s really very simple to do when written out in plain English:

1. Make a file called myLDAP.class.php in your application’s lib folder. Put this in it:

class myLDAP extends sfGuardSecurityUser
{
}

2. Now put a method inside the class you just created:
public static function checkLDAPPassword($username, $password)
{
// put your preferred LDAP-checking code in here
// should return true or false
}

3. Add this to your application’s app.yml file:

all:
  sf_guard_plugin:
    check_password_callable: [myLDAP, checkLDAPPassword]

where myLDAP is the file called myLDAP.class.php that you created in step 1 above. checkLDAPPassword is the method you made inside myLDAP.class.php that you made in step 2.

And there you have it. Basic LDAP password checking.

How does checkPassword() actually work?

The above code is all very well, but what’s really going on?

Well, there are a couple of steps which I managed to work out:

1. sfGuardUserValidator contains a call to $user->checkPassword($password) where $user is a result of sfGuardUserPeer::retrieveByUsername($username)

2. PluginsfGuardUser (extends BasesfGuardUser) has a method called checkPassword() too. This calls the method you listed in your app.yml file (checkLDAPPassword in this case), or does a standard sfGuard password check (checkPasswordByGuard) if you didn’t specify anything:

public function checkPassword($password)
{
if ($callable = sfConfig::get('app_sf_guard_plugin_check_password_callable'))
{
return call_user_func_array($callable, array($this->getUsername(), $password));
}
else
{
return $this->checkPasswordByGuard($password);
}
}

Which it does depends on how you set up your app.yml file (see Step 3 above).

At this point, if you set up your app.yml correctly you’ll be in your custom checkLDAPPassword() method, which will do whatever you want it to do to check LDAP passwords. So long as it returns true or false depending on whether the check was successful or not, you’ll be ok.

Note that sfGuardSecurityUser class also contains a checkPassword() method. This basically does the same thing as the in sfGuardUserValidator, but in a slightly different way, and allows access to checkPassword() in templates and controllers through the $sf_user object. The important thing to realise is that both pieces of code end up calling the same checkPassword() method in PluginsfGuardUser.

So how about LDAP and sfGuard checking?

Now the fun begins. The problem is that it’s seemingly impossible to call checkPasswordByGuard() yourself from inside your custom-built checkLDAPPassword() method. Why would you want to do this? Because that way you can do some standard password checking first to see if your user is in fact a sfGuard user and not an LDAP user. Only if that failed would you do LDAP checking (I suppose you could do it the other way round if you wanted.) Even better, you wouldn’t need to alter checkPasswordByGuard().

So why is it impossible to call checkPasswordByGuard() ‘by hand’? As far as I can see, it assumes that you’re not in a static context, and that you have some kind of user object available. But of course, my checkLDAPPassword() is in a static context. It had to be that way because it was called by call_user_func_array() which uses the output of sfConfig::get() as an argument.

Sigh…

If you find a way of doing this, please let me know. Losing the will to live is merely one of the symptoms of battling with this sort of thing. I have a life to live.

All I can suggest to you in the meantime is to modify your copy of PluginsfGuardUser.php to allow both sorts of password checking. That’s what I did and it worked for me. It’s just a pity that you can’t override the checkPasswordByGuard() method in some way without changing the core of the plugin.

Matrix GUI, asset builders (and mums)

After fun and a few frustrations sorting out designs in MySource Matrix in my last post (probably mostly caused by me not knowing what I was doing, and with limited time to sort things out), I’ve been looking at the overall user interface for editors.

The interface is possibly a tad intimidating for some (could you imagine your mum using it?), but the great thing about the system is that it’s flexible enough that you can build your own very simple content edit interfaces.

Lots of buttons and flashing lights…

The thing to remember here is that MySource Matrix is a powerful system, with all kinds of features on all kinds of screens. Tickboxes, drop-downs, options, buttons all over the place. It’s great, but it’s complex. And as we all know, making something easy to use is a real trick. Just think back (if you can) to the days of video recorders that no one knew how to use because they made actually fairly simple things seem very hard to do. It’s the kind of thing companies like Apple have spent decades trying to perfect.

Actually the developers of Matrix have done a reasonable job, and in any case much of the interface is only for system administrators (like me) setting the thing up.

But if all you want to do is make a new page with some text, images, and links in it (this must be like 95% of all web pages) you don’t need or want to know about all the fancy stuff in the backend, all the different types of assets, etc, etc. You probably don’t even want to know what an asset is. You just want a nice box you can type stuff into, press a button, and away you go. You want something like wordpress with all its web 2.0 goodness.

Asset builders and simple edit

Thankfully Matrix has two features which you can get to work together to build a simple editing interface the way you want it.

The simple edit interface is basically a way of allowing users to edit a page without all that backend admin stuff. On its own it’s a nice feature, but not much use if you actually want to make a new page. Asset builders are really the other side of the coin, because they give you the flexibility to allow any user to add in whatever new assets you want them to add.

So, make a new page with asset builder. Edit it with simple edit.

I have to say that I haven’t tried getting them to work together yet. I don’t even know to what extent that’s possible. What would be nice is to be able to make a new page and edit it at the same time, then either publish it or save it for later if I don’t want it to go live. That’s what I’m doing now in wordpress. That’s what google docs does. It’s intuitive and quick. That said, I’m probably hoping for too much, because bolting something that simple onto what can often be a considerably more complex process (think workflows, nested content, etc) is a real problem.

Summary

Matrix offers a very comprehensive and rich backend interface. If you want something easier for your users you can build your own very simple frontend new page/edit interface, which is a really nice feature. Users still have to go into the admin interface to delete or move those pages, and do other fairly simple operations.

I’d love to see a friendlier backend interface to Matrix, and I have no idea if that’s planned for Matrix 4 (due out sometime next year apparently). Sprucing it up somewhat would certainly turn it into a pretty much unrivalled open source CMS.

MySource Matrix design

I’ve been playing around with mysource matrix on my laptop recently, as an early development stage to our pilot while we’re waiting for the server hardware to be set up.

One of the first things I’ve been looking at has been setting up designs, and I have to admit it’s been a somewhat mysterious process.

OK so I haven’t been on Squiz’s design training course, but I have taken a long hard look at the 94-page design manual (to their credit, Squiz have about 1300 pages of documentation, most of which is freely available.)

Part of the problem is our pretty complex set of css files. The other part of the problem seems to be the way mysource matrix deals with links to images in css files. Suffice to say that getting an existing design into mysource matrix isn’t much fun. Once you’ve got over that hurdle, performance seems to be an issue.

A far easier option is just to lift your static template, pointing to all those static css files and images sitting on your live webserver. In fact this works just great, and it’s what I’m now doing. But something makes me feel it’s not the ‘right’ way to do things. I should point out that I may have misunderstood something in how to set things up, so this review may not be entirely fair. Apologies in advance it that’s the case.

If you enjoy pain, and want to share some of mine, read on…

Getting Started

Let’s say you have an image that you want to call from a css file with something like

background-image:url(images/snap.gif)

How does matrix know where images/snap.gif lives? Unless it’s living somewhere on a publicly accessible web server, it has to be uploaded into matrix. You can then refer to it as mysource_files/snap.gif, and the system will know what you mean.

If this sounds simple enough, then wait. There’s much more. By the end of this you’ll think having teeth pulled is a fun experience. Read on…

CSS Files

You first have to tell matrix that each css file which has a url() is an asset, not just some static text file sitting there. OK, so I go through all 30 of my css files that are like this, creating them as children to my main design. Something is making me feel uneasy about this, but I carry on anyway.

Images

Right, now I have to make all the images that are referred to in each css file children of the corresponding css file. Ugh. Yep, well I don’t have anything better to do, so I reorganise all my images according to the css file they appear in. Some images in more than one place, but oh well I guess I could make them as copies if only I could remember which css file I already uploaded them to. Whatever. I battle on.

The process continues for some time…

Cool. It was painful, but now I have all my css files uploaded, with all their corresponding images. I’ve changed the css files to refer to mysource_files/imagename.gif rather than their original url.

Now I need to link these files into the main design. It turns out that I can’t actually do this directly. I have to create special design areas in the main design which point to the css files.

OK, not too bad? Erm, well it’s not that simple. I actually also need to put in what’s called a customization which acts as a bridge between design and css files, and it’s that thing which point to the css assets themselves. This feels very wrong, but I carry on.

Too slooowww…

Hoorah! Now after a few hours persistence I have a working design. I test it out. After about 8 seconds, my page loads with the correct design. Hoorah again!

Then I stop and think what’s just happened. 8 seconds? Hmmm, ok well I know this is just on my macbook, but that’s really a tad slow isn’t it?

At that point I realised that there’s a lot going on here. You can cache the css files, although that threw an error when I tried it. I eventually fixed that ‘feature’ temporarily, and it’s true things did start to speed up a lot.

But at that point I pretty much gave up. Life’s too short. The option to point all urls at the currently live designs was just too alluring. Anyway it was all a useful experience, if only to tell me that getting a design to work properly and load quickly in MySource Matrix isn’t a quick or simple procedure.

mini symfony

I recently read this post by Francois Zaninotto (recently an ex-core symfony developer) about modifying symfony to make it lightweight enough for ajax calls. As he puts it:

That’s when the idea of a “small symfony” comes. Wouldn’t it be great if you could get access to the model layer, the configuration, the autoloading, the user object, the helpers, and keep a MVC separation, without initializing the whole framework?

As symfony stands, it’s sometimes just a tad too bloated for the kinds of really fast responses needed to make ajax applications work. Francois came up with this solution, which is to cut out the call to sfController::dispatch()

But you still get enough to make use of things like the basic MVC design, autoloading, and helpers. I still haven’t tried it myself, but it seems like a really nice idea. Just a shame he’s no longer in the core team…

XCRI project

XCRI (Exchanging Course Related Information) is a JISC-funded project to get HE institutions to produce course data in a universally exchangeable format, so it can easily be accessed and used by other institutions, end-users, and aggregators. It’s a bit like an RSS feed for courses.

Kent University have a JISC-funded mini-project to produce an XCRI feed, and I’m running the project. I’m adding entries to my personal blog because at the moment we don’t have Kent blogs, so this will have to do for now.

One of the main problems we have at Kent - a problem shared by many other institutions seemingly - is that although we obviously have a reliable and structured source of course data, we don’t have a structured source of public-friendly information. The sort of stuff that goes into a prospectus for potential students. But XCRI is attempting to gather exactly that sort of information: the stuff you want to advertise to the outside world.

This isn’t just a data-mining exercise. Sure, the first practical hurdle is picking out the relevant bits from html and flat text held in our prospectuses and webpages, and putting all this into a structured format such as XML or a relational database. But the bigger problem is changing the way people work. I won’t go into the details of how the publications are generated at the moment, but suffice to say it’s largely paper- and Quark-driven. The webpages have to be manually created after this process, and there’s no source of data other than flat documents.

So the biggest hurdle is going to be setting up a new system where the data comes first, and from this web and print can be generated. This in turn means collaborative, workflowed systems (a CMS?!), pushing out XML. The XML can then be turned into whatever you want.

Well, that’s the theory. I’ll have to see what happens over the next couple of months…

Zaninotto

Looks like Francois Zaninotto, one of the core symfony developers, has quit the team. He was largely responsible for the documentation, and co-authored the excellent symfony book, published by Apress.

One of the key successes behind symfony was the documentation. The fact that there was human-readable documentation, and not just an API, was a remarkable thing for an open source project. It’s surely to the detriment of the project that he’s no longer a part of it.

Hopefully he’ll continue to play an active role in the community, and might even consider writing documentation independently of the core team. We’ll just have to see what direction symfony head off in over the next couple of years.

The Squiz Licence

I’ve recently been looking more closely at MySource Matrix, an open source PHP-based CMS. It’s very important to realise that ‘open source’ is not necessarily what you might think in the real world of running an enterprise CMS.

Squiz

So what about the details of using MySource Matrix in anger? First off, the software is essentially managed by one company, Squiz, based in Australia. It was (as far as I can see) very much developed by them, and while additions and improvements may have been added by people in a user-community over the years, what you get when you download the system is essentially a Squiz product.

Dual licence

Then there’s the dual licence. This basically means that you can download the software, install it on your laptop or server or whatever (as I did) and start playing around with it. This comes under a GPL licence, and you’re free to do whatever you want with it. Squiz have even kindly released hundreds of pages of documentation on how to use the system.

But you’d have to be a brave soul to do this for anything other than your department’s website, etc. Imagine your university or large corporation wanted a CMS, and you offer them something for which you have no training, and only a superficial idea of the mechanisms behind it because you can only spend 50% of your time on it? Would you really want to be to blame for that disaster-in-waiting?

And I think this is exactly what Squiz have realised, and exactly why they have a dual licence. That is, if you want something a bit shinier, more bells and whistles, proper training, proper support… well, really you need their SSV (Squiz Supported Version) licence. This is where things seem to change somewhat. 

The SSV bit

Once you enter SSV territory, you seem to agree that in essence everything you have is owned by Squiz. Moreover, you must tell Squiz about any changes you make to the code, and those changes aren’t yours. They belong to Squiz. This may scare some, and horrify true blue open source devotees. I should point out that it appears that, according to this page, that you can actually opt to limit your obligation to Squiz (for a fee). You can even opt to resell your own changes commercially, although I’d imagine this option demands an even bigger fee.

In any case, I wonder, is it really so terrible not to be able to modify the code, and still retain ownership of those changes? Would you really want to make any changes to a complex and well-supported system? About a year ago I had to tinker with phpBB to make it work in the way we wanted. Great. But now we have a system which would be hard to update if there were any new security patches. I also remember going to a Plone talk once, at which some developers proudly demonstrated how brilliant Plone could be. The only caveat was that it had taken them years of expert development to get to that point, and their version of Plone was so wildly different from the out-the-box version that updates were a significant problem.

It’s free

As far as I’m concerned, Squiz’s dual licence is sort of open source. It isn’t true open source, but really I don’t care. The main thing is that it’s much much cheaper to get hold of than other CMS vendors (ie no up front licence fee of £80k), and you can see all the code and work out what’s going on.

OK, changing that code and retaining ownership of the changes is another thing, but maybe you really need to be looking at your motives for getting a CMS. If it’s to get: a system which works, which users like, is easy to maintain, has good support, and doesn’t cost too much… well then something like this Squiz’s licence and MySource Matrix might seem like a good idea.

But if you want to spend the next few years tinkering and playing and developing your own modules, and at the end of this process you want to claim the changes as your own… well, maybe this licence scheme isn’t for you.

Summary

Squiz have developed what looks like a good CMS. They provide the software for free, but make their money out of support, design, and training. They also develop a number of extra modules, which they sell to non-educational and non-governmental clients.

But just because the software is free, it doesn’t necessarily mean it’s truly open source. Their SSV licence (essential if you want to use the system to its full potential and do serious stuff with it) means Squiz end up owning any changes you might make. I’m no lawyer, but it even seems like any modules you might develop end up being theirs too.

I don’t think this is necessarily a bad thing, at least if you feel it’s unlikely that you’d do a massive amount of development yourself. And it’s important to know that any changes you do make end up being incorporated into the product, so other Matrix users benefit. I also get the impression that many new bits of code are developed by users with collaboration from Squiz. So overall it seems like a collaborative situation where everyone gains from the development of a new requirement: Squiz make more money, and users end up with a better CMS. Is this bad? I don’t think so.

symfony admin generator

The symfony admin generator is one of the best features of the framework. With just a couple of commands and a few changes to a config file, you can have something pretty close to a database access interface. There’s a simple built in security module, and sfGuard is fairly simple to understand if you need a more complex security module.

At first it seems like the admin code is pretty much pre-baked, and there’s not much you can do to change the behaviour of the interface, save change the config files. But one of the most useful things to know about the symfony admin generator is that you can do extra things. It’s not well documented, but sticking this method:

protected function updateModuleNameFromRequest()

in your actions.class.php file allows you to add in extra code each time anything is done in that module (substitute ModuleName for your own module).

For example, if you need to clear the cache each time something is done, put it here. If you need to update a set of data in a one-to-many relationship with a class, put it here.

It seems curious that more is not made of this function in the book and documentation, because it’s so useful.

And don’t forget you can also use the standard executeAction() methods, preExecute(), postExecute(), etc. in your module’s action.class.php file, in exactly the way a frontend action would work. This means you can add in any kind of custom action to the admin section of your application.