Author Archives | Roger

Our shop is now open!

Our shop is now open!

Yes, it’s true. We might be an innovation consultancy but we have in fact just opened our first shop over at print on demand specialists Cafe Press to showcase our high resolution Wordles generated automatically from the complete works of William Shakespeare!

Visit our shop

Visit our shop

“Wordle” what’s that all about then?

You may have already come across wordles on the web when they appeared in the press earlier in the year. A wordle is a graphic layout of text where the size of a word is in proportion to its frequency in a document.

Here’s an example;

The Merry Wives of Windsor in 100 words

The Merry Wives of Windsor in 100 words

How did you do it?

The credit belongs to Jonathan Feinberg who wrote the original java application for generating a wordle. Although it looks simple, there is some complex mathematics going on under the hood and whilst I’d love to say we wrote it, we didn’t. What we did do was read the public domain text of the complete words of Shakespeare, parse the data to drop stage directions and so forth and then feed the text into a wordle generator written by Jonathan (see, told you he was the key to it all), kindly licensed by IBM and available on their alphaworks site here. We then generated 60 wordles of each play to pick the layouts we liked most and then built a shop so we could share them with you.

As a side note it’s worth mentioning that the images we use for the shop are special high resolution files generated at print resolutions and not simply web images taken from the wordle website.

What about the images, can I use them?

Absolutely, you are free to use the images under the same terms that wordle.net licenses them to us. Please read the FAQ on usage here.

Enjoy

Posted in Products0 Comments

Why you need to de-fluff your Javascript

I don’t know if it’s the same for you, but in recent times I’m finding that the number of javascript files my projects rely on is growing arms and legs. It’s not uncommon to discover that you’re soon using maybe a dozen jQuery components, the excellent shadowbox plugin (complete with multiple javascript files) and more often than not, some mapping (cue the very handy geoxml project). That’s before we even start with the code we will be writing. Things got so bad on a recent project (23 – yes TWENTY THREE included javascript files) that I decided it really was time to get a grip on things.

The mission was on to a) cut the bandwidth taken up in delivering these files to the users’ browsers b) reduce the number of HTTP requests and thereby improve page loading times and c) potentially help out with SEO by getting content closer to the top of the page.  That last one is a bit of a reach, but it never does any harm for a page to have a higher proportion of real content nearer the top.

The strategy was;

1) Ensure Javascript was being served compressed (g-zip/deflate)

2) Minify / Pack / otherwise compress the source javascript

3) Suture the compressed javascript files together into one large file

1) The starting point was to ensure that Apache was correctly serving javascript in compressed form to compatible browsers.  This can bring dramatic benefits and research shows an average 75% SAVING in bandwidth for html/css and javascript when delivered in compressed form. It’s easy, it’s an instant fix and you really ought to be doing it. Sadly some hosting companies out there still won’t turn on compression for you and use some lame excuse or another – find a better host I say!

2) The next step was to reduce the source file sizes before they are served using various compression techniques. You have probably come across this before – either as minified jQuery files, or perhaps as javascript that has been packed using the dean edwards tool, but because I wanted to script the conversion on my own server, I chose the excellent Javascript::Packer CPAN module by Merten Falk. The nice thing about this module is that I can choose from different sorts of compression – minify and pack being but two of them.

OK so by now we’re making improvements, but not actually addressing the main problem. Our files are much shorter (90% shorter in some cases) having been minified. They are reaching the user faster that before as we have tuned our webserver to deliver the javascript in compresed form, but we still have a huge number of files being downloaded and that means additional HTTP requests and THAT is the real performance killer. Remember, between 40% and 60% of visitors to your site are likely to have an empty cache, so this is a very important issue – in fact it is the most important issue affecting page performance.

So onto;

3) Suture the files together into 1.

And this is where all my problems began.

The site stopped working.  Or at least various bits of it did. I could often control which bits worked by swapping the order of the stitched files and then I twigged that perhaps the minimising or packing was failing in some way and that when the files were all together in one file, this was being highlighted.

And this is the crux of the matter and the point of my post. NOT ALL FILES WILL SURVIVE being minified or packed. The process of removing white space which is inevitably part of the compression process can semantically change the meaning of a poorly written javascript and result in it breaking, despite it running as desired on the target browser.

So what to do … ?

This is what led me to discovering the joys of JSLint, a tool that you can run over your javascript to give it a clean bill of health – or otherwise. Indeed it’s the “or otherwise” most of the time to be honest as the default options are rather strict, but the great thing about JSLint is that it will tell you exactly what coding rules are being broken AND (usually) how to fix them.

After working through all the files and correcting all the faults as reported by JSLint (barring some stylistic ones I chose to ignore), the files are now all minified / packed and correctly sutured together into one large file. The site works as intended and page loading times have been plunged by almost 70% for new visitors on a decent connection.

Conclusion

JSLint might seem a bit pointless at first, all it does is nag you about lots of seemingly petty coding rules, but I would say BEAR WITH IT, learn why it is making the suggestions it is making and if needs be, turn off some of the stricter rules if you can justify why. Your patience will be rewarded as your files will now survive minifying and packing and you will be able to stitch them all together into one large file to dramatically improve your website performance.

Roger

Posted in Development, Javascript0 Comments

How Amazon’s cloud saved the Golden Hour

How Amazon’s cloud saved the Golden Hour

Roger created and runs an educational site about the science of the so-called “golden hour” – the time around sunrise and sunset that has the most fantastic light for taking photographs. Although there were lots of websites giving tables of sunrise and sunset times, I wanted to create something more visual that showed the golden-hour visually and allowed you to explore how it changes with latitude and time of year.

It was an interesting project as it ties together various libraries such as the poly9 3D flash globe and openlayers mapping alongside astronomical algorithms and geolocation work. I was quite pleased with the site, put it online and then went back to work …

The Golden Hour Website

The Golden Hour Website

For a few months, there wasn’t really much interest in the site until all of a sudden in July, my traffic spiked quite significantly.

July Traffic Spike

July Traffic Spike

Nothing too dramatic, a peak of 2,000 page views per day, so in the real world not really a “peak” at all, however I was intrigued to discover that the traffic came from a mention in a Canon newsletter. With enough traffic now to make it worthwhile, I spent some time adding in some advertising to the site, some Amazon widgets to sell Canon camera gear, and again, just left it and went back to work, this time pleased that other people were enjoying the site.

My headache however arrived a month later when all of a sudden I received a terms of service violation notice from my hosting company with a note that I had exceeded their CPU allowance and had disabled the site.  Nice!

It became clear that something had triggered a much larger traffic spike and to get back online I needed to act fast.

Amazon EC2 to the rescue!

It just so happens that my work at Mindsizzlers has been exploring and evaluating cloud computing and I had been building applications in Amazon’s compute cloud for several months. This meant I had an account ready to go and was able to spin up a suitable server instance, upload the appropriate cgi scripts other resources that the golden hour required and have an alternative server up and running, literally within the space of 15 minutes or so.

My other good fortune was that I’m in the habit of using short DNS time-to-live values on many sites through DNS made easy (daft name, great service). This meant I could change the A-records for the golden hour and get the traffic flowing across pretty quickly. In the meantime I was able to put a redirect in at the old host to bounce the traffic into the cloud until things calmed down.

Incidentally, this was the traffic spike. It’s the first one in the graphic.

First spike took the site down, the second I was prepared for.

First spike took the site down, the second I was prepared for.

The spike was due to a mention on www.lifehacker.com and generated an intense burst of visitors that I had absolutely no warning about.

Without Amazon’s computing cloud, I’d have been sunk. My hosting company were fast to turn the site off, but then unacceptably slow at responding to any follow up. It took over 4 hours to restore even limited access and in fact it turned out that I had a widget that wasn’t delivering cached output which was the real cause of the problem in the first place.

With Amazon and EC2, I was able to get a new server on-line in minutes and the best part is I only needed to pay by the hour I was using it. Actually it’s even better than that. One USP about the Amazon cloud is that I’m not tied into having a particular operating system ( think Azure ) or writing my application in a particular language ( think Google Appengine ). This really is a killer feature of Amazon’s cloud, I can pick from hundreds (if not thousands) of machine images or build my own from a basic starting point, but at all times I am in control, I can pick the operating system, how much memory I need, what sort of CPU power I want – it really is the perfect instant on-line server shop and what makes it even more amazing is that it comes from a company that started life as a book seller.

Anyway, as things calmed down, what I did was have round robin DNS so that half the visitors are served by my original hosting company, and half are served by a cloud instance that I use for other projects too.  This was very handy a few weeks later because the more observant of you will have noticed a second traffic spike on the chart – this time it came from being voted “cool site of the day” by America’s Digital Goddess  (no, really, I’m not making this up). Two great things about that. Firstly they were kind enough to warn me a day or so in advance – thanks guys, and secondly with the round robin DNS in place, and the cloud humming away nicely, the servers barely broke into a sweat.

Posted in Cloud Computing, Featured, Personal0 Comments

Mindsizzlers launch first iPhone application!

Mindsizzlers launch first iPhone application!

Following on from our news about the launch of www.dodadog.com, we’re delighted to announce the release of our first iPhone application which complements the site by tracking users’ walks and giving them access to a special mobile web version of the website.

Written entirely in native objective-c for the iPhone, this is the first of a range of iPhone applications that Mindsizzlers has planned for the coming months, but if you have a project you would like to see realised on the iPhone, please don’t hesitate to get in touch as this is very much a space we are now confident working in.

Launch Screen

Launch Screen

Posted in Development, Featured, iPhone Development, Products0 Comments

Got a dog? Join DODADOG!

Got a dog? Join DODADOG!

We’re delighted to announce the release of www.dodadog.com which is our new gregarious network for dogs and their owners. The site has discussion groups, breed information, photo galleries, news and articles and uniquely, the ability to track your walks using our related iPhone application.

DODADOG was the brainchild of TV producer / director Colin Brewer and has been developed by Mindsizzlers who are joint partners in the new venture.

Built entirely on Amazon’s compute cloud and utilising Amazon’s SimpleDB infrastructure as opposed to a conventional relational database, DODADOG is at the cutting edge of new web technologies and we wish it every success.

www.dodadog.com

www.dodadog.com

Posted in Featured, News, Products0 Comments

Memcached and Clouds

Memcached and Clouds

I’ve been spending a lot of time with Amazon’s cloud database SimpleDB recently and was invited to give a presentation in July on how Memcached and SimpleDB work really well together. We’ve made a copy of the presentation available online and you can view it below. If you want a PDF copy of the presentation, you can order it for free on the same page.

www.mindsizzlers.com/presentations/amazon_simpledb/

Posted in Cloud Computing, Development, Featured, News, Presentations0 Comments

Announcing Web Trends Now

Announcing Web Trends Now

We’ve been working on a little research tool that helps keep you in the loop of what’s going on in cyberspace and are pleased to launch www.webtrendsnow.com

The site works like this.

Ever hour, Google trends publish data about the hot search terms as determined by analysis of their US based users.

We then run a search against each of the top 100 search terms to find the top sites that web users will be visiting as a result.

We show you this information, together with a thumbnail of the website (made by another mindsizzling service we will soon be launching to the public) which makes it very quick and easy to see if the site is something you are interested in.

If you find it useful, do let us know …

Posted in Featured, News, Products0 Comments

Boost performance with LibXML

We’re working on a project at the moment that has a lot of XML flying about, for example we wrap data coming out of Amazon SimpleDB in XML and then consume that data in the rest of the program.

I’ve been using XML::XPath to extract the data from the xml, so I can write this sort of thing;

my $xp = XML::XPath->new( xml => $xml );
foreach my $walk ($xp->findnodes('/walks/walk'))
{
my $walkid = $walk->findvalue('./@itemname');
etc ...
}


It’s easy to write, easy to read and works well. However recently I’ve begun noticing that the project has become a bit, well, sluggish. I was kind of hoping that XPath would be using the C (and hence very fast) LibXML under the hood since I had recently installed that parser on the system, however the lack of speed led me to think this might not be the case.

Reading around, I discovered that there already is XPath support built in to LibXML and so I was able to rewrite my code as follows;
my $parser = XML::LibXML->new();
my $doc = $parser->parse_string($xml);
my $xp = XML::LibXML::XPathContext->new($doc->documentElement());

foreach my $walk ($xp->findnodes('/walks/walk'))
{
my $walkid = $walk->findvalue('./@itemname');
etc ...
}


Note how it is just the setup that has changed, the actual data processing stays the same (in most cases).

This makes things *MUCH* speedier as you would expect. My perception is perhaps as much as 10 times faster for large XML files, but I haven’t done any quantitative analysis.

BEWARE though, it’s not a completely transparent drop-in as the parser in LibXML has some quirks. For example if there is a namespace declared in the xml file, then you will get no data returned unless you correctly attach this to the context.

For example, when writing an Atom parser, note the registerNs line

$PARSER = XML::LibXML->new();
$DOC = $PARSER->parse_string($xml);
$XP = XML::LibXML::XPathContext->new($DOC->documentElement());
$XP->registerNs( xatom => "http://www.w3.org/2005/Atom" );

foreach my $data ($XP->findnodes('//xatom:entry/xatom:content[@type="text/xml"]'))

This despite the fact that inside the atom feed, NO namespace is explicitly used in elements. The atom file contains <entry> and NOT <xatom:entry> but you MUST attach a namespace to be able to read the data. You could choose any namespace, I picked xatom but it could just as well have been fred. Go figure …

Posted in Development0 Comments

KML Parsing

We’re working on a consumer project at the moment that makes heavy use of mapping technologies. We built the project with google maps, but ran into a deal breaker where placemark links in parsed KML always open in a new browser window, rather than the same window that is sometimes preferable.

Google seem to have acknowledged this is an issue, but they acknowledged it 2 years ago and have done nothing since – presumably for their own good reason – however for someone developing tools against the API, it can be a real headache.

Enter the open source community and a great product by Lance Dyas called GeoXMLwhich is a new parser you can use for KML files (and much more) within the Google Maps API. And the BEST feature … well for us, it DOES allow you to open links in placemark info windows in the SAME browser window. Great!

Beware that sadly the project lacks any real documentation and getting started can be rather hard. The documentation for another project EGeoXML is a useful starting point.

Posted in Development0 Comments

SimpleDB backup solution launched

SimpleDB backup solution launched

We’re a fan of Amazon cloud computing and have been using SimpleDB within some projects that we will be launching soon. The problem though is that there is no backup solution for the service and whilst Amazon’s redundancy etc means that we should never lose any data anyway, that doesn’t protect you against the accidental deletion of data (been there, done that) or the desire to have data in files that you can import into other databases (eg Mysql)etc.

I was a bit surprised that this hadn’t been addressed by the Amazon community at large so I took a few days (well, a week) out of my schedule to pull togetherwww.backupsdb.com – the world’s first online backup solution for Amazon SimpleDB.

We launched it last week and you can read more over at the backupsdb blog

Posted in Featured, News, Products0 Comments

Advert

For more information about our services…

Contact Us

Wordle of the Day

Image from software at http://wordle.net
Data by Web Trends Now

Categories