watchtheRoad - development footnotes
Tuesday, 29. May 2007, 15:44:32
It's always the smallest things that cause the hardest work, isn't it?
I've just released watchtheRoad, a new widget in my touchtheNet series. I had the idea many moons ago, (I think it was during the first weekly widget marathon, where I ended up going with hearthePeople instead), as soon as I'd seen that the BBC were producing XML data feeds for much of their content at http://backstage.bbc.co.uk specifically for mash-up purposes.
Simple enough, I thought. Okay, I've never used Google Maps in Javascript before. Alright, so the BBC data was huge, complex and daunting. It's all just XML and Javascript, right?
The easiest part was ripping the chassis off touchtheSky and refitting it. Ah, the joys of class-based Javascript! It was also suprisingly trouble-free to incorporate a Google Map. Then the troubles began.
1) When I tried to read the XML data from the Beeb, I found that most of the useful attributes were coming back blank. The TPEGML traffic markup format relies on custom XML entities to reference enumerated types - including all the fun stuff like traffic condition, surface conditions, obstruction types and so on. There's dozens of them.
As the users in that thread pointed out, Opera wasn't trying to parse the entities - can't say I blame it, but I needed something that would work regardless of whether the user had that option selected or not. I eventually went with the plan of taking the raw XML file as text, doing some search/replaces ("$" in place of "&", effectively making them non-entities), and then doing the parsing again. This way I got raw strings I could handle internally, at the cost of parsing twice.
2) It became pretty clear I would need to store the user's current location - firstly, I couldn't assume that the centre of the selected traffic region would be the most useful place to start, plus I also wanted to add the feature to filter traffic reports down to a user-defined radius, to target results better.
Simple enough - Google Maps provides a geocoding function, feed it a street and city, it'll do the rest. Except for some reason, I couldn't get any results. It turns out that there's some kind of licensing restriction on Google providing UK location data. No problem, I'll try Yahoo!. Same deal. I did find a few other geocoders that were free and seemed to work, but in most cases they were too specific - I could search for a city, or a specific address, but not the street. The code for these different geocoders still stand in the widget, although unused.
I eventually had a breakthrough in tracking down a method on Google's own site using the Local Search API. By providing the full address as the origin for the search, and then specifying the same address to look for, I could get the origin back as co-ordinates. It's not ideal, but it seems flexible and accurate.
3) I've had to hard-code a lot of style-attributes into the HTML instead of using CSS classes - as a designer, it turns my stomach, but it seems to be the only way to get the widget to animate faster than molasses going uphill in January. On crutches. Anyone got any more tips on speeding up redraws?
4) I had originally hoped to get data for more countries - unfortunately it appears that no-one else has adopted tpegML for listings yet! Please let me know if you can find any, because I should be able to incorporate them straight away. I am looking at the Yahoo! Traffic feeds to see if I can make them compatible.
5) Picking icons was a headache - I had already decided to go with UK road signs, but what to use for accidents? Should I use the accident symbol for all accidents, or should I use No Entry if the road is blocked? How do I distinguish between roadworks that are slowing traffic, and those that are causing surface problems? Essentially, I had to create a priority scheme for selecting icons based on different criteria, and hoped that I'd picked sensibly!
That was the worst of it, though. I hacked together some new graphics (I love SVG for icon and background work, I can get far more accuracy and tweakability than I could in Photoshop, and I can render them to bitmaps when I'm happy) and posted it in the wee small hours of the morning.
If you want to see how it all ended up, you can download it here!
I've just released watchtheRoad, a new widget in my touchtheNet series. I had the idea many moons ago, (I think it was during the first weekly widget marathon, where I ended up going with hearthePeople instead), as soon as I'd seen that the BBC were producing XML data feeds for much of their content at http://backstage.bbc.co.uk specifically for mash-up purposes.
Simple enough, I thought. Okay, I've never used Google Maps in Javascript before. Alright, so the BBC data was huge, complex and daunting. It's all just XML and Javascript, right?
The easiest part was ripping the chassis off touchtheSky and refitting it. Ah, the joys of class-based Javascript! It was also suprisingly trouble-free to incorporate a Google Map. Then the troubles began.
1) When I tried to read the XML data from the Beeb, I found that most of the useful attributes were coming back blank. The TPEGML traffic markup format relies on custom XML entities to reference enumerated types - including all the fun stuff like traffic condition, surface conditions, obstruction types and so on. There's dozens of them.
As the users in that thread pointed out, Opera wasn't trying to parse the entities - can't say I blame it, but I needed something that would work regardless of whether the user had that option selected or not. I eventually went with the plan of taking the raw XML file as text, doing some search/replaces ("$" in place of "&", effectively making them non-entities), and then doing the parsing again. This way I got raw strings I could handle internally, at the cost of parsing twice.
2) It became pretty clear I would need to store the user's current location - firstly, I couldn't assume that the centre of the selected traffic region would be the most useful place to start, plus I also wanted to add the feature to filter traffic reports down to a user-defined radius, to target results better.
Simple enough - Google Maps provides a geocoding function, feed it a street and city, it'll do the rest. Except for some reason, I couldn't get any results. It turns out that there's some kind of licensing restriction on Google providing UK location data. No problem, I'll try Yahoo!. Same deal. I did find a few other geocoders that were free and seemed to work, but in most cases they were too specific - I could search for a city, or a specific address, but not the street. The code for these different geocoders still stand in the widget, although unused.
I eventually had a breakthrough in tracking down a method on Google's own site using the Local Search API. By providing the full address as the origin for the search, and then specifying the same address to look for, I could get the origin back as co-ordinates. It's not ideal, but it seems flexible and accurate.
3) I've had to hard-code a lot of style-attributes into the HTML instead of using CSS classes - as a designer, it turns my stomach, but it seems to be the only way to get the widget to animate faster than molasses going uphill in January. On crutches. Anyone got any more tips on speeding up redraws?
4) I had originally hoped to get data for more countries - unfortunately it appears that no-one else has adopted tpegML for listings yet! Please let me know if you can find any, because I should be able to incorporate them straight away. I am looking at the Yahoo! Traffic feeds to see if I can make them compatible.
5) Picking icons was a headache - I had already decided to go with UK road signs, but what to use for accidents? Should I use the accident symbol for all accidents, or should I use No Entry if the road is blocked? How do I distinguish between roadworks that are slowing traffic, and those that are causing surface problems? Essentially, I had to create a priority scheme for selecting icons based on different criteria, and hoped that I'd picked sensibly!
That was the worst of it, though. I hacked together some new graphics (I love SVG for icon and background work, I can get far more accuracy and tweakability than I could in Photoshop, and I can render them to bitmaps when I'm happy) and posted it in the wee small hours of the morning.
If you want to see how it all ended up, you can download it here!
You must be logged in to write a comment. if you're not a registered member, please 
(0%)
For the entities parsing, wouldn't it be faster to XHR both the XML and the external entity as plain text and then insert the external entity into the xml file so it becomes part of the internal subset, that way, you only need one find and replace and one parse operation?
By JeroenH, # 30. May 2007, 10:01:19
As for the entities, I don't want them to be parsed at all, I'd like them to stay ampersanded. The reason for this is that I can look up the string in a JS hashtable which I've set up to give me more dynamic details than a direct string-replacement can do.
I know it's a little sluggish on the animation parts, I'm looking to hardcode more style parameters to speed up the redraws. I'll also be re-introducing touchtheSky's 'instant animation' setting.
By johnnysaucepn, # 30. May 2007, 10:49:51