My OperaThe New My Opera (Codename: Dexter) 10 days later...
By Martin Myrsethubermyme. Friday, November 26, 2010 11:12:29 AM
So now that most of you are at least getting familiar with the new chrome of My Opera, it's time to probably mention a few of the things that went on behind the scenes for this release. Jacob has already written a nice post about our new grid system, check it out. Fredrik also wrote this post quite a while ago, indicating what we were planning to achieve and do with the new My Opera.
NB: This is not a changelog, but there will hopefully be one put out a later date.
To start off with, I don't only speak for myself when I say that this was the biggest side-branch of a project I've worked on. The final diff from the old trunk/main codebase ended up close to 1 million (that's ONE MILLION) lines, which is not including binary content and added files. About half a year of development went in to this release, while we were simultaneously maintaining the live site. There has been several obstacles that has occasionally slowed progress, not to mention we were also under attack, but what doesn't kill you and so on ..
Several motivational factors were behind the decision to begin such a large project as Dexter. Probably the main one, was a wish within the team to rid ourselves of our current templating system (XSLT) and switch to something more manageable. From the alternatives we've investigated, Template Toolkit (TT) seemed the most reasonable for our purpose. One of the consequences of switching the templating system of My Opera, is of course that the front-end of the site would have to be ported/rewritten to TT, which coincidentally coincided with design changes to www.opera.com. This led to a goal of trying to unify the design of Opera's online services, to provide a uniform look and feel to our sites.
In order to support TT, we had several possible solutions where one alternative was to modify our current in-house "MVC" "framework", hopefully reducing the amount of infrastructure required to be changed. Another, more comprehensive alternative, was to start incorporating a third party MVC framework into our existing system. The most reasonable candidate for that was Catalyst. After several weeks of research and discussions, the decision fell on Catalyst, since we found a transparent way to deploy it together with our current system. This allowed us to convert parts of the site step-by-step into TT, rather than having to change everything in one bulk operation. Another obvious reason to pick Catalyst, is that as we progress converting pages into the Catalyst/TT combination, we're able to reap many of the benefits of an actively developed framework. These include standard, generic modules and plugins for common tasks, a more lose coupling with the web-server and potentially the databse, and not to mention it's well tested. The first working pages running under Catalyst, are the community photos pages http://my.opera.com/community/photos, and we are aiming at converting pages at a blazing speed.
Our business logic is of very varied age and quality, so it's definitely the first few pages that are converted that will take the most time. Many of our back-end modules are currently only able of providing data output as XML, which is quite useless when using TT. So the first step when converting a page into TT, is looking at the required modules for displaying that page and eventually making sure they're able to output data in a native perl data structure. The modules still have to work for our old system, so that means that backwards compatibility must be ensured. Also, to make sure that our front-end developers are not halted by having to wait for the appropriate data from the back-end, we define what data should be output from a module and create TT data files with example contents. This way, a page can take form as a static example, while we're busy updating the back-end modules. Since many of these modules are shared for pages over the entire site, and many are used in almost every page, once a module has been changed, updating a page using that module should be significantly quicker.
So the most major change with the newest iteration of My Opera, is the introduction of Catalyst and Template Toolkit. They are very welcome into the My Opera family. Besides that, there is soooo much candy related to this release I get watery eyes:
I know there have been many issues these first 10 days after the release, but we have been able to continuously and transparently ship fixes to the most severe cases and also many smaller ones. We try our best at reading your feedback and creating issues for whatever is reported. Of course, not every issue is given the highest priority, so it has to wait for higher priority tasks to be completed. Please be patient as we do our best to crush whatever nasty bugs there might be. And most importantly, have faith, for My Opera is moving in only one direction: Up-up-up!
NB: This is not a changelog, but there will hopefully be one put out a later date.
To start off with, I don't only speak for myself when I say that this was the biggest side-branch of a project I've worked on. The final diff from the old trunk/main codebase ended up close to 1 million (that's ONE MILLION) lines, which is not including binary content and added files. About half a year of development went in to this release, while we were simultaneously maintaining the live site. There has been several obstacles that has occasionally slowed progress, not to mention we were also under attack, but what doesn't kill you and so on ..
Several motivational factors were behind the decision to begin such a large project as Dexter. Probably the main one, was a wish within the team to rid ourselves of our current templating system (XSLT) and switch to something more manageable. From the alternatives we've investigated, Template Toolkit (TT) seemed the most reasonable for our purpose. One of the consequences of switching the templating system of My Opera, is of course that the front-end of the site would have to be ported/rewritten to TT, which coincidentally coincided with design changes to www.opera.com. This led to a goal of trying to unify the design of Opera's online services, to provide a uniform look and feel to our sites.
In order to support TT, we had several possible solutions where one alternative was to modify our current in-house "MVC" "framework", hopefully reducing the amount of infrastructure required to be changed. Another, more comprehensive alternative, was to start incorporating a third party MVC framework into our existing system. The most reasonable candidate for that was Catalyst. After several weeks of research and discussions, the decision fell on Catalyst, since we found a transparent way to deploy it together with our current system. This allowed us to convert parts of the site step-by-step into TT, rather than having to change everything in one bulk operation. Another obvious reason to pick Catalyst, is that as we progress converting pages into the Catalyst/TT combination, we're able to reap many of the benefits of an actively developed framework. These include standard, generic modules and plugins for common tasks, a more lose coupling with the web-server and potentially the databse, and not to mention it's well tested. The first working pages running under Catalyst, are the community photos pages http://my.opera.com/community/photos, and we are aiming at converting pages at a blazing speed.
Our business logic is of very varied age and quality, so it's definitely the first few pages that are converted that will take the most time. Many of our back-end modules are currently only able of providing data output as XML, which is quite useless when using TT. So the first step when converting a page into TT, is looking at the required modules for displaying that page and eventually making sure they're able to output data in a native perl data structure. The modules still have to work for our old system, so that means that backwards compatibility must be ensured. Also, to make sure that our front-end developers are not halted by having to wait for the appropriate data from the back-end, we define what data should be output from a module and create TT data files with example contents. This way, a page can take form as a static example, while we're busy updating the back-end modules. Since many of these modules are shared for pages over the entire site, and many are used in almost every page, once a module has been changed, updating a page using that module should be significantly quicker.
So the most major change with the newest iteration of My Opera, is the introduction of Catalyst and Template Toolkit. They are very welcome into the My Opera family. Besides that, there is soooo much candy related to this release I get watery eyes:
- We have introduced HTTPS/SSL for a more secure login and also management of your account pages. We will continue to imporve our HTTPS solution, hopefully ending up with an entirely secured My Opera for all logged in users.
- We are converting all pages to HTML5 and CSS3, one at a time.
- As a counter-measure for the DDoS attempt at My Opera, we have added Varnish caching for almost every My Opera page, protecting us against aggressive and bursty traffic.
- We have also introduced a new thumbnail generation service, allowing us to generate thumbnails for all photos on the fly, rather than having to store several versions of an image.
- We are more aggressively concatenating and minifying CSS for much faster pageload times, less required connections and less data transfer required with each pageload.
- Prior to the release, we replaced almost all of our servers with new, shiny hardware.
- Not to mention, we have massively improved our testing infrastructure allowing for better quality assurance of all new features.
- To go along with the improved testing infrastructure, we also put out a proper beta server, allowing our users to try out the new version prior to the release.
- Last, but not least, we have resolved MANY large and minor issues.
I know there have been many issues these first 10 days after the release, but we have been able to continuously and transparently ship fixes to the most severe cases and also many smaller ones. We try our best at reading your feedback and creating issues for whatever is reported. Of course, not every issue is given the highest priority, so it has to wait for higher priority tasks to be completed. Please be patient as we do our best to crush whatever nasty bugs there might be. And most importantly, have faith, for My Opera is moving in only one direction: Up-up-up!
