An inside look at how Opera works on site compatibility
27 CommentsPublished June 5th, 2006 12:07 PM EDT By Hallvord Steen
Editor’s note: Hallvord is a Core Quality Assurance JavaScript tester, site patcher and web opener for Opera Software. He has written extensively about site compatibility and standards on his personal blog.
The Problem
There are a number of sites that don’t work in Opera. It is a challenge that we have to tackle head-on to make sure users can successfully switch to Opera and use it as their main and only web browser.
In general, there are only two types of problems: when the site tells Opera to use different code than what the other browsers get, and when Opera gets the same code but fails to handle it correctly due to bugs or lack of features. For the end user it does not matter much what type of problem it is, of course. Either a site is usable or it isn’t. But in this article I’ll explain some of the stuff we do to make sites work, so it helps to start with some background reading.
To take the last category first, bugs and missing features are quite simple to understand and relatively easy to solve: analyze the bug, find the specification for the feature, fix and implement. It takes some time before the next version is out, but we’ll get it out there. An example of a missing feature is rich text editing support – in Opera 8.5 you could not edit text with WYSIWYG formatting on Blogger.com, in 9.0 betas you can.
A specific sub-class of this category is error handling – what to do when the site isn’t coded according to the “coding recipes” from W3C. The unruly growth and sheer scope of the Web means a lot of the content there was created very quickly, targeted at one or two browsers, and filled with errors. The only way we can get enough experience with such problems is to throw ourselves and our users into the deep end of the pool – yes, ladies and gentlemen, you are all guinea pigs helping us find, classify and work around the virtual rainforest of web coding errors. And I can tell from the stream of bug reports that you’re doing a brilliant job out there :) .
The other category is much harder. If a website actively detects Opera and responds by omitting content or telling us to do completely different things than the others, we really cannot do that much about it. There are a few options, and I will get to them.
The Solutions
So, now we have a bird’s eye view of the problems.. what do we do about it? Well, since we have several different types of problems we need a diverse toolbox for working on them:
Testing
To measure our progress on site-compatibility we have a hit-list of about 300 very important sites. These are tested systematically through the “OASIS” program and color-coded from green (”works perfectly”) to black (”unusable”). Bugs from these sites are given high priority.
Education
In Opera Core QA where I work, we sift through code analyzing problems all day long. Ever so often, it turns out that a problem isn’t a bug in Opera but a genuine error in the site. Now, if we simply marked the bug as invalid, that analysis and the time it took us to research it would be entirely wasted… the obvious solution is to try to get that information in the hands of the webmaster and suggest that they fix it. Educating webmasters about web standards helps Opera, helps webmasters, and improves the web. We have only just started really putting resources into this work, and there is lots and lots to be done so we’ll expand the team and keep pushing for web standards.
Core changes
Sometimes Opera might be able to handle the problem with some internal change. This isn’t easy though: we have to analyze very carefully what the other browsers actually do, and even if we try copying that exactly we may run into unexpected problems. Let me digress into the story about how I personally broke Google Maps, Flickr, a French bank and probably a ton of other sites…
For a long time we’ve known sites where all the links are dead because they are underneath other parts of the page. In IE you can click “through” invisible elements and hit a link that is “underneath” something else. So I thought: why don’t we do this click-through thing? It works fine in IE, it breaks pages when we don’t, and it sort of makes sense anyway. Some people thought it was risky, but I couldn’t quite imagine that we would break sites by emulating IE – as long as we could get 100% accurate, how could we possibly break anything?
So, with great help from Hixie I worked out exactly what IE was doing. We added this behavior to Opera. And Google Maps navigation, Flickr notes and French bank logins simply died. It turns out that all these services put something invisible on top of an image. If we click through the invisible bit and click the image nothing happens at all, because all the action is handled by the invisible parts we now ignore! The sites use various hacks to work around IE’s click-through behavior and for obvious reasons don’t apply these hacks to Opera because we’ve never been a click-through browser before.
So core changes are useful and fun but not a silver bullet. We fix something for site a, and it causes havoc on site b. Even when we get it perfectly right and do exactly what another browser gets away with.
Standardization
Sometimes webmasters use elaborate and buggy code because – well, they are doing something that the HTML / CSS / JS triplet isn’t terribly good at, or something which is not yet covered by a standard and thus not so reliable across browsers. Through the W3C and WHATWG we collaborate with others on standardizing solutions. For example, if you are a webmaster and want to let users choose dates from a calendar, in the past you had to use quite a bit of JavaScript. Today you can just use the WebForms2-specified < input type="date" /> – try it out! In the long run, browser collaboration and standardization of new and existing features can only help compatibility.
Indeed, all implementation difficulties past and present gain us experience that we can use for standardization efforts. It may be a small consolation if your favorite site behaves badly that if you take a long-term view today’s implementation problems might just be a part of the development of a better web.
Patching and site-specific preferences
Finally, the truly magic bullet in our toolbox: patching and site-specific preferences. With these tools, Opera can adapt to a site’s quirks in extremely detailed ways. For example, when mentioned blogger.com remembers that Opera 8.x doesn’t support rich text editing and is not aware of the fact that Opera 9 does, we use a browser.js patch to step right inside their JavaScript and change that specific browser detection to get the rich text editor anyway.
Patching and site-specific preferences are built directly on our experience from back when we added DOM support. During the days of Opera 6 more and more sites started using DOM scripting in fancy ways, and our DOM support was very limited. It took a major rewrite and the release of Opera 7 to get all those sites working, except that most of them STILL didn’t work in Opera 7 because they were detecting “Opera” and turning their fancy features off. Now we’re expecting the same reaction for our rich text editing support, but this time we’re making sure we have the tools to deal with it!
In Opera 9, every user can edit site-specific preferences from the [F12] menu and tweak settings to see if the site works. In addition, all Opera installations will download from us a set of fixes we have prepared for important sites and problems we have analyzed. This is the only way we can make things work if a site actively detects Opera, gives wrong instructions and doesn’t change it when we try to contact them.
Site patching is true magic, but it doesn’t scale. For privacy reasons, all installations download and apply all the patches. We will reach a performance limit. The magic bullets are wonderful but we can only shoot so many times. So the second part of this solution is a vibrant and positive community where users can help each other find the right preference or user javascript to make a site work. Say hello to Opera’s mailing lists, newsgroups, chat channels, My Opera site and userjs.org – a great and diverse user community where you are almost certain to find help somewhere. At the time of writing you have more than 380 000 other users at your fingertips over in the My Opera community. That’s scale.
If you enjoyed this post, then make sure you subscribe to my RSS Feed.
using
Great write-up!
using
Thanks for the write up. I knew most of this (and collaborate with you on some aspects), it’s still nice to see it in story form. Keep fighting the good fight; there are lots of us rooting for you, and for an ever-richer and increasing interOPERAble ecosystem of user agents.
Thanks,
Nate
using
Very interesting article. I need to work on my javascript (which is quite bad) because I’d like to help write the magic bullets mentioned in the article.
using
One of the ways that I follow through is to mail to the webmaster and remind him that his site is not “optimised” to the standards and doesn’t support Opera.
This article rightly points out the need to push towards uniform standards. The same argument can be extended towards Open Source and Open Standards which donot result in any lock in.
It takes time to make the voice heard and webmasters are lazy souls. However, if there are a lot of dissenting noises, they might be able to work on their code and make it happen. I hope!
Other way out, in my opinion, is to boycott the websites. I wrote in a mail to Delicious and Yahoo support (who as usual replied with their corporate ****) that they are “working on the issues”. If the majors cannot work on their web sites, it would be hard to expect anyone lower down the stream. On similar lines, Microsoft’s coders can’t make ANY bug free (or reasonably good) software. Strange.
using
A few days ago one of my friends was asking me why Opera cant handle the ajax based post rating system on his blog. Quite naturally he thought that it was opera’s fault, and the script didnt work as Opera didnt support Ajax. However, in reality it was because of the script. It sent the code only if Ie or Fx was detected.
using
I simply boycott the websites that don’t work in Opera. It’s the right thing to do – there are millions of sites to choose from and only one Opera!
using
Being Opera the fastest and the safest browser around, I’ve given up on sites that don’t work properly with it, either because they don’t recognize Opera or because their coding isn’t up to the standards.
It’s like Jakub81 writes: there are millions of sites out there.
And if there’s one that I’d really like to use (such as blinklist or clipmarks), tough luck: I just click on the «report a site problem», fill in the form and hope for better days to come
I strongly believe Opera is the safest future for surfers.
using
Hi,
this is a really cool articel. i know and i can follow the problems.
it is a realy big problem on many sites. a german bank (hvb: http://www.hypovereinsbank.de ; now a member of UniCredit) did make problems with old opera versions if you where known as opera. if you said I’m ie (^^) it was no problem :p
really crazy :p
also a big problem is the w3c, but i code with w3c since a few months and it is ok
and better for the users, because every browser should/or had work fine with it!
mfg
mabdul
using
Very nice article. I hope for less “standarization” problems in the future.
In most cases I can just ignore the sites, that don’t work under Opera, but sometimes it’s not possible…
using
I would like to hear more about this “OASIS program”. Does the word program imply any automation? I have started to test a Dutch magazine’s top-500 websites to create some statistics on compatibility (by judging the rendering result and checking whether a browser sniffer is used for the wrong reasons), but it’s an awful lot of work.
using
I think that education is the best solution, while more webmasters use the web standars will be better the compatibility in all browsers.
using
Scipio,
some things can be automated but you’re right that proper testing is labour-intensive. So is OASIS. While we have automated test setups OASIS is about a manual run-through with visual and functional comparisons with other browsers. It takes a lot of work and time but we get interesting stats and can prioritise fixes and patches that we might not have considered so important otherwise.
using
I wonder if Slashdot is one of those 300 important sites. The new slashdot design freezes Opera 9, when viewing a slashdot post (not the main page, though).
using
Great article. Nice to see how it works from the inside.
using
I use this site daily. Unusable with Opera 9.
using
i believe that opera can view any homepages without any problems, with next release of opera…
cmon opera, beat firefox & explorer!!!
using
I’m using opera now, but I’m going to switch.
>So the second part of this solution is a vibrant and positive community where users can help each other…
Great for people with tons of free time on their hands. I’d rather have something that works than have to dig through user feedback and communities in order to set up every website out there!
The REAL solution is to compete on a level like Mozilla did. Websites now almost always work with Firefox. Get people downloading your browser, get it in Linux distros, and the end user won’t have to read BLOGS all day to get it to work with Zillow, Yahoo sports, etc…
using
I certainly feel that a lot more sites work in Opera 9.x and now I know why. Thanks for this great post!
using
can you PLEASE modify Opera or make a javascript user that allows to view webpage in ajax
using
>can you PLEASE modify Opera or make a javascript >user that allows to view webpage in ajax
Yes. I need this so.
using
Where to report wrong rendering, wrong ‘behaviour’ or mistakes?
I don’t wan’t to switch to firefox or sth. else.
Please make the follwoing sites working.
http://www.wikispaces.com/help
https://developer.db4o.com/ProjectSpaces/view.aspx/Gstpl
In both cases editing of ‘wiki’ pages does NOT work.
Thank you, Peter.
using
Peter, you can report rendering (and other) bugs to Opera here:
https://bugs.opera.com/wizard/