Neil's Place

April 18, 2010

Status Update for April 18

Filed under: Mozilla, Status Updates — enndeakin @ 7:03 pm

Dropped Link Handling

I just checked in some notable changes to how the browser handles dropped links. It shouldn’t affect the behaviour of dropped links, but it does affect how script would respond to dropped links. This involves a new interface nsIDroppedLinkHandler. From an extension, one can use this directly or import the Services.jsm script and just access it via ‘Services.droppedLinkHandler’.

For browser front-end developers, this means that the single object located in browser.js will handle dropped links. This object and its related helper component will determine if a link is being dropped and whether security allows it to be dropped. This is important as one doesn’t want to be able to just create a page with a chrome url that can be dropped on the browser to do some nefarious thing. The old way used the nsDragAndDrop.js script for this, which is now obsolete.

See bug 545714 for more information about these changes.

The nsDragAndDrop.js script and its security check are now not used, except in one place in the tabbrowser which will be fixed up soon.

Resizers

A number of theme issues with resizers need to be fixed up (bugs 553796, 553752, 554810 and 558201. It seems like the best way to fix these is to include some static resizer images instead of relying on the native theme. I have a fix but need some images that are suitable for each platform.

Other

  • In bug 554635, <object> elements are been given a default tabindex of 0 so that they can be tabbed into. Those that load plugins already default to 0 so this applies to those that load other pages.
  • For bug 559561, I was investigating firing blur events when focused elements are removed from a document.
  • Took a short break from panel bugs this week. More to come next week.

April 9, 2010

Status Update for April 9

Filed under: Mozilla, Status Updates — enndeakin @ 10:16 pm

Improvements to panels are going steadily. I have worked out the issues with coordinate calculations on both Windows and Mac. This means that panels with titlebars appear and work properly. I also think I’ve almost worked out how popup activation and focusing should work. More details of what steps need to be done are on the Panel Improvements page.

I also have been investigating better support for the arrow style popups. These display an arrow that points to an element which can be used to get further information or perform actions related to the popup. I’m working on making the arrow properly position itself even when the popup itself is pushed around to fit on screen. See bug 554937 for this feature, and bug 558072 for some preliminary work.

Bug 379745 adds support for templates and to the sort service for setting three special sort hints. The first allows one to sort by integers instead of strings, overriding the type of the data. The second allows case sensitive sorting, which probably isn’t as useful except for file listings on some systems. The third allows only sorting in ascending or descending order, and not having the natural sort state. This is accomplished via attributes placed on the root element, or the <treecol> element.

Also, I investigated some minor focus issues and resizer rendering problems. For instance, in bug 554635, it was discovered that <object> elements default to not being focusable.

March 17, 2010

Panel Improvements

Filed under: Uncategorized — enndeakin @ 9:26 pm

Currently the <panel> element is used to create temporary floating popup windows. The normal type of panel should be used for popups that the user interacts with temporarily, then dismisses by clicking away from it, pressing a close button or presses the Escape key. In a sense, they act like modal dialogs but aren’t actually modal. The bookmarks ‘star’ panel is an example.

The second type of panel, created by setting the noautohide attribute to true, is of a more persistent nature. However, it currently isn’t as useful as it should be. I’d like to make it more useful by allowing it to be used for floating tool palette windows.

In some cases, a separate window could be used instead. However, this can be inconvenient when there are only a few buttons or fields due to having to create a separate file. Also, windows tend to be separate, more permanent entities that appear on the operating system window list, whereas panels are smaller and are generally associated with another window.

See Panel Improvements for some ideas on what kinds of features would be useful. Let me know if there are other features I’ve not included on the list, or if you’d like to help.

March 4, 2010

Clipboard Data

Filed under: Uncategorized — enndeakin @ 1:24 am

The drag and drop API available in Firefox 3.5 and later has simplified much code both in Firefox itself as well as elsewhere. The next step is to create a similar improvement for clipboard usage, which has a number of things in common with drag and drop. Once this is done, many redundant underlying pieces can be removed and we can rely only on the new APIs. Some parts of Mozilla code (such as the editing components) are still relying on the older interfaces; these will need to be changed. Fortunately, this will mostly involve simplifying code.

For instance, the current code needed to paste from the clipboard, or to cut or copy anything more complicated than simple text is generally 15 or so lines of code.

The general idea is to to allow the use of the DataTransfer object for clipboard operations, in a similar manner as can be done with drag and drop.

There are three possible approaches that could be used to improve clipboard access.

Method One

The first method is to fire three events, cut, copy and paste, which provide a data transfer which can be used to set or retrieve data. For example, for a copy event, a listener might set some text onto the data transfer. Once the event has finished firing, any data added would be put on the clipboard. This technique was first implemented by IE, but is also supported in Safari and Chrome. Mozilla implements the events, but does not supply the data transfer object needed to retrieve or modify the clipboard. Thus, in Mozilla, one can know when the events occur, but cannot affect their behaviour except to prevent it by cancelling the event.

A second piece implemented by IE, Safari and Chrome is three additional events, beforecut, beforecopy, and beforepaste. Mozilla does not implement these events. These are odd events, made more odd by poor and incorrect documentation. Their intent is to indicate whether the cut, copy or paste events are allowed or not. They are fired when, for example, the Edit menu is opened, to update the enabled state of the clipboard related menu items.

There are some disadvantages of this approach. The before events are an unusual design, as firing several events every time the Edit menu opens seems tedious. Compounding the problem for Mozilla is that Firefox has toolbar buttons for the cut, copy and paste operations which need to be updated every time the focus is changes, or when a selection is made. This means that the events need to fire quite often. Other browsers ‘work around’ this problem in different ways. IE has toolbar buttons, but never updates the enabled state of them, instead only updating the state on the menu when it is opened. Safari doesn’t have toolbar buttons so gets off easy. And, oddly, Chrome doesn’t update the enabled state of anything, even for its own UI. Likely, it’s just not implemented yet.

The other disadvantage is that the browsers only implement the events for text selections and textboxes, so you can’t really use it for clipboard operations on other kinds of elements.

Method Two

The HTML specification offers a different model that treats clipboard operations as a special form of drag and drop, in fact, treating the clipboard as just a special virtual drag source and drop target. It is expected that the normal drag and drop events would fire when using the clipboard commands on the menu, although the specification is vague about how this is supposed to work.

There are a number of disadvantages to this approach. Most notably, it isn’t compatible with other browsers. In addition, writing a simple handler for a clipboard action requires handling several events instead of one, separating the code to put items on the clipboard from the code to delete it in the case of a cut, which in some cases can be unweidly. Finally, this model doesn’t provide a means to distinguish a clipboard operation from a real drag and drop operation, nor to indicate whether a cut or copy only is allowed.

If you haven’t guessed already, I don’t like this approach at all. The only advantage of note is that keyboard only usage is provided (presumably) in typical browser usage automatically without having to write additonal code, although it is assumed that keyboard usage will use the clipboard instead, which may not be a given.

Method Three

An issue for Mozilla applications with either of the above approaches is that one can only manipulate the clipboard when the event is fired. Thus, one cannot, for example, empty the clipboard or retrieve data from it outside of an event. This isn’t an issue for web pages though; as security concerns would want us to prevent web pages from accessing the clipboard unless there has been a real request from the user to do so.

In a sense, IE (unintentionally) handles this by providing the data transfer as a global property of the window called clipboardData. An application can check this property to retrieve the data on the clipboard at any time, and can modify it at any time as well.

In Mozilla, this approach could be used as well, although changes from web pages would be restricted to accessing and modifying this object only during the clipboard events. Extensions and other privileged code could access this at any time. The disadvantage here is when you’re putting multiple types of data on the clipboard, since it would need to update the whole set on the real clipboard every time one type of data is changed.

In case you’re wondering what security issues exist, consider if a web page could retrieve the clipboard data at any time without user intervention. It’s possible that the clipboard could contain passwords or other sensitive information. Likewise, modifying the clipboard is also undesirable, as a web page could put ads onto the clipboard. Naturally, of course, this could pose a problem anyway, but at least we can restrict this to those cases only when a user requests a clipboard operation.

Conclusion

I’m favouring method one, with an appropriately secured method three, possibly chrome only. I’m not sure about the before events though, but there is a need to be able to indicate whether a cut, copy or paste event is allowed at a given time, and to be able to determine this state quickly.

One possibility I’d thought of is to make use of the global clipboardData’s effectAllowed property. Essentially, an application would update it when necessary, and the corresponding UI will check this state as needed. We can solve the issue of the events only firing on selections and text inputs only by instead firing the events on the focused content instead, perhaps only in chrome contexts. For example, when a mail message is selected, the effectAllowed property might be changed to indicate that a cut or copy was allowed, but not a paste. The property would be reset automatically when the focus changes.

An example:

<listbox onfocus="clipboardData.effectAllowed = 'copy';"
         oncopy="event.dataTransfer.setData('text/plain', this.selectedItem.label)">
   <listitem label="Item Value"/>
<listbox>

For XUL, of course, the controller and command mechanism would still be used as well, and would likely be a cleaner implementation in the case above.

December 10, 2009

Template Logging

Filed under: Uncategorized — enndeakin @ 5:03 pm

I’ve been going over some of the patches I’ve had sitting around for a while. One of the oldest was adding some error checking and logging to XUL templates.

See bug 321169 to try this out.

October 24, 2009

Command Updating Performance

Filed under: Uncategorized — enndeakin @ 2:13 pm

This week, I’ve been investigating how to improve command handling in XUL, both in terms of manner they’re used and particularly, performance. Command handling is what deals with performing actions in response to menu or keypresses. There are several types of commands. The first are those like cut and paste that typically appear on menus, where the bulk of the handling deals with ensuring that the menus, toolbar buttons and actions are enabled and disabled at the right time. The second are input field cursor and selection commands (such as move to next word, page down, etc). Mozilla cheats a bit and doesn’t check if they should be enabled as they don’t normally have UI that shows this. The third are complex edit commands such as bold, font changes and so forth. These are similar to the first set really. As an aside, it would be quite nice to be handle all of these in a similar fashion.

For whatever reason, most of the handling is done in C++ with a small amount of code done in script. In many cases, though the script is unnecessary. For example, a typical cut command looks like the following:

<command id=”cmd_cut” oncommand=”goDoCommand(‘cmd_cut’)”/>

This is quite pointless as the act of ‘do’ing the command should just be the default behaviour.

Updating command state is another issue and one which can be quite slow. In fact, a workaround was added just to improve performance due to this. Updating command state refers to the process of updating the UI when the focus or selection changes to reflect which commands are now enabled. For example, a cut operation is only valid when there is text selected.

Here’s an overview of how the update process breaks down in time used:

Fire commandupdate event at updaters: 17%
Retrieve the topmost window: 14%
Retrieve the command dispatcher: 8%
Retrieve the right controller: 33%
Checking if the command is enabled: 15%
Set disabled attribute on command elements: 13%

I was expecting the latter two steps to be the bulk of the time used, but this wasn’t the case. This was a good sign since the other steps are easier to optimize away. One particular issue is that, except for the first step, the remaining steps occur again and again for every command that needs to be updated, which includes additional time due to calling into Javascript and back several times.

Instead, by optimizing to perform the repeated steps only once, as well as not calling into Javascript unnecessarily we can reduce the amount of time taken up. Most of the time, the script just calls back to update each command in the same way every time much like goDoCommand is called in the earlier example, so this is also a bit silly.

I created a simple performance test which updates the typical eight edit-related commands one thousand times. With current code, the test took over 625 milliseconds to complete (which means about half a millisecond each). This isn’t much individually, but might be noticeable if lots of commands were being updated.

I created a simple implementation of the changes described above, still a work in progress, and have reduced this to about 60 milliseconds to update one thousand times. This is a speedup of over 10 times.

I have also been experimenting with caching the list of commands and their disabled state, rather than setting a disabled attribute on the command elements. This can improve the time to 40 milliseconds. One advantage to this is that it could mean that the command element isn’t needed for many commands — if we cache the disabled state internally, in many cases we don’t really need the command element any more.

In many cases, the UI for commands such as undo isn’t actually visible until the Edit menu is opened. Using some smart caching, we can make the work of updating these commands only occur when the menu is opened. This already happens to some degree, but much additional and extraneous work is done upfront rather than later.

Next, I’m going to investigate some of the other types of commands and further see how a smart command caching mechanism might work.

October 5, 2009

Visible Focus Rings

Filed under: Uncategorized — enndeakin @ 3:59 pm

Platforms differ in the way that they show which element is focused. So does Firefox, which differs in a number of ways from platform conventions.

The typical way is to draw either a glow or dotted outline around a button or input field that accepts keyboard entry. However, this behaviour can sometimes cause unwanted visible display or bad interactions when the keyboard is not being used. For instance, when clicking a link on a page, the behaviour is to draw a visible dotted outline around the link, then load the next page. This is useful to know that the link has been clicked, but causes an unnecessary outline that can disrupt the appearance, especially if the link is an image or part of an image map.

One significant advantage of showing the link however, is that when going back to a page, one can immediately see where the keyboard focus is, and for non-keyboard use, indicates clearly which link was clicked, a useful indicator when visiting a series of links in succession.

Another behaviour is that, especially on Mac, various elements don’t switch the focus when clicking on them. For instance, clicking a checkbox in a dialog box or selecting from a dropdown list doesn’t change which element is focused. In this image, the first dropdown has a visible focus indicator (a blue glow around it), but clicking the later dropdown has not changed this.

On Windows, what used to be the norm is to show dotted outlines around focused elements, and show underlines under letters of the labels of controls or menus to indicate a key which can be used to focus or activate it. In today’s versions of Windows, though, that behaviour isn’t used, although it can be enabled through a system setting. Instead, these indicators are only visible once the user presses a key. For instance, pressing Alt shows the menu label indicators, as well as shows a visible focus ring around the element that accepts keyboard input. Tabbing to an element does similar.

The goal of bug 418521 is to improve the situations described above in Mozilla to be more compatible with how the system behaves. Followup bugs will address additional issues with specific controls. More specifically, this means:

On Windows, don’t show indicators at all until the user uses the keyboard to access an element. But, respect the system setting if the user wants them visible at all times.

On Mac, don’t focus elements when clicking on them, except for lists and text input fields.

On Linux, always show indicators. This is the current behaviour so Linux users won’t notice changes from this bug.

One change of note on all platforms is that links will no longer show the dotted focus outline when they are clicked, whether they are image links or otherwise. However, the dotted outline will be made visible when navigating back to the page, or when switching tabs back to that page. This solves the issue of not adding visible noise to the page in normal usage, but doesn’t prevent the user from seeing which link was clicked when going back to that page again.

For this change to be effective, a new css pseudoclass is proposed, to accompany the existing :focus pseudoclass.

The :focus pseudoclass will continue to work as it does now. If will match any element which is focused. However, the new pseudoclass :-moz-focusring will only match when the element is focused and the focus indicator should be shown. So if you are creating a theme for instance, the :-moz-focusring pseudoclass should be used to show focus indicators as this deals with all of the platform specific details. For example:


button:-moz-focusring {
  outline: 1px dotted black;
}

For details of the changes see comment 57 in the bug.

September 8, 2009

How to Fix Focus Related Problems in Tests

Filed under: Uncategorized — enndeakin @ 10:27 pm

One common problem with the existing tests is that they make a number of assumptions about a window being focused. For instance, tests tend to assume that once the load event fires, everything is ready to go. This is incorrect. In fact, the window may not even be visible yet. Because of this, some tests tend to fail randomly, especially on Linux.

The focusing of a window can occur before or after the load event, depending on circumstances and platform specific variations. For instance, an incrementally loaded page can be focused long before the page has finished loading, whereas a new XUL window can be focused afterwards, since the entire XUL file needs to be parsed and laid out before the window size is known.

Even the performance tests aren’t immune to this. If they operate based on the load event, they test only the time it takes to load, not the time to focus a window or page.

A number of things rely on a window being focused in order to work. For instance, tests that synthesize key presses or open popups. These are kinds of things that will fail when the window isn’t focused.

One common, yet not actually correct solution is to call window.focus(). This function is typically used to raise (bring a window to the front) and focus the window. However, it should not be assumed that the window.focus() method will do so before it returns. (Technically, on Linux the window manager isn’t required to honour this request at all). For this reason, it is best to wait for a focus event on the window before proceeding.

This can be a bit of hassle, so a few days ago I added a new function to do this for you, called waitForFocus. It’s very easy to use, and does lots of magical incantations for you. Just add something like the following at the end of the test’s script:

SimpleTest.waitForFocus(callback, window)

Set callback to a callback function to be called when the specified window has been loaded and focused, typically this will be the function that executes the actual tests. (The window argument is optional, if it happens to be ‘window’) If the window is not focused, an attempt is made to focus it. Don’t worry if the window has already loaded or is already focused, the function will take care of checking that for you. All you need to do is add the line above and everything will work out. No need to listen to any load or focus events. At least that’s the intent. The tests that now use this function haven’t failed due to focus issues yet.

Currently, waitForFocus also logs some debugging output. I’d expect this to be removed at some point — for now it’s being used to look for problems that might occur in the near future.

This method should be used for tests that:

  • synthesize key presses
  • tests that open popups
  • tests that open a new window
  • tests that close a window
  • tests that rely on focus rings to be drawn
  • tests that rely on focus or blur events to occur

Know any tests that could have problems? Why not fix them?

September 1, 2009

Resizing Text Boxes

Filed under: Mozilla — enndeakin @ 7:42 pm

In bug 167951, I am implementing text areas which can be resized, which will look something like the following:

The notch in the corner will allow the box to be resized smaller and larger. This will be available for all HTML <textarea> elements as well as XUL multiline textbox elements. However, for XUL elements, a resizable attribute set to either true or false may be used to enable or disable this feature. The default is false as it is likely that multiline textboxes in dialog boxes won’t have room to be resized anyway.

Adding support for this feature involved adding a number of other XUL features. The first, in bug 510335, adds support to the <stack> element for right and bottom attributes, in addition to supporting left as top as it does currently. This allows alignment of the children of a stack to any of the four sides by specifying a margin. For instance, this example positions a button 10 pixels from the right edge and 5 pixels from the bottom edge.

<stack width="100" height="100">
  <button label="Hello" right="10" bottom="5"/>
</stack>

You can also specify all four positions, and they then just act like margins from the stack’s edge.

The second feature is extending the <resizer> element in bug 511180 to allow resizing any element. Currently, the <resizer> is always used to resize the window. You can see this kind of resizing notch in use in Firefox in the lower right corner of a window below the scrollbar.

But, by adding an element attribute, you can specify the id of a specific element to resize. Most common, you would use the special value ‘_parent’ which means to resize the parent element. The following pattern might be common:

<stack>
  <button label="Resizable"/>
  <resizer dir="bottomend" right="0" bottom="0" element="_parent"/>
</stack>

All of these features are works in progress, so note that the specifics may change.

August 27, 2009

Mouse Capturing

Filed under: Mozilla — enndeakin @ 11:49 pm

I have been working on adding a mouse capturing API, mostly compatible with the IE API. This means that when the mouse button is pressed on an element, you can trap the mouse movement and redirect the movement events to this element instead. This mechanism appears in Mozilla in a number of places already, although there is currently no public way of doing this. For example, try clicking on a scrollbar thumb and dragging it. Notice how when the pointer is moved away from the scrollbar, it continues to move or traps all mouse movement such that other parts of the UI do not respond. When the mouse is released, everything reverts back to normal again. This is used in few other places as well, for example the draggable splitters between columns in a tree view, the splitters between frames and when drag-selecting.

In bug 503943, I am implementing this feature for both XUL applications and web pages.

Using the feature is quite easy:

onmousedown="this.setCapture(false)"

Within a mousedown event handler, just call the setCapture method. This enables capturing on the element until it is either stopped by calling the releaseCapture method, by releasing the mouse button, or if a drag begins. To prevent odd or possibly nasty things from happening, setCapture will only work while processing a mousedown event.

The argument to setCapture specifies whether children of the capturing element will receive mouse events or not. If false, then they do. If true, then only the capturing element receives them. In the former case, you could set capturing on a containing box, yet still handle the mouse as normal inside it.

There are three small incompatibilities with the IE implementation. First, IE allows setCapture to be called at any time and not just when a mouse button is pressed down. This isn’t recommended and usually causes the mouse to go into a funky state where it doesn’t seem to work properly. Naturally, it’s also very easy to abuse this. So Mozilla won’t allow this and instead only allows capturing during a mouse down.

The second difference is that the argument is optional in both IE and Mozilla, but defaults to false in Mozilla, but true in IE. (We’d have to implement some idl changes to change this). It isn’t too much of a burden since you can always supply the argument if this is a problem.

The third difference is that IE has a different event model so compatibility can be affected and can impact what elements seem to receive events.

However, these differences should be minor for normal usage.

« Newer PostsOlder Posts »

The Shocking Blue Green Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.