User Interface bug tests

Test your Windows applications for common bugs in the User Interface (UI). This article shows 10 ways to find nasty bugs.

Your app is almost ready? Now it's time to find those nasty UI bugs that will show up on other computers, but not yours.

You don't necessarily need another computer to run these tests. Just change your system settings. If you have access to other computers, it's always a good idea to test your application on them, of course. Settings and hardware will vary from system to system. These instructions are written for Windows XP, but you can apply them to other Windows versions as well. Be your program in C++, C#, Java, VB, VB.NET or anything else, test it!

1. Change the color scheme

Color test. Change the Windows color scheme. Try with a few color schemes such as Windows Standard, Wheat and Desert. This test is almost certain to reveal a few controls with bad color settings. It is common bug to use fixed colors in controls that would require system colors. As some users will have different system colors than yours, the fixed colors will look odd next to the system colors of their tastes.

High contrast test. Now change the color scheme to a High contrast scheme. Try especially the mode with the black background. Can one still use your program? The High contrast test reveals controls that use fixed colors when they badly require system colors. An example is a text box with a fixed black foreground color and a system background color. Under regular settings, it displays black text on a white background. When the background color is black, the black text doesn't display.

To change the color scheme, right-click your Desktop and select Properties. Then choose the Appearance tab.

2. Run with Large Fonts

Programs often behave badly when Windows is set to run in another font size mode. The typical system setting is Small Fonts (100%), but many systems run under other sizes to keep text readable.

Large Fonts test. Select Large Fonts (125%) to reveal the most important bugs. If you never tried this, you'll be surprised about how programs look in it. Typical bugs: Text doesn't fit in a control, labels are truncated, icons and toolbar buttons look bad.

Once your program runs correctly with Large Fonts, try some non-standard font sizes as well.

To change the font size mode: Right-click the Desktop, select Properties, open the Settings tab and press Advanced.

3. Change the appearance settings

Try with different appearance settings. In Windows XP, try Windows Classic style and Windows XP style.

Scrollbar test. Did you know that users can change the Windows scrollbar size? Change the Scrollbar size setting and see that your scrollbars appear in the defined size. — This test is especially nasty for a Visual Basic 6.0 application. In VB6, scrollbar sizes are fixed at design time. They don't obey the system-wide settings unless you code it. Call the GetSystemMetrics API function with parameters SM_CXVSCROLL and SM_CYHSCROLL.

Size tests. Also change the title bar size and window border width. Check to see that your windows still display correctly. An excessively thin (or wide) border can make some windows look bad.

To access the appearance settings: Right-click the Desktop, select Properties, open the Settings tab and press Appearance (and the Advanced button).

4. Run in VGA mode and few colors

Now we're heading to the real acid test: the dreaded VGA test.

VGA test. Set your screen to VGA mode with 640x480 pixels and 16 colors. Is it possible to use your program in this mode? Do your windows scale to the small screen? Are they truncated on the borders, or do they just appear too big to use? How about the colors?

You're right, VGA is really, really old. Almost nobody uses it. Even though today's monitors generally display 1024x768 pixels or more, it is good to support less pixels too. Once your program works in VGA mode, it also supports small window sizes for other uses. One can use it as a small window next to other programs. This is very handy for drag & drop and copy & paste. It's also useful when running your software in a virtual machine such as a small VirtualPC window. VGA mode may be required if the computer is booted in Safe Mode. It also helps with mobile devices with a low resolution display.

To switch to VGA mode, right-click your Desktop and select Properties. Then choose the Settings tab. Drag the Screen resolution slider to 640x480 and make sure the color quality is 16 colors.

SVGA test. If you cannot go down to 640x480, do a minimum test with 800x600 (SVGA resolution). This is still the default mode on many systems. You can also run this test on another computer or a virtual machine if required.

256-color test. Even if your application runs correctly in the 16-color VGA mode, it might not do so under 256 colors. Set your screen to 256 colors to try it. While not a typical mode on modern computers, the 256-color mode is unique in that it uses a color palette. Is your application compatible with palettes? Some programs are not.

Why would someone use the 256-color mode these days? It is sometimes required to run legacy programs. Once the display is set to 256 colors, some programs may cease to run. Make sure yours isn't one of them.

Your program doesn't need to look great in VGA, SVGA or 256-color mode. If one can run it, that's enough.

5. Run multiple monitors and other resolutions

Multiple monitors test. Install 2 or more monitors. Check to see if your windows open up in the correct monitor. What if you drag a window from a monitor to another, does it still display correctly? And — can your application take advantage of the extra monitor(s)?

Resolution test. Run your program with atypical screen sizes. How does it look maximized on a wide screen, say a 1440x900-pixel notebook monitor? Try as high a resolution as possible. When a user maximizes your application, does it still look good?

Run these tests on other computers when necessary.

6. Change regional and language settings

Local settings are a big source of UI bugs, misbehavior and even crashes. If there is a chance your program will be used in a different country or under different regional settings, you need to test this. Even if your program is written in English for English-speaking users, you need to test it. On the other hand, if your program will only be used in your own office with identical settings, regional settings aren't an issue, so you can skip this section.

The settings in this section can be found in Control Panel under Regional and Language Options. At a minimum, try a couple of European locales and a Far East locale, say Japan. Change all of Regional Options, Location, and Default Input Language.

Keyboard test. Different languages have different keyboard layouts. Check to see if all the keyboard shortcuts still work. Does your application use exotic shortcuts such as Alt+< or Ctrl+-? Keep in mind that many keys are in a different position when running under other language settings. Even if it's possible to press the equivalent of Ctrl+[ it may be too awkward to use.

Pay special attention to the use of Alt Gr key, which is an important key on many European keyboards. Alt Gr is located in the place of the right Alt key (immediately to the right of the spacebar). Alt Gr is the equivalent of pressing Ctrl+Alt. It is required to produce many characters. To name an example, getting the @ character requires pressing either Alt Gr+2 or Ctrl+Alt+2 on the Finnish keyboard. If you reserved Ctrl+Alt+2 for a shortcut, your program might not support the @ character. This bug actually happened with an old Word version. It was impossible to type the @ character if you were Finnish.

It is safe to use the letters A-Z and the function keys F1-F9 & F11-F12 in Ctrl and Alt combinations. F10 has a special functionality in Windows.

Character set test. In addition to the keyboard layout, some languages use character sets other than yours. Try running your program in the Japanese locale to see if it still treats non-ASCII characters correctly. In addition to Japanese characters, check to see how it treats common European characters such as ä or é.

Regional settings test. Regional settings affect the way your application displays and reads date values, currency values and decimal numbers. If your program is mainly for English speaking users, it might not be necessary to localize the display of these values. However, you must make input work! Programs do crash and/or behave oddly under different regional settings.

Dates present a common input problem. Your program expects a certain input format, but displays another. Your program might display the date 1/15/2007 as 15.1.2007, yet it might expect to get the date back in m/d/yyyy format.

If your program has a calendar feature, verify it still works when the week starts on Monday instead of Sunday (or vice versa) and when the date is displayed in a different format to yours. A localization bug can even cause your program display dates in a garbage mixed-language format such as "15. Januaryta 2007".

Decimal points are a big source of problems. It's often a decimal comma, not a point. Your application might well display the decimal value of 1½ as "1,5" but still expect "1.5" as input. — As a side note, a bug (feature) in Excel's text file import turns decimal values such as "1.5" into "1 May" when run under Finnish regional settings. As it happens, "1.5" is not the way to write "1 May", so it's simply a nasty localization effect. If localization effects can hit Microsoft, they can hit you too.

Currency values are a source of confusion. Suppose your program handles dollar values such as $123. If you display them localized as £123 or €123, the user will wonder what the price is. Is it dollars, pounds or euros? What is more, some locales display currency values as 123 € or 123 kr, with a suffix rather than a prefix. Be prepared if you accept monetary values as input. What is more, don't assume the user is always working with his/her own currency. It's better to drop the currency symbol altogether if you aren't sure what the currency is.

Localized strings. Numeric values are sometimes "stringified". A function can return a Boolean value as "True" or "False". Another function can display months as text ("Jan", "Feb", "Mar" etc.). Does this still work the same under other locales? You might well get "Tosi" or "Epätosi" for a Boolean and "tammi", "helmi", "maalis" as the month. Rather than stringify values, it's safer to use numbers. — This tip is based on bad experience with Visual Basic 6.0 automatically converting numbers to localized strings. Sometimes this happens without the programmer noticing it. It's a really bad thing if it happens when saving a data file that should be loaded in a different locale. You risk losing the data.

Sort test. The language setting also affects the default sort (collation) order. If your application uses sorted lists, check to see how they look and if they still function. To your surprise, they may use a local sort instead of the default ASCII order A to Z.

Search test. Searching in a sorted list is a potential source of bugs. The problem affects binary search and every other search done in an alphabetically ordered list.

From now on, we use the term simple sort for any simple order rules not specific to a language or region. A typical simple sort order is the ASCII order. A simple sort is the default in many programming languages. Local sort means language/region specific sort rules.

Sort and search must use the same sort order. Searching with simple sort rules in a local sorted list will eventually fail. Similarly, searching with local rules in a simple sorted list fails as well. Example: A simple algorithm looks for Y words between X and Z, but the Y words are between I and J in a Lithuanian list. The search will not find any Y words.

Search bugs are hard to reproduce unless you know what you're looking for. Check to see if your program uses local sort or search rules. Test it if it does.

Resource test. If your application uses resources (resource strings, bitmaps and so on) stored in the executable or dll, make sure the resources still load in another locale. Use the Neutral culture if required. Also check to see that your help file displays correctly.

7. Test alternative input devices

Everyone uses a mouse, right?

Mice are not similar. Some have just 2 buttons, some have a wheel, some have a 4-way wheel, some have 5 buttons or even 8. Some come in the form of a touchpad, pointing stick, trackball, touchscreen, rollermouse, pen or a tablet + pen.

New mouse test. Find some of these non-mouse pointing devices to get a feeling of them. Can you click all the small buttons of your UI? How about drag & drop? Scrolling? It's not always that easy.

No mouse test. Drop the mouse altogether. Can you access all features? Is it awkward?

The obvious solution is to enable keyboard access to all features. For each mouse action, provide a keyboard alternative. Add access keys to your menus, buttons, text fields and other widgets. Add menu/keyboard alternatives for all toolbar buttons.

With some pointing devices it's impossible to click accurately, even with a steady hand. Make all buttons big enough. Tiny 9x9 pixel buttons are not cool (even if WinAmp uses them). Enlarge drag areas, such as splitter bars. Again, add access keys.

Support the "application key" on the Windows 95 keyboard. It's the one between the right Windows and Ctrl keys, the one you press to open a popup menu. It's a quick & easy way to access context-specific features from the keyboard.

The mouse is a great device. It's especially great for novice users. For power users, have those keyboard shortcuts available. Power users will be more productive when they can avoid moving their hand between their mouse and keyboard.

8. Run off a read-only drive

Read-only test. Burn your program to a CD-ROM and run it off the CD-ROM. Alternatively, run it off a read-only USB stick. You could also run it off a write-protected network drive.

This is a quick way to reveal file access bugs where your program cannot write to the installation directory, nor to the entire drive. Does your program even start? When write access fails, does your app look for another drive? Where does it create its output and temporary files? Note that it's not necessarily your own code that needs to write to files. It can be an external library you're using.

This test also shows you the performance when disk access is slow. Even when write access is not a problem, extremely slow read access might prevent use.

9. Test on another operating system

OS test. Be sure to install and run your program on another operating system than yours. There are nasty effects to expect. As far as the user interface is concerned, you can find out that the mouse wheel doesn't function, it functions in a different way you thought, mouse back/forward buttons don't respond, the keyboard settings are different, the screen mode is different, icons are of a different size, system scrollbars look different to yours, window borders are different, etc.

10. Test on a virtual machine

Virtual machine test. Use virtual machine software such as Microsoft VirtualPC for more testing. A virtual machine lets you safely install several operating systems (OS) under the same system. You don't need to mess up with your own system or buy more hardware just for the purpose of testing.

Once you have installed and configured an OS on a virtual machine, make its virtual hard drive read-only. Run your tests on the system. Once done, you can delete the changes and restore the original configuration. This way you don't need to worry about uninstallation, changing system settings, corrupting the registry etc.

User Interface bug tests
URN:NBN:fi-fe20071002