Monday, December 18, 2006

NetCF 3.5's Finalizer Log

The .NET Compact Framework has had several loggers (error, interop, loader, networking, and security) in previous versions.  NetCF 3.5 introduces the finalizer logger, which is useful for performance tuning your app by identifying which objects are not being Dispose()'d before being discarded for the garbage collector.  This post discusses this logger, how to enable it, use it, and it's file format.

Enable the logger

The finalizer logger is part of NetCF's logging infrastructure, and must be enabled in two places: once to enable the entire logging facility, and once more to enable the finalizer logger specifically.  Why twice?  Logging takes its toll on performance of all NetCF apps running on the system with logging enabled.  By providing a master on/off switch for all NetCF logging as well as individual switches for the individual loggers you're interested in, we hope to make it easy for you to have your favorite set of loggers ready to go, but only activated when you want them.

The logger settings are kept in the registry of your device.  Your device has no registry editor built in, so we use the Remote Registry Editor on your PC.  Here are the steps:

  1. Start your ActiveSync connection between your development computer and your device.
  2. Click Start -> All Programs -> Microsoft Visual Studio 2005 -> Visual Studio Remote Tools -> Remote Registry Editor.
  3. In the "Select a Windows CE Device" dialog, select your device (or emulated device).  Click OK.
  4. In the left pane, drill down through HKEY_LOCAL_MACHINE to the Software\Microsoft\.NETCompactFramework\Diagnostics\Logging key.  If you have never used logging before on this device, you will probably have to create the last two keys.  Remember keys are the folders on the left.  Values are the key/value pairs on the right.
  5. Add or set the Enabled value name to the DWORD value of 1 within that key.  This enables NETCF's logging facility.
  6. Create a ...\Diagnostics\Logging\Finalizer key and select it.
  7. Add or set the Enabled value name to the DWORD value of 1 within that key.  These two Enabled values enable NETCF's finalizer logger.
  8. (Re)start your NetCF app on your device.

Analyzing the log file

Once your app starts, a file in your application's directory is created called netcf_Finalizer.log.  You can read this file even while your app is running, both from the device and over your ActiveSync connection on your desktop in Notepad.  Here is some sample output from the log:

[2006-09-22T10:11:04] [Microsoft.Win32.SafeHandles.SafeRegistryHandle] finalizer called.
[2006-09-22T10:11:04] [Microsoft.Win32.SafeHandles.SafeRegistryHandle] finalizer called.
[2006-09-22T10:11:04] [System.AppDomain] finalizer called.
What we have here is the timestamp of when the GC kicked in and ran the finalizer for an object.  The second element is the class of object that was finalized. 

 

Using the logger to tune your app

The GC works in the background to free memory that is no longer being used.  If it finds a discarded object with a Dispose method that has not yet been called, that object gets put on the queue for the finalizer thread to Dispose of, and its memory is not freed until later.  This delay takes its toll on memory and performance for your app.  Best practice suggests that you explicitly call Dispose on any object that provides it before you discard it.  (This is especially important for objects that tie up unmanaged resources, but that's another subject.)  This log identifies where your trouble spots are so you can add Dispose calls to your code.  In fact the log also includes where NetCF doesn't call Dispose, but you can't do anything about that.

In an ideal world, the log file would indicate where YourObject.Dispose() should have been called.  In a nearly perfect world perhaps the log file would indicate where YourObject = new YourClass() was called for the object being finalized.  Unfortunately that kind of information is not available to the CLR.  But having the class name of the objects not being finalized will definitely be a huge step in the right direction to your search for what is slowing down your app.

Monday, October 02, 2006

Platform detection III: How to detect a touch screen on Windows CE in .NET CF

Pocket PC's have touch screens.  Smartphones don't.  While it is straightforward to determine which of these a Windows Mobile device is, there are Windows CE devices that are neither.  Some may offer touch screens while others don't.  Since a touch screen is really what you may be after (to determine whether you can expect a user to click somewhere on the screen, for example), the better long-term solution would be to check for the touch screen itself.  We can't assume that a future version of Windows Mobile won't change the distinctions that currently exist.  Here I will discuss ways you can do your own detection.

The .NET Compact Framework doesn't offer a way to easily check for the presence of a touch screen.  I worked on adding this feature to NetCF Orcas, but we had to abandon the project after it was very nearly there because we found that some manufacturers did not implement the proper detection APIs, which gave us erroneous results on some hardware.  Instead, I'll show you how we were going to do it, and you should be well on your way.

Strategy

Any Windows CE or Windows Mobile device with a touch screen will have a driver whose filename will be in the registry under the HKLM\Hardware\DeviceMap\Touch key, value "DriverName".  If this value exists in the registry, that is your first clue that a touch screen is present.  Even if the value doesn't exist, the default driver name of touch.dll may still be present and there may be a touch screen.  Then all that's left is check to see if the driver is in \Windows.

And a caveat: A bug in the Windows Mobile Smartphone 2005 emulator leaves a touch screen driver present even though there is no touch screen.  As a result, your detection function should have a special check for Smartphone 2005 and below and return false in that case, regardless of whether a touch screen driver is present.

A C# implementation

This post builds on the first and second posts in the Platform Detection series.  You'll need to build on the code from those posts for this post to compile and run.  Like the previous posts, this code uses partial classes, so you can copy and paste the previous  code into one file and this code into another (removing the prior posts' Main methods) and your project will compile and run.

using System;
using System.IO;
using System.Windows.Forms;
using Microsoft.Win32;
using System.Runtime.InteropServices;
using System.Text;

namespace PlatformDetection
{
internal partial class PInvoke
{
const int MAX_PATH = 260;
[DllImport("Coredll.dll")]
static extern int SHGetSpecialFolderPath(IntPtr hwndOwner, StringBuilder lpszPath, int nFolder, int fCreate);

public enum SpecialFolders : int
{
CSIDL_WINDOWS = 0x0024,
}
public static string GetSpecialFolder(SpecialFolders specialFolder)
{
StringBuilder path = new StringBuilder(MAX_PATH);
if (SHGetSpecialFolderPath(IntPtr.Zero, path, (int)specialFolder, 0) == 0)
throw new Exception("Error getting Windows path.");
return path.ToString();
}
}

internal partial class PlatformDetection
{
public static bool IsTouchScreen()
{
string driverFileName = Registry.GetValue(@"HKEY_LOCAL_MACHINE\Hardware\DeviceMap\Touch",
"DriverName", "touch.dll").ToString();
string windowsFolder = PInvoke.GetSpecialFolder(PInvoke.SpecialFolders.CSIDL_WINDOWS);
string driverPath = Path.Combine(windowsFolder, driverFileName);
bool driverExists = File.Exists(driverPath);

return
driverExists &&
// Windows Mobile 5.0 Smartphone emulator and earlier has a driver, but no touch screen.
!(IsSmartphone() && IsEmulator() && Environment.OSVersion.Version.Major < 6);
}
}

class TouchscreenProgram
{
static void Main(string[] args)
{
MessageBox.Show("Touchscreen: " + (PlatformDetection.IsTouchScreen() ? "Yes" : "No"));
}
}
}

A C++ non-implementation

I won't actually include the C equivalent of the above code, but be forewarned if you do it yourself: For the C implementation I thought it would be a good idea to actually LoadLibrary("touch.dll") and query the driver to check whether the driver is valid and running.  In my tests, the later call to FreeLibrary would crash my process.  Watching the library reference counter revealed that my LoadLibrary incremented the counter from 1 to 2, and FreeLibrary returned it to 1.  But for some reason some touch screen drivers don't like having FreeLibrary called when they aren't actually being unloaded.  Don't load the library as part of your check, or you may find your app crashing on some devices.

This is the last post in a series of three on platform detection.  This post builds on the previous two: discerning Smartphone and Pocket PC, and detecting the Microsoft Device Emulator.

Friday, September 22, 2006

Platform detection II: Is your app running on Smartphone or Pocket PC?

While both Smartphones and Pocket PCs are based on Windows Mobile, there are some very important differences for developers who are targeting both platforms.  Not the least of which are lack of LinkButtons and other clickable elements on Smartphones (since they have no touch screen).  In this post I show how to detect whether your app is running on a Smartphone or a Pocket PC style Windows Mobile device.

It is a simple call to SystemParametersInfo to find whether you are running on a Smartphone or a Pocket PC.  You need a few constants defined and the P/Invoke marshaling code to make it work from managed code however.

This post builds on the first post in the Platform Detection series.  You'll need to build on the code from that post for this post to compile and run.  Like the last post, this code uses partial classes, so you can copy and paste the previous post's code into one file and this code into another (removing the first post's Main method) and your project will compile and run.

using System;
using System.IO;
using System.Windows.Forms;
using Microsoft.Win32;
using System.Runtime.InteropServices;
using System.Text;

namespace PlatformDetection
{
internal partial class PInvoke
{
public static string GetPlatformType()
{
StringBuilder platformType = new StringBuilder(50);
if (SystemParametersInfo4Strings((uint)SystemParametersInfoActions.SPI_GETPLATFORMTYPE,
(uint)platformType.Capacity, platformType, 0) == 0)
throw new Exception("Error getting platform type.");
return platformType.ToString();
}
}
internal partial class PlatformDetection
{
public static bool IsSmartphone()
{
return PInvoke.GetPlatformType() == "SmartPhone";
}
public static bool IsPocketPC()
{
return PInvoke.GetPlatformType() == "PocketPC";
}
}
class PlatformProgram
{
static void Main(string[] args)
{
string platform;
if (PlatformDetection.IsSmartphone())
platform = "Smartphone";
else if (PlatformDetection.IsPocketPC())
platform = "Pocket PC";
else
platform = "Other WinCE";
MessageBox.Show("Platform: " + platform);
}
}

}

This is the second post in a series of three on platform detection.  Coming up next: detecting the presence of a touch screen.

Monday, September 18, 2006

availability-management.com ripping off my blog

availability-management.com has been ripping off my blog and posting it as their own.  No credit is given (a tiny link to the original).  They are using it to boost their search results to a page of theirs that is nothing more than advertising to various pages.  They don't leave any contact info for themselves so that I contact them either.

They seem to be affiliated or sponsored by http://theartofservice.com.  Whatever.  Service?  More like... ripping off other people's copyrighted works in order to promote their own advertising.

So far I've found five entire blog posts that they host as their own.  Grrr.....

http://availability-management.com/how-i-find-performance-bottlenecks-in-my-aspnet-projects.htm
http://availability-management.com/microsoft-2005-launch-events-in-orem-and-slc-areas.htm
http://availability-management.com/enabling-log4net-in-net-20.htm
http://availability-management.com/betas-2005-unleashed.htm
http://availability-management.com/vp-of-ucnug.htm

I'm not making these hyperlinks to avoid boosting their page ranking.

Friday, September 15, 2006

Platform detection I: How to detect that your app is running in the emulator

When you develop your Windows CE or Windows Mobile application in .NET Compact Framework, you probably do a lot of testing on the Microsoft Device Emulators for Smartphone and Pocket PC.  Here I describe how to detect whether your program is running on an emulator or a physical device.

Microsoft's Device Emulator gives itself away through a WinCE API called SystemParametersInfo when you pass in the argument SPI_GETOEMINFO.  We'll use this to check for the emulator.  When we detect something other than the Microsoft value, we must be running on a physical device. 

I use partial classes because in later posts in the Platform Detection series I'll add more to these classes.

using System;
using System.IO;
using System.Windows.Forms;
using Microsoft.Win32;
using System.Runtime.InteropServices;
using System.Text;

namespace PlatformDetection
{
internal partial class PInvoke
{
[DllImport("Coredll.dll", EntryPoint = "SystemParametersInfoW", CharSet = CharSet.Unicode)]
static extern int SystemParametersInfo4Strings(uint uiAction, uint uiParam, StringBuilder pvParam, uint fWinIni);

public enum SystemParametersInfoActions : uint
{
SPI_GETPLATFORMTYPE = 257, // this is used elsewhere for Smartphone/PocketPC detection
SPI_GETOEMINFO = 258,
}

public static string GetOemInfo()
{
StringBuilder oemInfo = new StringBuilder(50);
if (SystemParametersInfo4Strings((uint)SystemParametersInfoActions.SPI_GETOEMINFO,
(uint)oemInfo.Capacity, oemInfo, 0) == 0)
throw new Exception("Error getting OEM info.");
return oemInfo.ToString();
}

}
internal partial class PlatformDetection
{
private const string MicrosoftEmulatorOemValue = "Microsoft DeviceEmulator";
public static bool IsEmulator()
{
return PInvoke.GetOemInfo() == MicrosoftEmulatorOemValue;
}
}
class EmulatorProgram
{
static void Main(string[] args)
{
MessageBox.Show("Emulator: " + (PlatformDetection.IsEmulator() ? "Yes" : "No"));
}
}
}

This is the first post in a series of three on platform detection.  Coming up next: discerning between Smartphones and Pocket PCs.

Windows and Linux

This article is intended to be a store for some fair comparisons between the Windows and Linux operating systems.  It's not a Windows versus Linux document.  Comments are allowed, but if they contribute to or alter the facts in this article they may be removed after their suggestions have been applied to the body of the document. 

In the spirit of avoiding flamebait, I will write the facts (as I see them) and not my take on which approach is better.  I certainly have my opinions, but I want this to be more useful to others than just a rant.

Security

I won't cover individual security exploits here.  Instead I will cover common practices and security design and features of the platforms.

File system

Windows has one primary file system: NTFS.  Linux has many, but the security capabilities of the popular ones (ext2, ext3, reiserfs) are all the same (to my knowledge).  So this discussion will use ext2 wherever a specific file system needs mentioning, but I believe it applies to all Linux file systems.

Linux

Linux file system security allows for three individual permissions that can be set on a file or folder: read, write and execute.  Every file has an owner and an owning group.  The three permissions (which are abbreviated rwx) can be set individually for the owner, the owning group, and everyone else (called other).  In this way the owner can be given full control over a file, a group of individuals can have a different set of permissions, and everyone else still another. 

One of the quirks of this security model is that although a user may not have any permissions at all over a file, if a user has write permissions to parent directory he/she can still delete the file.  You also cannot give a user modify permission to a file while denying permission to delete the file.  Since execute is one of the permissions, security and linux' concept of an executable program become blurred.  Any file can be marked as executable by changing the permission set on it.  Thus granting "full control" to someone for a whole directory of files is a bad idea, as you mark them all as executables even if they are data files.

Recently support for ACLs have been added to the linux kernel to allow rwx permissions to be set for any combination of users and groups for any file or folder.  While you are still limited to the simple read, write, execute permissions, being able to allow some individual to read your file without allowing others to also is a very useful feature.  This feature isn't very visible in bash, and to use them the feature must be turned on and utilities to assist in setting the ACLs installed.

An NTFS partition mounted in Linux has only a single owning user and owning group, and as such only a single set of read/write/execute permissions can be applied to the entire drive.  The ACLs set on the partition from within Windows are ignored.

Windows

NTFS supports a very rich set of permissions for file and folder access: full control, traverse folder / execute file, list folder / read data, read attributes, read extended attributes, create files / write data, create folders / append data, write attributes, write extended attributes, delete, read permissions, change permissions, take ownership.  That's the advanced set.  The more common set are: full control, modify, read & execute, read, write, and special permissions. 

These permissions can be applied for individual users and groups, and as many of them as you need to share the file with exactly who you need to.  Uniquely, NTFS offers the "deny" setting for any of these permissions, allowing you to (for example) allow Everyone to Read your file except the user "Denny" with only two ACLs.

With the proper driver, a Windows computer can mount a linux partition.  Within Windows, all permissions on the linux partition are ignored and everything is freely accessible. 

User accounts

Windows

Windows is configured to encourage different user accounts for each administrator.  Although Windows comes preconfigured with a single Administrator account, it is easy, encouraged, and even automated (for XP) to create an individual administrator account for each person who needs one.  This helps ensure that system logs can accurately identify which admin did some action to the system.

Linux

In every Linux system I have encountered, it was normal and accepted that every admin knew the password to root.  Administrators would (hopefully) log in as regular users for most of the time, and then sudo just for those tasks that require admin privileges.  The way root is shared across the whole team seems to neutralize the benefit of log files that can identify who did what. 

Now I am looking for Linux user feedback here: I've been told that logs track who sudo's (from what account it was invoked), so that even though everyone does it, their actions are individually tracked.  I'm curious if this is true.  Also, if I'm a member of the wheel group, does that just mean I can sudo using the root password, or my own?

Protection for and from administrators

Linux

Once an admin logs in and sudos to root, some Linux distributions print a statement similar to "be good, respect privacy".  That simple request is the only thing between a root user on a *nix system and every file of any user's on that computer.  As root, no file system permission stands between you and reading, writing or deleting a file.

Linux encourages users to log in with normal, unprivileged accounts and sudo when necessary to install or run applications that require superuser privileges.  This lends to a secure system because admins are less likely to accidentally make system changes that affect other users or infect the whole system with a virus when they are logged in as a normal user.  Recently desktop managers make sudoing transparent by automatically popping up a password box when starting up system configuration applications. 

In addition, most Linux applications are designed to run with standard user privileges.  As a powerful example, User-mode Linux allows an ordinary user to emulate a self-contained Linux system (complete with internal admin privileges) without the overhead of a VM.

Windows

Administrators in Windows can't just read all files on the hard drive as root can on Linux.  If the NTFS permissions do not allow for Administrators to access files, an administrator must Take Ownership of the file (and reset permissions in the process) in order to access the file in any way.  The brilliance behind this is that while taking ownership of a file is possible for an administrator, giving ownership is not.  This means that while the admin may be able to read my files without my permission, I will definitely be able to find out by noticing that I am no longer the owner of my own files.  Besides the logs that can be kept for admin privileged actions like reading privileged files, this is a sure way that any user can be alerted to the prying eyes of administrators.

All the way through Windows XP and Windows Server 2003, almost everyone needs an administrator account on Windows to be able to get things done.  Whether that is installing applications or even running applications.  Running apps really shouldn't require admin privileges for most cases, and the fact that many do is the fault of the app developer.  The downside to Windows is that since it is a cold hard fact that these admin-requiring apps exist, everyone logs in as administrator all the time.  This leaves them open to virus attacks that can infect the whole system. 

Windows Vista changes this by introducing User Account Control, allowing anyone to log in with their administrator account, but you only have admin privileges when you need them and you explicitly approve each action.  Vista is not released yet, but early responses seem to agree that the popup boxes can be annoying.  It's all for the best though, right? :)

Patches

Some may point out that Windows has fewer or more patches than Linux, or that Linux delivers them faster, or that one's security exploits are more critical than others... yada yada.  Again, the point of this article is to describe design (read: potential) of each operating system.  Every complex system will have vulnerabilities.  And each will have seasons when a rash of exploits will be discovered.  So please don't post your comments on your opinions of which OS is more secure based on the individual patches it needs.

Windows

Windows is patched via the Microsoft Update site, which can patch both Windows and Office, and perhaps other Microsoft apps in the future.  These are made available on a fixed schedule (second Tuesday of each month), except for the occasional update that Microsoft deems critical enough to release in between these days.  Unless you are visiting Microsoft Update on that Tuesday, your copy of Windows will likely discover the update is available some days after it becomes available (probably to ease the download toll on Microsoft's servers).

Other apps can choose to implement their own automatic patch mechanisms.  Microsoft's .NET platform includes some great tools for software installation and patching, if your apps happen to use that platform and those features.  Most apps don't, however.

Linux

The linux patch mechanism varies with the distribution, but all the distros I've tried (Debian, Ubuntu, SuSE, Red Hat, Gentoo) have package managers that connect to some open source repository maintained by volunteers to get updates for just about any open source software you may be running on your computer.  This works as long as you used that package manager to download the software in the first place. 

These updates can vary from stale to bleeding edge depending on what server you choose to point at and the distro.  In my experience, for example, Gentoo tends to get new releases of software within a day or two of its release, whereas the others tend to be weeks or even months later.  There are system stability tradeoffs with being the first to accept updates, of course.

Administration

Background processes

Both Windows and Linux offers two ways for your application to be running in the background.  For an application with a UI, you can minimize the app (including to a small tray area icon).  This section will focus on the second kind--the one without an UI.  In Windows we call them services.  In Linux we call them daemons.

Windows Services

Windows services are written against a specific API and follow strict rules regarding UI, logging, handling system events (shutdown, restart, etc.) and security.  As a result, all services show up in a single management console where an admin can start, stop, pause, disable, and otherwise adjust each service. 

Linux Daemons

Linux daemons are just ordinary linux processes that happen to run in the background without a UI, and usually have a controlling script available in /etc/init.d (or wherever your particular distro chooses to put them).  Recently the desktop managers have been helping to consolidate management of these daemons by offering a tool that lists those services that play by their rules (or that the desktop manager is specifically written to handle).  Some services may not be listed though.

Configuration

Configuration can vary for any app on either operating system.  But each operating system tends to attract app developers who prefer a specific style for allowing their applications to be configured.  So this section necessarily generalizes the topic to overall feeling and common everyday experience.

Windows

At its core, Windows uses configuration files (mostly XML files now) and the Registry to control the operation of the OS and its applications.  Most apps leverage the registry's capability to store settings per machine and per user, and as a result one can predict where and in what format an applications settings will be fairly accurately.  But administrators and normal users will usually use the more user-friendly GUI interfaces that offer most or all of the same flexibility (as directly editing the registry) but at a faster pace. 

Windows has a reputation for needing to restart the whole OS for fairly common (though not regular routine) configuration changes.  On the other hand, most of your day-to-day configuration changes won't even require a process restart if you use the GUI to make the change.

Linux

Linux has a much more freestyle mode for configuration files.  User-specific settings are usually found in files starting with a period in the user's home directory, or in subdirectories that start with a period.  The starting period is a hint to command shells that these files should be "hidden" unless specifically asked for.  System-wide settings are usually found within the /etc directory.  Regardless of where they are found or whose settings they store, configuration file formats vary widely.  XML formats are not very common.  Plain text files with [headings] and key=value lines are very common.  Then you have the XML-text hybrid that (for example) Apache uses.  These files can be touchy, and a misplaced character can break parsing the entire file and keep your process from running at all until you fix the problem.

Changing configuration files in linux almost always requires restarting whatever process reads that configuration file in order for the changes to take effect.  Depending on the configuration file you change, you may have to restart a core OS process.  If you know the right commands, you can restart virtually everything except the kernel itself without restarting the computer.  This helps minimize downtime.

System and application logs

Linux

Linux applications log to text files.  Some of these logs include escape codes that allow these plain text files to colorize their text for easier human searching.  They are usually readable while the logged application is still running, and can even be deleted at runtime as well for some applications.  Cron scripts can be set up to automatically rotate and archive log files to keep them small, or even alert an administrator if interesting activity is logged.

I don't know anything about configuring security logging.  If anyone can educate me, I'll fill this paragraph in with something more interesting.

Windows

While any program can generate a log file anywhere, Windows does encourage app developers to leverage their Windows Event Log system.  These logs are centralized, can be protected, rotated, archived, backed up, programmatically parsed, and perused while the logged application is still running.  Any number of logs and applications can be set up with their own space within the Windows Event Log.   

Security logging is configurable all from one location, and allows a very high degree of control over what gets logged and what doesn't.

Tuesday, August 01, 2006

SystemParametersInfo: buffer size in unicode characters or bytes?

The SystemParametersInfo function in Windows CE can be tricky, even deadly, when it comes to passing in buffers for SPI_GETOEMINFO or SPI_GETPLATFORMTYPE.  Each of these uiAction values fill a given buffer with a string.  The uiParam argument takes the size of that buffer.  The catch is, for SPI_GETOEMINFO, uiParam takes a size of buffer in bytes and for SPI_GETPLATFORMTYPE uiParam takes the size of the buffer in count of unicode characters.



The documentation on SPI_GETOEMINFO is not precise on which type of buffer size is requested, and if you pass the wrong one to SPI_GETPLATFORMTYPE, you could find yourself in a buffer overrun situation.  Be careful.  The code below is safe:



#include "stdafx.h"

#define _countof(x) (sizeof(x) / sizeof(x[0]))

int _tmain(int argc, _TCHAR* argv[])
{
/**********************
* Get OEM information
**********************/

WCHAR wszOemInfo[50];
UINT cbOemInfo = sizeof(wszOemInfo);

if (SystemParametersInfoW(SPI_GETOEMINFO, cbOemInfo, wszOemInfo, 0))
MessageBoxW(NULL, wszOemInfo, L"OEMINFO", MB_OK);
else if (GetLastError() == ERROR_INSUFFICIENT_BUFFER)
MessageBoxW(NULL, L"Insufficient buffer", L"OEMINFO", MB_OK);
else
MessageBoxW(NULL, L"Some other error.", L"OEMINFO", MB_OK);


/**********************
* Get platform type
**********************/

WCHAR wszPlatformType[50];
UINT cchPlatformType = _countof(wszPlatformType);
if (SystemParametersInfoW(SPI_GETPLATFORMTYPE, cchPlatformType, wszPlatformType, 0))
MessageBoxW(NULL, wszPlatformType, L"Platform type", MB_OK);
else if (GetLastError() == ERROR_INSUFFICIENT_BUFFER)
MessageBoxW(NULL, L"Insufficient buffer", L"Platform type", MB_OK);
else
MessageBoxW(NULL, L"Some other error.", L"Platform type", MB_OK);

return 0;
}

Saturday, July 29, 2006

Want a Writely invite?

It turns out that although you can't directly sign up for Writely during this transition to Google, existing Writely accounts can invite others to Writely to collaborate on existing documents.  Once in, a new collaborator has a full-fledged Writely account.

A generous donor just gave me the Writely invite I have coveted for these months.  Woo hoo!  Actually, I find Writely quite comparable to ZohoWriter, another free online word processor which freely allows new registration.  ZohoWriter is excellent. 

But so is Writely, and I assume you're here for one of the Writely invites I can now give away.  Post a comment to this post and I'll send you a Writely invite.  After you get the invite and click on it, you will choose a password for yourself.  If after that you get an error saying you can't view the document, don't worry.  Just go to the Writely home page and you'll see that you have a Writely account.

I imagine I'll tire of checking the comments of this blog eventually, so at some point I'll close off comments to this post.  Please don't email me directly for these.

Update: I'm closing comments, as Writely has just opened up free registrations.

Saturday, July 08, 2006

How to empty a SQL Server database transaction log file

I have 1GB databases that grow several GB log files.  How do you safely clear those logs?  Well, probably the best practice includes backing them up.  That was overkill for me, and sifting through all the documentation and forums (eventually) led me to this simple pair of statements that do the trick immediately:

backup log [dbname] with truncate_only
go
DBCC SHRINKDATABASE ([dbname], 10, TRUNCATEONLY)
go

Update: Mathias added a comment that deserves calling out:

Warning: Clearing the transaction log with TRUNCATE_ONLY will break your backup chain needed for FULL recovery. A FULL database backup should be executes right after truncating the logs. More info.

Saturday, June 03, 2006

How I got OpenID working with Ruby on Rails

After spending two hours searching for good documentation for using OpenID as my authentication for my Ruby on Rails app and being disappointed in lack of examples and other documentation, I decided to figure it out myself.  I did, with help from OpenID's author Brian.  Here is what I found. I did all this in a brand new Rails app rather than my pride and joy one that I've been working on so I could get all the kinks out. You may want to do the same your first time.

First I downloaded the Ruby port of OpenID:

$> gem install ruby-openid

At the time of writing, Ruby OpenID 1.0.1 is the current version. The documentation suggests that the next step is to run this command from your rails app:

$> ./script/generate openid_login Account

This step did not work for me.  I got the error: "Couldn't find 'openid_login' generator".  I am still fairly new to Rails so I could not figure out why installing the ruby-openid gem did not automatically register it as a Rails generator.  Other gems seem to automatically hook themselves up during install.  Ah well.  Brian later told me that the gem does not automatically register the generator, but there will be a dedicated gem later on just for the generator.

Upon exploring the install directory for the gem (in my case under /usr/lib/ruby/gems/1.8/gems/ruby-openid-1.0.1), I read the INSTALL file, which suggested that if properly installed, openid should be accessible through

 $> irb
irb(main):001:0> require "openid/consumer"
=> true

That didn't work either.  False came back.  Now I had a place to start.  I found that if I put myself in the lib directory of ruby-openid-1.0.1 that this worked.  Also, I discovered that this worked:

 $> irb
irb(main):001:0> require "rubygems"
=> true
irb(main):002:0> require_gem "ruby-openid"
=> true
irb(main):003:0> OpenID::Consumer
=> OpenID::Consumer

I discovered that downloading the .tar.gz file included several more files (and directories!) than the gem did.  So I downloaded that and tried to run the standalone example consumer.rb file, which is supposed to validate logins.  It kept failing.  It wouldn't work at all on my Linux box.  On my Windows box, I got a "Bad auth key - wrong length" error.

I emailed Brian (an author of the library) and asked for help.  He gave me the next few tips...

First, I needed to read the /usr/lib/ruby/gems/1.8/gems/ruby-openid-1.0.1/examples/README file.  It had additional information on setting up the Rails generator that the top level README file did not.  Specifically, I needed to do this:

$> mkdir -p ~/.rails/generators
$> ln -s /usr/lib/ruby/gems/1.8/gems/ruby-openid-1.0.1/examples/rails_openid_login_generator ~/.rails/generators/openid_login

This made Rails finally recognize the openid_login as a Rails generator. Yay!

$> ./script/generate openid_login account

So now this worked. And now I had a new README_LOGIN file in the root of my Rails app. Reading that for guidance, I needed to create my users table now. The README_LOGIN file had various database syntax listed, but I chose to use ActiveRecord migrations:

$> ./script/generate migration create_users_table
$> vi db/migrate/001_create_users_table.rb

And I filled in this content to the file, which is the minimum ruby-openid requires:

class CreateUsersTable < ActiveRecord::Migration
def self.up
create_table :users do |t|
t.column :openid_url, :string
end
end

def self.down
drop_table :users
end
end

And executed the migration, having already created my database.

$> rake migrate

I also added those lines to app/controllers/application.rb that the README_LOGIN file prescribed:

require_dependency 'openid_login_system'

# Filters added to this controller will be run for all controllers in the application.
# Likewise, all the methods added will be available for all controllers.
class ApplicationController < ActionController::Base
include OpenidLoginSystem
end

I found that the last line the README_LOGIN file suggests to add (model :user) is just that--a suggestion if you run into trouble later on. I left this line out. I'd like to learn it firsthand when this line becomes necessary.

So that was it. I launched my server:

$> ./script/server

And tested my app from across the Internet by visiting http://myhost.mydomain.com:3000/account/login. It worked! I was able to login with my OpenID, get redirected to my OpenID provider to authorize the login, and then the site recognized me as logged in.  I don't know if hosting from localhost in this example would work, with all the redirections between your server and an OpenID provider.  Maybe it would. I didn't test it.

One gotcha in case you run into it in your testing/exploring: the /account/welcome action that the openid_login generator creates does not actually require the client to be logged in. So although /account/welcome comes up by itself after login announcing you are now logged in, if you had not logged in but just navigated directly to /account/welcome yourself, the system would still tell you that you were logged in. This is not a system malfunction as much as incorrect documentation. I fixed it by changing the app/views/account/welcome.rhtml file to this:

<h3>Welcome</h3>

<% if @session[:user_id] %>
	<p>You are now logged into the system...</p>
	<%= link_to "« logout", :action=>"logout" %>
<% else %>
	<p>You are NOT logged into the system...</p>
<% end %>

As a last finishing touch, the login page textbox field lacked that professional-looking I-swirly picture on its left that immediately gives the OpenID recognition.  I downloaded it from https://www.myopenid.com/static/openid-icon-small.gif and added it to my /public/images folder. Then I modified app/views/layouts/scaffold.rhtml by adding the following to the inside of the tag:

<style>
input.openid
{
background: url(/images/openid_login.gif) no-repeat;
background-position: 0 50%;
padding-left: 18px;
border-style: solid;
border-color: lightgray;
border-width: 1px;
}
</style>

And I modified app/views/account/login.rhtml to include a class="openid" attribute for its login input box.

Now I feel I'm ready to integrate this into my actual Rails app. I hope this extra example serves you well. Comments are always welcome. If you ask a question about the process, I'll try my best to answer, but remember I'm not a Rails expert. If you feel you can improve on my methods, please share!

Tuesday, May 16, 2006

Initial and abstract impressions for my first week at Microsoft

So I’ve finished my first week here at Microsoft working with the .NET Compact Framework team.  They are a very friendly group, and I am happy to be working with them. I daresay some exciting technologies are in the works that I am excited to be contributing to (soon). Microsoft’s internal development tools require some getting used to, though.

I already knew that Microsoft grades employees on some kind of a curve. I was then and I still am not looking forward to seeing how that process works in detail. I personally wish we could all be graded independently of each other. It seems we would be more of a all-for-one and one-for-all team if we could help each other without wondering how it may impact our relative rating.

What I did not know was that although most employees (from what I hear) get their own offices, seniority in time served here at Microsoft dictates whether you actually get your own office, whether it will have a window, how many monitors you have, etc. On the other hand, I have not felt any hazing at all. The people here seem to be rooting for everyone’s successes. What they say about the culture here at Microsoft, that employees are accountable to their customers and to each other, is very true. Every time I ask a coworker a question, that employee seems to take ownership of the responsibility to see that I understand the answer, or helps direct me to someone who can better answer me.

The relocation experience was good, overall. There were some hiccups, and the biggest area for improvement would be to add a web page where my recruiter and everyone involved in the relocation process could provide me with a centralized place to read up on status updates and work items for me in preparation for the relocation. Managing the dozens of emails that went back and forth to coordinate it, and trying to remember who to contact for which problems was a real challenge.

It took two hours during new employee orientation to explain all the benefits that Microsoft offers to its employees. I can see why it is said that some people work here just for the benefits. Absolutely amazing!

Sunday, March 05, 2006

WSE3 and ASP.NET Membership Provider authentication

I am not an expert in WSE3 by anyone's definition.  So when I went looking for a way to add authentication requirements to my web service that tied into the ASP.NET 2.0 Membership Provider model class that I had already written for my web site, I was sorely disappointed when I could not find any articles on that exact topic.  One very good MSDN article came close, but just like all the articles I saw, it seemed to focus more on using Active Directory authentication, or heavy message-level encryption, or some other advanced feature I did not need or want.

My primary focus was to write a web service that required authentication while preserving the ability to serve to PHP and Java clients that have not yet been graced by WSE3.  I would use SSL for encryption to simplify the process.  But since no one covered the topic of what I would do, I finally managed to adapt the article mentioned above to what I was doing and I share what I have learned with you.  I still have not managed to get a PHP web service client to call my WSE3 web service though.

It seems like a very common situation to want to have your web services simple to access, yet require authentication through the same mechanism that your ASP.NET Forms Authentication uses.  So I hope my findings will help you.
  1. Visual Studio 2005 should already be installed, and not running.
  2. Install the WSE 3 extensions for Visual Studio.  During the install, be sure to install all the development tools.
  3. Open your Visual Studio 2005 solution file.
  4. Configure your project WSE Settings:
    1. Right-click on your web project, and click WSE Settings 3.0.  If this is your first time clicking this, a wizard will start.  The rest of these steps assume you've been through the wizard, and that the regular settings box appears.  If you get the wizard, try to figure out how to apply these next steps to the wizard, and then come back to the settings box afterward to make sure all is well.  Sorry for the confusion here.
    2. Under the General tab, check both the Enable this project for Web Services Enhancements and Enable Microsoft Web Services Enhancement Soap Protocol Factory.
    3. Under the Security tab, in the Security Tokens Managers area, click Add.
      1. Fill in the fields as follows:
      1. Type: "CustomUsernameTokenManager, __code"
      2. Namespace: "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
      3. LocalNode: "UsernameToken"
      4. Click OK.
    4. Under the Policy tab, check Enable Policy, and click Add.
      1. Name your new application policy "usernameTokenSecurity".
    5. Click OK to the WSE Settings dialog box.  This will add a new file called wse3policyCache.config to your web project.
  5. Customize the contents of your new wse3policyCache.config file.
    1. It should start looking like this:
      <policies xmlns="http://schemas.microsoft.com/wse/2005/06/policy">
      <extensions>
      <extension name="usernameOverTransportSecurity" type="Microsoft.Web.Services3.Design.UsernameOverTransportAssertion, Microsoft.Web.Services3, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
      <extension name="requireActionHeader" type="Microsoft.Web.Services3.Design.RequireActionHeaderAssertion, Microsoft.Web.Services3, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
      </< span>extensions>
      <policy name="usernameTokenSecurity">
      <usernameOverTransportSecurity />
      <requireActionHeader />
      </< span>policy>
      </< span>policies>
    2. Add an section if you would like to require your web service consumers to belong to a specific role or roles.  It is very important that you put the tag above your tag.  When you're done, it may look something like this:
      <policies xmlns="http://schemas.microsoft.com/wse/2005/06/policy">
      <extensions>
      <extension name="usernameOverTransportSecurity" type="Microsoft.Web.Services3.Design.UsernameOverTransportAssertion, Microsoft.Web.Services3, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
      <extension name="requireActionHeader" type="Microsoft.Web.Services3.Design.RequireActionHeaderAssertion, Microsoft.Web.Services3, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
      </< span>extensions>
      <policy name="usernameTokenSecurity">
      <authorization>
      <allow role="webserviceprofessional"/>
      <deny role="*"/>
      </< span>authorization>
      <usernameOverTransportSecurity />
      <requireActionHeader />
      </< span>policy>
      </< span>policies>
  6. Create your CustomUsernameTokenManager class in your web project.
    1. In your web project's App_Code directory, create a CustomUsernameTokenManager.cs file.
    2. Copy and paste this code into your CustomUsernameTokenManager.cs file:
      using System;
      using System.Xml;
      using System.Web.Security;
      using System.Security.Permissions;
      using System.Security.Principal;

      using Microsoft.Web.Services3.Security;
      using Microsoft.Web.Services3.Security.Tokens;

      ///
      /// By implementing UsernameTokenManager we can verify the signature
      /// on messages received.
      ///

      [SecurityPermissionAttribute(SecurityAction.Demand, Flags = SecurityPermissionFlag.UnmanagedCode)]
      public class CustomUsernameTokenManager : UsernameTokenManager
      {
      ///
      /// Constructs an instance of this security token manager.
      ///

      public CustomUsernameTokenManager()
      {
      }

      ///
      /// Constructs an instance of this security token manager.
      ///

      /// <param name="nodes" />An XmlNodeList containing XML elements from a configuration file.</param>
      public CustomUsernameTokenManager(XmlNodeList nodes)
      : base(nodes)
      {
      }

      ///
      /// Returns the password or password equivalent for the username provided.
      /// Adds a principal to the token with user's roles.
      ///

      /// <param name="token" />The username token</param>
      /// The password (or password equivalent) for the username
      protected override string AuthenticateToken(UsernameToken token)
      {
      bool validCredentials = Membership.ValidateUser(token.Username, token.Password);
      if (!validCredentials) throw new UnauthorizedAccessException();

      GenericIdentity identity = new GenericIdentity(token.Username);
      GenericPrincipal principal = new GenericPrincipal(identity, Roles.GetRolesForUser(token.Username));
      token.Principal = principal;

      return token.Password;
      }
      }
  7. As a reminder, I am assuming you already have ASP.NET 2.0 Membership and Roles providers set up in your web site and included in your Web.config file.  If not, you might as well stop here and get that working first.  And an explanation of how to do so is beyond the scope of this particular blog.
  8. Make sure that the following segments appear in your Web.config file.  Pay attention to detail.  I'm pretty sure one of the problems I struggled with was that just one of these lines were not automatically put in by the WSE Settings dialog, and this only works if it's all there.
    <configuration xmlns="http://schemas.microsoft.com/.NetConfiguration/v2.0">
    <configSections>

    <section name="microsoft.web.services3" type="Microsoft.Web.Services3.Configuration.WebServicesConfiguration, Microsoft.Web.Services3, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
    </< span>configSections>

    <system.web>
    <compilation debug="true">
    <assemblies>

    <add assembly="Microsoft.Web.Services3, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
    </< span>assemblies>
    </< span>compilation>

    <webServices>
    <soapExtensionImporterTypes>
    <add type="Microsoft.Web.Services3.Description.WseExtensionImporter, Microsoft.Web.Services3, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
    </< span>soapExtensionImporterTypes>
    <soapServerProtocolFactory type="Microsoft.Web.Services3.WseProtocolFactory, Microsoft.Web.Services3, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
    </< span>webServices>
    </< span>system.web>

    <microsoft.web.services3>
    <policy fileName="wse3policyCache.config" />
    <security>
    <securityTokenManager>
    <add type="CustomUsernameTokenManager, __code" namespace="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" localName="UsernameToken" />
    </< span>securityTokenManager>
    </< span>security>
    </< span>microsoft.web.services3>
    </< span>configuration>
  9. Now open up each of your web service classes (either your .asmx files or associated code-behind files) and add the following class attribute to your web service class:
    [Microsoft.Web.Services3.Policy("usernameTokenSecurity")]
  10. You are now done configuring you web service to authenticate each web service request via a WSE 3 SOAP header, relying only on encryption provided by your transport (HTTPS, for example).
Now to configure a client to test your new web service's authentication feature:
  1. Configure your client project for WSE.
    1. Right-click on your client project in Visual Studio and click WSE Settings 3.0.  If this is your first time clicking this, a wizard will start.  The rest of these steps assume you've been through the wizard, and that the regular settings box appears.  If you get the wizard, try to figure out how to apply these next steps to the wizard, and then come back to the settings box afterward to make sure all is well.  Sorry for the confusion here.
    2. Under the General tab, check "Enable this project for Web Service Enhancements".
    3. Under the Policy tab, check "Enable Policy" and add a policy called "usernameTokenSecurity".
    4. The wse3policyCache.config file should look like this:
  2. Add a couple of using statements to the file that you will be scripting your client in:
    using Microsoft.Web.Services3.Security;
    using Microsoft.Web.Services3.Security.Tokens;
  3. And if your WSE-enabled web service was called RIRegistration, you would use the following code to call its ValidateLogin method:
    RIRegistrationWse wse = new RIRegistrationWse();
    UsernameToken token = new UsernameToken(username, password, PasswordOption.SendPlainText);
    wse.SetClientCredential(token);
    wse.SetPolicy("usernameTokenSecurity");
    // Now you can call methods repeatedly, and the authentication
    // is automatically passed to the server each time.
    Debug.Assert(user.Username, wse.ValidateLogin());
  4. Notice how we call ValidateLogin without passing any parameters, yet it returns the username of the user you are logging in with. Let's see how this ValidateLogin method might be implemented:
    [WebMethod(Description = "SOAP header test.  If successful, it will return the username of the logged in user.")]
    public string ValidateLogin()
    {
    return Microsoft.Web.Services3.RequestSoapContext.Current.IdentityToken.Identity.Name;
    }
  5. Run your web client.  You should get the username you passed to authenticate back from the web method. 
  6. Try changing to an invalid credentials.  Your ASP.NET Membership provider should reject the login and an exception will be thrown to the client before your web method is ever called.
  7. If you added an section to your wse3policyCache.config file, try changing to credentials that are valid but do not belong to a required role.  Verify that your web service likewise rejects the invocation.
I hope this is helpful to you.  Feel free to ask questions, if your project is pretty close to the situation I describe.  I am still pretty new to this myself, so if your project deviates from this path much, I'm afraid I won't be able to help you.

Thursday, February 09, 2006

Why I'm giving up Ayende's NHibernate.Generics (for now)

Ayende's NHibernate.Generics is truly an incredible advance in supporting generic collections with NHibernate, and solving the two-way relationship problem at the same time.  I switched all my projects to use it because it made the code so much prettier than my own MyNHibernateContrib project. 

However, after running into several unforeseen issues, I've been forced to revert all my code back to MyNHibernateContrib for my largest projects.  The code may look a little uglier, but it works quite well.  Here are the issues I've had in trying Ayende's library.  He already knows about some of these and warns about them on this blog, but the extent to which they cause problems were greater than I anticipated based on his web page.
  1. As yet unpersisted entities added to a lazy and not yet loaded collection do not show up in the collection at all.  And cascade save from the collection to the entity never happens!  So my tens of thousands of lines of code written to take advantage of cascade save has bugs ridden through it suddenly, as entities are not saving all over the place.
  2. When I try to solve the the previous problem by disabling his extra lazy-loading feature for collections, I get lazy-loading exceptions thrown from NHibernate that I just haven't been able to solve.
  3. List collection persistence seems shaky (indexes within the list randomly changed).
  4. Map collection (Dictionary) persistence is non-existent.
  5. NHibernate collection cache doesn't match added contents, when an entity is added to an unloaded collection with something like apple.Tree = someTree;  someTree.Apples, when loaded in some later ISession, does not include the apple, even when reloaded.  The NH cache must be refreshed for it to ever show up.
I hope Ayende and/or the NHibernate team can fix these issues so I can get back to pretty code.  I applaud Ayende's work, and I am still using his library on my smaller projects. Keep up the good work, Ayende!  And if anyone (including myself) has some time to help with these bugs or implementation holes, by all means, please contribute to his project.

How to crash IE with just 4 levels of recursion in CSS and JavaScript

I can crash your Internet Explorer 6.0 (with all the latest patches) anytime you visit my page, with no warning at all.  Fun stuff.  I'm explaining how to do it here, not so that you can go crashing other people's pages, but so that I have a URL reference to give Microsoft for a bug report, and so that others who run into this bug accidentally (as I did) may have a page to find that explains exactly what is going wrong and how they can workaround it.  This isn't a security hole, as far as I can tell, so I don't think it's unsafe for me to post this.
DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" >
    <head>
        <link rel="stylesheet" type="text/css" href="1deep.css" />
        <script type="text/javascript" src="css_bug.js">script>
    head>
    <body>
        The CSS and JavaScript will soon crash this page!<br/>
        If you see a scripting warning in IE, let script run.  This 
        warning does NOT come up when this web page is accessed over 
        the Internet (just over your hard drive, ironically!)<br/>
        <input type="button" onclick="alert('CSS depth: ' + discoverCssDepth())" 
            value="Click me to calculate CSS depth (and crash)"/><br/>
        Sometimes IE won't crash immediately after clicking the button.
        You may have to click around, or even close IE, for the crash to occur.
    body>
html>
1deep.css
@import url(2deep.css);
2deep.css
@import url(3deep.css);
3deep.css
@import url(4deep.css);
4deep.css

css_bug.js
function discoverCssDepth(sheet) {
    var maxDepth = 0;
    if( !sheet ) { // if this is the outermost call
        var depth = 0;
        for( var i = 0; i < document.styleSheets.length; i++ ) {
            depth = discoverCssDepth( document.styleSheets[i] );
            if (depth > maxDepth) maxDepth = depth;
        }
        return maxDepth;
    } else {
//        alert('looking at ' + sheet.href);
        maxDepth = 0;
        var depth = 0;
        // Enumerate over subsheets, if any
        if( sheet.imports ) {
//            alert('Has ' + sheet.imports.length + ' sub css files.');
            for( var i = 0; i < sheet.imports.length; i++ ) {
                depth = discoverCssDepth( sheet.imports[i] );
                if (depth > maxDepth) maxDepth = depth;
            }
        } else if (sheet.cssRules) { // Mozilla style
            for( var i = 0; i < sheet.cssRules.length; i++ ) {
                if (sheet.cssRules[i].styleSheet) {
                    depth = discoverCssDepth( sheet.cssRules[i].styleSheet );
                    if (depth > maxDepth) maxDepth = depth;
                }
            }
        }
        return 1+maxDepth;
    }
}
Note that if you try this on your local hard drive, you'll see IE give an Active Scripting warning that you must approve before the code will run and the browser will crash. But over the Internet, the warning does not show up, and the code just executes!

Tuesday, February 07, 2006

Adobe SVG Viewer in .NET WinForms

I was working on a C# WinForms project where I had to embed the Adobe SVG Viewer in my form.  The only distribution available comes in the form of an ActiveX control.  The control becomes useful when you set its SRC property to some .svg file to render.  Unfortunately this must happen at design-time or else a ActiveXStateException is thrown:
System.AccessViolationException was unhandled
  Message="Attempted to read or write protected memory. This is often an indication that other memory is corrupt."
  Source="Interop.SVGACTIVEXLib"
  StackTrace:
       at SVGACTIVEXLib.ISVGControl.set_SRC(String pVal)
       at AxSVGACTIVEXLib.AxSVGCtl.set_SRC(String value)
The control also offers setSrc and getSrc methods, but these fail with the same exception.  More inspection led to the discovery that the control's SRC attribute can only be set after being freshly dropped onto the design surface.  What to do? (answer follows...)

In my case, I wanted to generate SVG files at run-time using temporary file storage space, which meant the filename would not be predictable at design time.  A lot of Googling turned up an Adobe support article revealed that only an ActiveX host that implements IHtmlDocument2 can set the SRC attribute for security reasons, starting at their SVG 3.03 version. 

Well, seeing as I didn't want to rewrite a web browser, I decided to re-use the .NET component for IE.  I would wrap the SVG Viewer ActiveX control into a web browser control, inside a User Control that would hide the ugly workaround.  It turns out that very little code was required.  The result is that my SVG Viewer can effectively display whatever SVG I want at run-time, with no apparent side effects.

I include the code below, licensed with X11:
SVGViewer.cs
/// 
/// A user control that wraps up Adobe's SVG Viewer in an
/// IE web browser to allow for changing the source file
/// at run-time.
///

///
/// This is necessary due to:
/// http://support.adobe.com/devsup/devsup.nsf/docs/54114.htm
/// Also references:
/// http://www.csharphelp.com/archives/archive146.html
/// http://ryanfarley.com/blog/archive/2004/12/23/1330.aspx
/// http://www.adobe.com/svg/viewer/install/svgtest.html
///
public partial class SvgViewer : UserControl
{
public SvgViewer()
{
InitializeComponent();
}

const string svgHtml = "<html><head><style>body {{ margin: 0; padding: 0; }} </style><embed src='{0}' width='100%' height='100%' type='text/html; charset=UTF-8' />";

private string source;
[Bindable(true)]
[Category("Appearance")]
[Description("The SVG file to display.")]
public string Source
{
get { return source; }
set
{
source = value;
UpdateHtml();
}
}

protected virtual void UpdateHtml()
{
if (webBrowser1.Document.Body != null)
webBrowser1.Document.OpenNew(true);
webBrowser1.Visible = !string.IsNullOrEmpty(source);
if (string.IsNullOrEmpty(source)) return;
webBrowser1.Document.Write(string.Format(svgHtml, source));
}
}
SVGViewer.Designer.cs
namespace Phylogenetics
{
partial class SvgViewer
{
///
/// Required designer variable.
///

private System.ComponentModel.IContainer components = null;

///
/// Clean up any resources being used.
///

/// <param name="disposing" />true if managed resources should be disposed; otherwise, false.
protected override void Dispose(bool disposing)
{
if (disposing && (components != null))
{
components.Dispose();
}
base.Dispose(disposing);
}

#region Component Designer generated code

///
/// Required method for Designer support - do not modify
/// the contents of this method with the code editor.
///

private void InitializeComponent()
{
this.webBrowser1 = new System.Windows.Forms.WebBrowser();
this.SuspendLayout();
//
// webBrowser1
//
this.webBrowser1.AllowNavigation = false;
this.webBrowser1.AllowWebBrowserDrop = false;
this.webBrowser1.Dock = System.Windows.Forms.DockStyle.Fill;
this.webBrowser1.IsWebBrowserContextMenuEnabled = false;
this.webBrowser1.Location = new System.Drawing.Point(0, 0);
this.webBrowser1.MinimumSize = new System.Drawing.Size(20, 20);
this.webBrowser1.Name = "webBrowser1";
this.webBrowser1.ScriptErrorsSuppressed = true;
this.webBrowser1.ScrollBarsEnabled = false;
this.webBrowser1.Size = new System.Drawing.Size(150, 150);
this.webBrowser1.TabIndex = 0;
this.webBrowser1.Url = new System.Uri("about:blank", System.UriKind.Absolute);
this.webBrowser1.WebBrowserShortcutsEnabled = false;
//
// SvgViewer
//
this.AutoScaleDimensions = new System.Drawing.SizeF(6F, 13F);
this.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font;
this.Controls.Add(this.webBrowser1);
this.Name = "SvgViewer";
this.ResumeLayout(false);

}

#endregion

private System.Windows.Forms.WebBrowser webBrowser1;

}
}

Sunday, January 29, 2006

Generating non-pixelated thumbnail images in .NET

The .NET Framework comes with a large set of image manipulation libraries.  Generating thumbnail images becomes very easy.  But image pixelation, if it were a feature, is turned on by default. :)  Here is the basic code to generate a thumbnail without pixelation.  It's inspired by TheLomex on an ASP.NET forum thread.  I have adapted it to C#.
Bitmap original; // your original image
Size resolution = new Size(150, 150); // size of your thumbnail
Image thumbnail = new Bitmap(original, resolution);
Graphics g = Graphics.FromImage(thumbnail);
// The InterpolationMode was the catalyst to eliminate pixelation.
g.InterpolationMode = InterpolationMode.HighQualityBicubic;
// I'm not sure whether CompositingQuality or SmoothingMode
// contribute at all to good image resizing. Anyone know?
g.CompositingQuality = CompositingQuality.HighQuality;
g.SmoothingMode = SmoothingMode.HighQuality;
g.DrawImage(original, new Rectangle(0, 0, thumbnail.Size.Width, thumbnail.Size.Height));
// Thumbnail now contains the resized image, all smoothed out and
// non-pixelated!
To save the image now as a JPEG while controlling the compression vs. quality of the saved file, add this code:
// Prepare for a controlled-quality JPEG export
ImageCodecInfo jpegCodec = GetEncoderInfo("image/jpeg");
Encoder jpegEncoder = Encoder.Quality;
EncoderParameters jpegEncoderParameters = new EncoderParameters(1);
EncoderParameter jpegEncoderQuality = new EncoderParameter(jpegEncoder, jpegQuality);
jpegEncoderParameters.Param[0] = jpegEncoderQuality;

string thumbnailPath; // some path to save your JPEG to
thumbnail.Save(thumbnailPath, jpegCodec, jpegEncoderParameters);
And never forget to call Image.Dispose on all your images when you're done using them, using either using or a finally block.

Friday, January 27, 2006

C# Dijkstra's algorithm implementation

I implemented Dijkstra's algorithm using C# for a Computer Science course. I implemented it in a generalized way, that still allows for optimization by the consuming code. I release the code under the MIT license

Here is the code: (it is not as long as it is well-documented)

using System; 
using System.Diagnostics;
using System.Collections.Generic;
using System.Text;

namespace VisualIntelligentScissors
{
/// <summary>
/// Implements a generalized Dijkstra's algorithm to calculate
/// both minimum distance and minimum path.
/// </summary>
/// <remarks>
/// For this algorithm, all nodes should be provided, and handled
/// in the delegate methods, including the start and finish nodes.
/// </remarks>
public class Dijkstra
{
/// <summary>
/// An optional delegate that can help optimize the algorithm
/// by showing it a subset of nodes to consider. Very useful
/// for limited connectivity graphs. (like pixels on a screen!)
/// </summary>
/// <param name="startingNode">
/// The node that is being traveled away FROM.
/// </param>
/// <returns>
/// An array of nodes that might be reached from the
/// <paramref name="startingNode"/>.
/// </returns>
public delegate IEnumerable<int> NearbyNodesHint(int startingNode);
/// <summary>
/// Determines the cost of moving from a given node to another given node.
/// </summary>
/// <param name="start">
/// The node being moved away from.
/// </param>
/// <param name="finish">
/// The node that may be moved to.
/// </param>
/// <returns>
/// The cost of the transition from <paramref name="start"/> to
/// <paramref name="finish"/>, or <see cref="Int32.MaxValue"/>
/// if the transition is impossible (i.e. there is no edge between
/// the two nodes).
/// </returns>
public delegate int InternodeTraversalCost(int start, int finish);

/// <summary>
/// Creates an instance of the <see cref="Dijkstra"/> class.
/// </summary>
/// <param name="totalNodeCount">
/// The total number of nodes in the graph.
/// </param>
/// <param name="traversalCost">
/// The delegate that can provide the cost of a transition between
/// any two nodes.
/// </param>
/// <param name="hint">
/// An optional delegate that can provide a small subset of nodes
/// that a given node may be connected to.
/// </param>
public Dijkstra(int totalNodeCount, InternodeTraversalCost traversalCost, NearbyNodesHint hint)
{
if (totalNodeCount < 3) throw new ArgumentOutOfRangeException("totalNodeCount", totalNodeCount, "Expected a minimum of 3.");
if (traversalCost == null) throw new ArgumentNullException("traversalCost");
Hint = hint;
TraversalCost = traversalCost;
TotalNodeCount = totalNodeCount;
}

protected readonly NearbyNodesHint Hint;
protected readonly InternodeTraversalCost TraversalCost;
protected readonly int TotalNodeCount;

/// <summary>
/// The composite product of a Dijkstra algorithm.
/// </summary>
public struct Results
{
/// <summary>
/// Prepares a Dijkstra results package.
/// </summary>
/// <param name="minimumPath">
/// The minimum path array, where each array element index corresponds
/// to a node designation, and the array element value is a pointer to
/// the node that should be used to travel to this one.
/// </param>
/// <param name="minimumDistance">
/// The minimum distance from the starting node to the given node.
/// </param>
public Results(int[] minimumPath, int[] minimumDistance)
{
MinimumDistance = minimumDistance;
MinimumPath = minimumPath;
}

/// <summary>
/// The minimum path array, where each array element index corresponds
/// to a node designation, and the array element value is a pointer to
/// the node that should be used to travel to this one.
/// </summary>
public readonly int[] MinimumPath;
/// <summary>
/// The minimum distance from the starting node to the given node.
/// </summary>
public readonly int[] MinimumDistance;
}

/// <summary>
/// Performs the Dijkstra algorithm on the data provided when the
/// <see cref="Dijkstra"/> object was instantiated.
/// </summary>
/// <param name="start">
/// The node to use as a starting location.
/// </param>
/// <returns>
/// A struct containing both the minimum distance and minimum path
/// to every node from the given <paramref name="start"/> node.
/// </returns>
public virtual Results Perform(int start)
{
// Initialize the distance to every node from the starting node.
int[] d = GetStartingTraversalCost(start);
// Initialize best path to every node as from the starting node.
int[] p = GetStartingBestPath(start);
ICollection<int> c = GetChoices();

c.Remove(start); // take starting node out of the list of choices

//Debug.WriteLine("Step v C D P");
//Debug.WriteLine(string.Format("init - {{{0}}} [{1}] [{2}]",
// ArrayToString<int>(",", c), ArrayToString<int>(",", d), ArrayToString<int>(",", p)));
//int step = 0;

// begin greedy loop
while (c.Count > 1)
{
// Find element v in c, that minimizes d[v]
int v = FindMinimizingDinC(d, c);
c.Remove(v); // remove v from the list of future solutions
// Consider all unselected nodes and consider their cost from v.
foreach (int w in (Hint != null ? Hint(v) : c))
{
if (!c.Contains(w)) continue; // discard pixels not in c
// At this point, relative(Index) points to a candidate pixel,
// that has not yet been selected, and lies within our area of interest.
// Consider whether it is now within closer reach.
int cost = TraversalCost(v, w);
if (cost < int.MaxValue && d[v] + cost < d[w]) // don't let wrap-around negatives slip by
{
// We have found a better way to get at relative
d[w] = d[v] + cost; // record new distance
// Record how we came to this new pixel
p[w] = v;
}
}
//Debug.WriteLine(string.Format("{4} {3} {{{0}}} [{1}] [{2}]",
// ArrayToString<int>(",", c), ArrayToString<int>(",", d), ArrayToString<int>(",", p), v + 1, ++step));
}

return new Results(p, d);
}

/// <summary>
/// Uses the Dijkstra algorithhm to find the minimum path
/// from one node to another.
/// </summary>
/// <param name="start">
/// The node to use as a starting location.
/// </param>
/// <param name="finish">
/// The node to use as a finishing location.
/// </param>
/// <returns>
/// A struct containing both the minimum distance and minimum path
/// to every node from the given <paramref name="start"/> node.
/// </returns>
public virtual int[] GetMinimumPath(int start, int finish)
{
Results results = Perform(start);
return GetMinimumPath(start, finish, results.MinimumPath);
}

/// <summary>
/// Finds an array of nodes that provide the shortest path
/// from one given node to another.
/// </summary>
/// <param name="start">
/// The starting node.
/// </param>
/// <param name="finish">
/// The finishing node.
/// </param>
/// <param name="shortestPath">
/// The P array of the completed algorithm.
/// </param>
/// <returns>
/// The list of nodes that provide the one step at a time path
/// from <paramref name="start"/> to <paramref name="finish"/> nodes.
/// </returns>
protected virtual int[] GetMinimumPath(int start, int finish, int[] shortestPath)
{
Stack<int> path = new Stack<int>();
do
{
path.Push(finish);
finish = shortestPath[finish]; // step back one step toward the start point
}
while (finish != start);
return path.ToArray();
}

/// <summary>
/// Initializes the P array for the algorithm.
/// </summary>
/// <param name="startingNode">
/// The node that has been designated the starting node for the entire algorithm.
/// </param>
/// <returns>
/// The new P array.
/// </returns>
/// <remarks>
/// A fresh P array will set every single node's source node to be
/// the starting node, including the starting node itself.
/// </remarks>
protected virtual int[] GetStartingBestPath(int startingNode)
{
int[] p = new int[TotalNodeCount];
for (int i = 0; i < p.Length; i++)
p[i] = startingNode;
return p;
}

/// <summary>
/// Finds the yet-unconsidered node that has the least cost to reach.
/// </summary>
/// <param name="d">
/// The cost of reaching any node.
/// </param>
/// <param name="c">
/// The nodes that are still available for picking.
/// </param>
/// <returns>
/// The node that is closest (has the shortest special path).
/// </returns>
protected virtual int FindMinimizingDinC(int[] d, ICollection<int> c)
{
int bestIndex = -1;
foreach (int ci in c)
if (bestIndex == -1 || d[ci] < d[bestIndex])
bestIndex = ci;
return bestIndex;
}

/// <summary>
/// Initializes an collection of all nodes not yet considered.
/// </summary>
/// <returns>
/// The initialized collection.
/// </returns>
protected virtual ICollection<int> GetChoices()
{
ICollection<int> choices = new List<int>(TotalNodeCount);
for (int i = 0; i < TotalNodeCount; i++)
choices.Add(i);
return choices;
}

/// <summary>
/// Initializes the D array for the start of the algorithm.
/// </summary>
/// <param name="start">
/// The starting node.
/// </param>
/// <returns>
/// The contents of the new D array.
/// </returns>
/// <remarks>
/// The traversal cost for every node will be set to impossible
/// (int.MaxValue) unless a connecting edge is found between the
/// <paramref name="start"/>ing node and the node in question.
/// </remarks>
protected virtual int[] GetStartingTraversalCost(int start)
{
int[] subset = new int[TotalNodeCount];
for (int i = 0; i < subset.Length; i++)
subset[i] = int.MaxValue; // all are unreachable
subset[start] = 0; // zero cost from start to start
foreach (int nearby in Hint(start))
subset[nearby] = TraversalCost(start, nearby);
return subset;
}

/// <summary>
/// Joins the elements of an array into a string, using
/// a given separator.
/// </summary>
/// <typeparam name="T">The type of element in the array.</typeparam>
/// <param name="separator">The seperator to insert between each element.</param>
/// <param name="array">The array.</param>
/// <returns>The resulting string.</returns>
/// <remarks>
/// This is very much like <see cref="string.Join"/>, except
/// that it works on arrays of non-strings.
/// </remarks>
protected string ArrayToString<T>(string separator, IEnumerable<int> array)
{
StringBuilder sb = new StringBuilder();
foreach (int t in array)
sb.AppendFormat("{0}{1}", t < int.MaxValue ? t + 1 : t, separator);
sb.Length -= separator.Length;
return sb.ToString();
}

}
}

The course I wrote this class for wanted me to add this algorithm to a simple graphics program that would "cut out" shapes using the shortest cost path. Here are the intelligent scissors that I wrote:

using System; 
using System.Diagnostics;
using System.Collections.Generic;
using System.Drawing;

namespace VisualIntelligentScissors
{
public class DijkstraScissors : Scissors
{
public DijkstraScissors() { }
public DijkstraScissors(GrayBitmap image, Bitmap overlay) : base(image, overlay) { }

int[,] traversalCost;
Rectangle relevantRegion;
protected int relevantPixelsCount
{
get { return relevantRegion.Width * relevantRegion.Height; }
}

public override void Trace(IList<Point> points, Pen pen)
{
if (Image == null) throw new InvalidOperationException("Set Image property first.");

using (Graphics g = Graphics.FromImage(Overlay))
{
for (int i = 0; i < points.Count; i++)
{
// Our segment travels from start to finish
Point start = points[i];
Point finish = points[(i + 1) % points.Count];

// Consider only some nearby region, to speed up processing.
relevantRegion = GetRelevantRegion(start, finish);
// Find the cost of moving to any pixel.
traversalCost = GetTraversalCost();

// Calculate what the array indexes are for the two known pixels
int startIndex = GetArrayIndex(start);
int finishIndex = GetArrayIndex(finish);

Dijkstra dijkstra = new Dijkstra(
relevantPixelsCount,
new Dijkstra.InternodeTraversalCost(getInternodeTraversalCost),
new Dijkstra.NearbyNodesHint(nearbyNodesHint)
);
int[] minimumPath = dijkstra.GetMinimumPath(startIndex, finishIndex);

// By now we should have found the best path between start and finish,
// considering all within the designated relevantRegion.
drawMinimumPath(minimumPath, pen.Color);

//g.DrawRectangle(Pens.Green, relevantRegion);
Program.MainForm.RefreshImage();
System.Windows.Forms.Application.DoEvents();
}
}
}

private void drawMinimumPath(int[] path, Color color)
{
// Show user goal point.
Point finish = GetPointFromArrayIndex(path[path.Length-1]);
Overlay.SetPixel(finish.X, finish.Y, color);

// Draw entire path
foreach (int index in path)
{
Point point = GetPointFromArrayIndex(index);
Overlay.SetPixel(point.X, point.Y, color);
//Program.MainForm.RefreshImage();
//System.Windows.Forms.Application.DoEvents();
}
}

private Rectangle GetRelevantRegion(Point start, Point finish)
{
const int minimumSpace = 5;
const float expansion = 0.01F;

Rectangle rect = Rectangle.FromLTRB(
Math.Min(start.X, finish.X),
Math.Min(start.Y, finish.Y),
Math.Max(start.X, finish.X),
Math.Max(start.Y, finish.Y)
);
rect.Inflate(Math.Max((int)(rect.Width * expansion), minimumSpace),
Math.Max((int)(rect.Height * expansion), minimumSpace));
// Make sure our relevant region stays within the bounds or calculating a gradient.
rect.Intersect(Rectangle.FromLTRB(1, 1, Image.Bitmap.Width - 1, Image.Bitmap.Height - 1));
Debug.Assert(rect.Contains(start), "Relevant region does not contain start point.");
Debug.Assert(rect.Contains(finish), "Relevant region does not contain finish point.");
return rect;
}

private int GetArrayIndex(Point point)
{
if (!relevantRegion.Contains(point)) return -1;
Point offset = point;
offset.Offset(-relevantRegion.X, -relevantRegion.Y); // remove effect of offset region
return offset.Y * relevantRegion.Width + offset.X;
}
private Point GetPointFromArrayIndex(int index)
{
Point point = new Point(index % relevantRegion.Width, index / relevantRegion.Width);
point.Offset(relevantRegion.Location);
return point;
}

private int[] GetPixelWeights()
{
int[] weights = new int[relevantPixelsCount];
for (int i = 0; i < weights.Length; i++)
weights[i] = GetPixelWeight(GetPointFromArrayIndex(i));
return weights;
}
const int maximumNearbyPositions = 4;
enum NearbyPosition : int
{
Above = 0,
Left,
Right,
Below
}
private int GetNearbyPixel(int origin, NearbyPosition relative)
{
return GetArrayIndex(GetNearbyPixel(GetPointFromArrayIndex(origin), relative));
}
private Point GetNearbyPixel(Point origin, NearbyPosition relative)
{
Point offset = origin;
switch (relative)
{
case NearbyPosition.Above:
offset.Offset(0, -1);
break;
case NearbyPosition.Below:
offset.Offset(0, 1);
break;
case NearbyPosition.Left:
offset.Offset(-1, 0);
break;
case NearbyPosition.Right:
offset.Offset(1, 0);
break;
default:
throw new NotSupportedException();
}
return offset;
}
private int GetRelativePosition(int start, int finish)
{
Point startPoint = GetPointFromArrayIndex(start);
Point finishPoint = GetPointFromArrayIndex(finish);
foreach (NearbyPosition position in Enum.GetValues(typeof(NearbyPosition)))
if (GetNearbyPixel(start, position) == finish)
return (int)position;
return -1;
}
private int[,] GetTraversalCost()
{
int[] weights = GetPixelWeights();
int[,] cost = new int[relevantPixelsCount, maximumNearbyPositions];
for (int i = 0; i < weights.Length; i++)
{
Point origin = GetPointFromArrayIndex(i);
foreach (NearbyPosition relativePosition in Enum.GetValues(typeof(NearbyPosition)))
{
Point relative = GetNearbyPixel(origin, relativePosition);
if (relevantRegion.Contains(relative))
{
int j = GetArrayIndex(relative);
cost[i, (int)relativePosition] = weights[j];
}
}
}
return cost;
}

private IEnumerable<int> nearbyNodesHint(int startingNode)
{
List<int> nearbyNodes = new List<int>(maximumNearbyPositions);
foreach (NearbyPosition position in Enum.GetValues(typeof(NearbyPosition)))
nearbyNodes.Add(GetNearbyPixel(startingNode, position));
return nearbyNodes;
}
private int getInternodeTraversalCost(int start, int finish)
{
int relativePosition = GetRelativePosition(start, finish);
if (relativePosition < 0) return int.MaxValue;
return traversalCost[start, relativePosition];
}
}
}

There are other dependencies not included here (such as the Scissors base class).  My purpose in this blog is to publish a generalized Dijkstra algorithm and give an example of how to use it.  If you would like the full source code, contact me to get me to email the code to you.  My BYU professor doesn't want BYU CS students getting the entire solution to their assigned projects.

[Updated 12/14/06]: The full source code is useful if you need a sample of how to apply the above algorithm in your own app.  Due to large demand for the source code and my limited resources to manually send it out (I can't just link to it here since my BYU professor doesn't want BYU CS students getting the entire solution to their assigned projects) I now charge a small processing fee of $5 to send you the source code.  If you would like the full source code, PayPal $5 to me and include your email address to get me to email the source code to you.  [Updated 8/30/07]: Do not use your credit card as the source of the $5 to send me PayPal money as PayPal will happily take half of the money for itself.  If you use your credit card, I'll reject payment and ask you to transfer money to your PayPal account from your bank account first.

You can also download the compiled assembly for free:

[Updated 3/16/07]: Comments now closed, due to people not reading the previous paragraph and still posting comments asking for the full source.