XF86Setup では、次のような設定をするとうまくいきました。

Mouse:
PS/2
/dev/psm0

Keyboard:
Generic 101-Key PC
U.S. English
Made CapsLock an additional Control

Card:
List: Matrox Millennium G200 8MB
Detailed Setup: Server=SVGA Chipset=mga2064W

Monitor:
Multi-frequency that can do 1280x1024 @ 74 Hz

MOdeselection:
1280x1024, 16bpp