19
Jan
15

Benchmark Software for DOS

Comparing the speed of computers is not a new thing, it has been done as long as there have been computers. It can be problematic picking a good benchmark, many output meaningless metrics, do not give consistent results, or simply fail by measuring the wrong things. Today I’m looking at a small collection of benchmarking utilities.

I am not equipped to capture from my DOS machine, so as a standard machine I used Dosbox with a setting of 3000 cycles. This isn’t ideal as it doesn’t mimic hardware exactly so it may cause some funny results. I’ve noted in particular that Dosbox’s video memory is faster than in real machines.

Topbench screen

Topbench screen

First up is a relatively new utility called TopBench. I’ve found this one handy for adjusting the emulation speed of Dosbox to approximate an actual machine. It benchmarks the system in real-time so any adjustments you make will be reflected on the screen straight away. The built-in database of machines gives you a good idea of roughly where you machine fits.

However the metrics provided aren’t really clear about what they are measuring. We get a measure of how long each type of test takes, but don’t get any indication of what each test does. So this is good for comparison, but isn’t going to tell you much about the various parts. Still I like this one a lot for tuning performance in Dosbox.

C&T MIPS

C&T MIPS

Chips and Technologies made a simple one that is supposed to measure the MIPS a system is capable of.  It would be good for comparing CPU speed and not much else, except that the MIPS rating doesn’t seem accurate for the speed. I thought Dosbox may have been the cause, but others have noted this as well on actual hardware.

Landmark Speed Test

Landmark Speed Test

Back in the day the Landmark Speed Test was frequently used to compare performance. It compares your system with an IBM AT and is measured in Mhz for that machine to achieve the same performance. It doesn’t tell you how fast the AT machine was, or what processor it was running. I guess you can still use this to compare between systems, but the metric is less useful.

It measures the speed of your graphic card in Characters per second which I thought was a bit odd. Mainly as very little performance limited software used text mode all that much, so I think the metric isn’t very useful. Also we don’t know if their test uses the BIOS routines or draws to the hardware directly, so this may not indicate anything about the hardware. Additionally this metric appears to be inaccurate.

Checkit CPU and FPU performance

Checkit CPU and FPU performance

Checkit isn’t just a benchmarking utility, it also provided technical information about the machine and offered some basic hard disk and floppy disk utilities. The benchmarks are better than some in that they provide the raw data in the form of Dhrystone and Whetstone loop counts.

Dhrystone and Whetstone were some basic benchmarking algorithms developed specifically for testing integer and floating-point instructions respectively. Both were synthetically designed for benchmarking machines, but will suffer inaccuracy due to compiler optimisations, and differences in the languages used to implement the benchmark. Despite the short commings these are still widely used benchmarks.

Checkit Character Through-put

Checkit Character Through-put

Notably Checkit also measures the graphics memory through-put using characters per second. Except they have separate measurements for rendering using the BIOS and directly handling the video buffer. You can see just how different the results are here. Whilst the results are a little more meaningful, it’s still measuring the text mode performance.

SpeedSys Results

SpeedSys Results

Lastly a commonly used tool amongst retro PC enthusiasts is SpeedSys. It is good for benchmarking faster DOS machines. I ran it on my Pentium MMX based MS-DOS 6.22 machine here, and you can see that it provides a lot of information about your hardware. The memory and hard disk graphs are perhaps the most interesting.

The memory speed graph shows the speed versus the data size (in KB). You can see on the graph several drops in speed, these roughly correspond to the L1 and L2 cache sizes. You’ll also notice how the both write graphs don’t seem to enjoy any speed boost from the cache. I can only assume this is because of the cache policy being write-through, but I can’t be certain.

The Speedsys test is probably the best one as it provides the most detail. The memory and hard disk tests are quite good as they give measurements that mean something outside of the benchmark. The only thing I would have liked is more detail about the graphics card, but there isn’t really any more room on screen.

Whichever benchmarking utility you use, remember to always only compare your results to those produced by the same program. Even where the same metric is used as in the case of Drhystone and Whetstone tests. Otherwise you’re really comparing apples and oranges.

Advertisements

6 Responses to “Benchmark Software for DOS”


  1. January 20, 2015 at 4:11 am

    I love benchmarking..Its silly but there’s something fun about seeing a visual representation of your machines speed. going back and tinkering with BIOS settings or adding better cards and then comparing. Its a shame none are extremely accurate. I like Speedsys as well for later era DOS stuff, 386 to early Pentiums. for older DOS stuff I use Norton sysinfo. it works well enough for PC, XT and AT systems and gives a wealth of info. A guy over at the Vogons website put together a little benchmarking pack of 4 utilities that works really well together to give an idea of speed, at least on 286ish systems and up. it has PCPBench and 3dbench as well as a little speedrun demo for DOOM and Quake that let you know your FPS after the demo runs. I like that cause it helps give me a good idea of how actual games will run. You mentioned VGA capture and yhea, I wish there was a good solution for that other then DOSBOX for those that run actual hardware but there isn’t. I’ve looked into it. best I’ve come up with for real hardware is just pointing a camera at the screen. If you have a 486 machine with PCI you can use a capture card I suppose or use a card with a S-video or composite out but then your compromising compatibility/video quality. I’m sure a device exists but i haven’t found it at a reasonable price.

    • January 20, 2015 at 11:03 am

      I had a look at Norton Sysinfo, but wasn’t quite sure what to make of the benchmarking capability in it. It does seem to otherwise be an excellent tool for looking at a systems specifications. I’ll have to look over on vogons to see if I can find that pack, I had a quick look but didn’t see that one, it sounds interesting. I’ve heard Doom and Quake have good benchmarking stuff built-in but I wouldn’t be able to use them on older systems.

      On the screen capture side of things I am eventually going to investigate some software options to see what can be done. I vaguely remember there being a DOS-based screen capture program called screen thief. I have no idea how well it will work, and it is certain to have issues with some graphics modes and games so it isn’t going to be a complete solution. It will also take up valuable memory so that could be a problem.

      Thanks for commenting 🙂
      Cheers
      Sparcie

  2. January 20, 2015 at 3:35 pm

    here you go. http://www.vogons.org/viewtopic.php?f=46&t=37844

    I do believe they can all be downloaded separately, the test in DOOM can be run via the shareware ver. I think. its a little obtuse how you get FPS from it. after DOOM runs though a demo it gives you two numbers, realticks and some other number, you have to divide one by the other and multiply by 32 and that’s your FPS rate. It does give a nice idea how your system will run certain games though. I know DOOM uses some VGA mode that some “newer” cards do not like. try playing DOOM on a Rendition Verite card. it has a horrible 2d VGA core, you get better fps on a ISA Trident card. I found these bench programs work well on everything back to a 386. on a 286 I couldn’t get Quake or Doom running but PCPbench and 3dbench worked fine. I really wish there was some consistent means of benching that was developed, some of those programs use kind of weird scales, like landmark you mentioned that compares everything to an AT.

    If you ever find a good VGA capture program or external device for actual vintage hardware let me know.

    • January 21, 2015 at 12:20 pm

      Cool I’ll have to check those out! The graphics mode that Doom uses is known as modeY which is a lower resolution variant of modeX (320×200 vs 320×240). Both are non-standard modes that I guess could be a problem if your card wasn’t 100% VGA compatible. Do those same cards also have trouble with some/all EGA games? The hardware features used by Doom to produce ModeY graphics are important in EGA compatibility as well.

      I don’t think there will ever be a benchmark that will be universal or standard. There is too much difference in optimal code between each generation of processor, so much that code that is fast on a 8086 can be relatively slow on a 386, not to mention it wouldn’t use the advanced features. So any benchmark isn’t going to show the best performance possible on all hardware. Using a games FPS for benchmarking purposes is actually probably one of the best ways of comparing systems as a whole, just it gives little detail about individual components contribution to speed.

      cheers
      Sparcie

  3. January 21, 2015 at 4:33 pm

    I never tested the Verite on any EGA stuff but I assume it would be pretty bad. its known for its terrible VGA 2d core. I ended up selling it awhile back after I was done testing it out. I did some benching with it and multiple cards on my Pentium 120mhz machine with DOOM

    just a sample

    PCI Rendition Verite = 12.1 FPS
    ISA Trident TVGA 8900D (one of there best ISA cards) = 18.69 FPS
    PCI Trio64v+ = 50.71 FPS
    PCI Matrox Mystique = 51.55 FPS

    I agree with using games to benchmark. Quake and Doom work well for 386 and up but I haven’t found a game to use to test FPS for 286 and prior though.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Blogs I Follow

Enter your email address to follow this blog and receive notifications of new posts by email.


Mister G Kids

A daily comic about real stuff little kids say in school. By Matt Gajdoš

Random Battles: my life long level grind

completing every RPG, ever.

Gough's Tech Zone

Reversing the mindless enslavement of humans by technology.

Retrocosm's Vintage Computing, Tech & Scale RC Blog

Random mutterings on retro computing, old technology, some new, plus radio controlled scale modelling.

ancientelectronics

retro computing and gaming plus a little more

Retrocomputing with 90's SPARC

21st-Century computing, the hard way

lazygamereviews

MS-DOS game reviews, retro ramblings and more...

%d bloggers like this: