Review and Testing Improvements with Calman Ultimate Software

March 16, 2021 Simon Baker

Share this content

Index

Introduction

In an effort to help present even more useful testing results and data points in our reviews we have recently completed various updates and improvements to our testing equipment and methodology for 2021. This includes our new UPRtek spectroradiometer device that will help us improve colour measurements in various areas, including most notably for colour gamut and colour spectrum analysis. We have also updated and improved our response time testing and analysis to provide a more accurate representation of performance and experience.

We have since been looking at additional software that might allow us to take further interesting measurements and provide useful and relevant data and analysis in our reviews in the future. To do this we are making use of Portrait Display’s Calman Ultimate software suite, which provides a truly massive range of options, tests and features. We aren’t planning to go in to loads of detail about all the features of this software in this article, but instead focus on some of the tests that might be of particular use to us in future display reviewing. We are planning to incorporate some, but certainly not all, of the useful data and charts in future reviews, with a close eye on what we consider useful to readers. We want the additional data to be interesting, easy to follow and relevant and don’t want to flood the reviews with loads of charts and data for the sake of it. We will talk through some of the useful sections and tests of the Calman software below, and then explain which parts we will incorporate in to our reviews and how.

If you enjoy our work and want to say thanks, donations to the site are very welcome. If you would like to get early access to future reviews and articles please consider becoming a TFT Central supporter..

Check Availability and Pricing – Affiliate Links
Amazon
TFTCentral is a participant in the Amazon Services LLC Associates Programme, an affiliate advertising programme designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com, Amazon.co.uk, Amazon.de, Amazon.ca and other Amazon stores worldwide. We also participate in a similar scheme for Overclockers.co.uk.

Greyscale Measurements

The ultimate goal of display greyscale calibration is to perfectly balance each primary color to produce the target white balance color (usually D65) at every greyscale level from black to peak white. The practical goal, however, is to balance the colours to within an acceptable error at each greyscale level, as indicated by the Delta E (dE) chart. For our testing we use a colour temperature / white point target of 6500k (D65) and so these measurements will be taken relative to that target.

This section measures 20 different grey samples from black to white. From here it will calculate several useful measures:

Accuracy of the grey shades in DeltaE (dE) and plotted on a small bar graph on the left. The aim ideally is for the dE to be as low as possible for each measured grey shade. Note that this is different to the normal colour accuracy dE figures you will be familiar with from our reviews. Those are about the accuracy of displayed colours, here it is about the accuracy of the grey shades relative to the target colour temperature. Beneath the graph it will also confirm the average greyscale dE, in this random example 3.6. In this example the reason for the fairly high dE is that the colour temperature of the screen is measured at 6984k average which is about 500k different to the 6500k target, and so each shade ends up being a little too cool compared with the intended grey shade. dE should ideally be below 1 for the difference to be indistinguishable to the user. Anything less than 3 results in small differences, but anything higher than 3 is a problem and needs correcting through calibration and setting changes.

We think this greyscale dE graph is a useful addition to our review results along with the average and maximum greyscale dE figures, so we will look to include that in the future. We will also include the white point dE (taken at the reading of 100 for white), which provides a specific view of the greyscale accuracy on white content, which is particularly relevant when using the screen for office and reading work.
The RGB relative balance across the grey shades showing if any part of the RGB balance is off. Ideally the RGB lines would be flat at the 100% level line horizontally on this graph, but in this particular example you can see the blue line is quite a lot higher than the red and green, particularly as you get nearer to white (100 on the horizontal scale). This leads to a slightly more blue / cooler image than intended in this example. We think this is a nice visual representation of the colour temperature balance across grey shades and a useful addition to our review results.
The colour temperature average is then confirmed, helping identify if the display is too cool or too warm. In this example it shows as “Avg CCT: 6984” which is a bit too cool (CCT = Correlated Colour Temperature). As a reminder the target for a desktop monitor as well as the sRGB and DCI-P3 display gamut standards is 6500k. We already provide the colour temperature measurement in our reviews which is captured for the white point only, and we will continue to include that specific measurement as it is useful for common white background office and reading activities. We will also incorporate the average CCT taken across this larger greyscale sample set in the future for improved accuracy when considering all the grey shades.
Additional RGB balance representation is then shown for each shade individually depending on which you select on the software. In this example above it is showing the white 100 measurement, again showing here that the blue channel is too high.
The gamma point line is then shown on the right, which shows the gamma at each grey shade relative to the target 2.2 gamma. Ideally the grey line representing the monitors gamma would be very close to the target yellow line at all points. Here you can see the gamma is a bit lower in darker tones and a bit high in lighter shades. The average gamma measurement of 2.27 in this example is also included. As a reminder the target gamma for our testing for a desktop monitor is 2.20. We already include the average gamma and its deviance from this target in our reviews and will continue to do so, while also adding the Gamma point curve graph.
Contrast ratio is also confirmed here, although we will also measure that in a separate section of the software for the purposes of measuring it across the full backlight adjustment range.

ColorChecker Tests

This section is particularly useful for colour accuracy measurements when measuring a display out of the box and before any profiling and calibration has taken place. In this test 47 different colour samples are measured including various grey shades, all the main colours and a wide range of different skin tones. This is a much larger sample set than our traditional measurement using the LaCie Blue Eye Pro software, which measures only 18 samples and so this will provide an improved level of accuracy when measuring a display in the future. There is an even larger sample set available in the Calman software but this is probably overkill for the purposes of evaluating a typical display. Perhaps we will include it when we test any professional colour critical displays in the future but the 47 sample size is more than adequate for everything else.

The most useful section of this report is the graph on the left hand side which provides a visual representation of the dE readings for each shade. dE (“DeltaE”) is a measure of the accuracy of a displayed colour, relative to what it should be. The lower this number the better as it means there is less difference between the intended and the displayed colour.

On the graph there are green, yellow and red lines plotted on this graph to represent the target lines, with anything below the green dE 1 being indistinguishable and providing excellent colour fidelity. Anything between the green dE 1 and yellow dE 3 line should show small differences between the desired shade and the displayed shade. Anything higher than dE 3 is a problem and needs correcting really. From here the average and maximum dE figures are confirmed, in this example 2.79 average and 6.22 maximum. Most of the rest of this section captures the specific data and is unnecessary for our reviews and analysis. We will show how we intend to use this dE data in our review results later.

Updating to dE 2000 for Colour Accuracy Measurements

DeltaE is a measure of the difference between two colours. If you’re wondering, the E stands for the German word ‘Empfindung‘ which translates to sensation, or impression. With one color as a reference, DeltaE is a measure of the error of that color from that reference. dE has generally be defined by older standards like dE 1994. DeltaE 1994 is what we have measured for a long time via our LaCie Blue Eye Pro software and is based on color differences in the L* C* H* color space. It is an application-specific calculation and is commonly used in video calibration. dE 1994 does not however take into account the fact that the human eye does not perceive differences in color the same for all colours.

The answer to help address this and provide a more accurate measurement relative to what you would see as a user is dE 2000. DeltaE 2000 is still based on colour differences in the L* C* H* color space. It is not an application-specific calculation and is commonly used in video calibration. Unlike DeltaE 1994, it does take into account the human eye’s perceptual sensitivity at different colours. You will see in the tests discussed here that dE 2000 is used as the reference for colour accuracy and greyscale measurements, and we will use this for our future reviews.

Improved Measurement of Colour Accuracy for Wide Gamut Displays

One limitation with our previous testing software was the colour space you were measuring relative to. While you could define your target white point (colour temperature) and gamma settings in the LaCie software, you could not select the colour space you were comparing with. The software is from a time when displays were pretty much all limited to sRGB (standard gamut) and so it was not necessary to consider whether the screen was actually a wide gamut display or stretched beyond sRGB.

This meant that any time you measured the colour accuracy and recorded the dE figures of a wide gamut display it was always comparing the colours measured on the screen against the sRGB colour space. This resulted in high dE numbers because of course the colours being displayed by the wide gamut display were extended beyond the sRGB reference and were more saturated and vivid. If you want to consider the “colour accuracy” of the display relative to working with the most common sRGB content standard then that is fine, and it helps show that the “accuracy” will be off without profiling and without a colour managed workflow. When viewing sRGB content on a wide gamut display the colours and stretched and skewed and look more saturated and vivid than they are meant to.

After calibrating the screen you have an ICC profile which tells the screen how to “map” the colours to one another, and can be used within colour aware applications. In the LaCie software the ICC profile is created relative to sRGB and then validated against the same sRGB reference space. Confirming and validating a profile is fine, but it’s the out of the box appearance and accuracy of the display that are going to be far more important to most users and readers who either don’t have a calibration device or the means to colour manage their applications. This is also important to consider the accuracy away from any colour managed workflow where you can’t even use your calibrated ICC profile.

With the Calman Ultimate software we can now define our target colour space and provide a more accurate representation of performance relative to different colour spaces. So if a screen is a standard gamut model only we can compare dE colour accuracy relative to sRGB as normal. But if it is a wide gamut display we can also provide the measurements of dE relative to DCI-P3 content for instance. That way if you are wanting to work with or view content mastered with the now far more common DCI-P3 colour space (like HDR games for instance), you can also see whether the display is accurately showing the intended shades for this colour space.

This selection of colour space is also really useful for any screen that has different preset colour space modes or emulation options. For instance a professional grade display might have modes for sRGB, DCI-P3 and Adobe RGB for instance. It might also come factory calibrated for accuracy in each mode. With this new method we can compare the colours being presented relative to the intended colour space to more accurately analyse the performance. See a bit later in this article for a view of how this would appear in our reviews for wide gamut displays including the difference this makes when measuring a wide gamut screen.

Improved Contrast Ratio Accuracy

We have always measured contrast ratio with a small sample size white image and a small sample size black image. This is a more accurate representation of real-world contrast ratio than measuring a full white screen vs a full black screen which can sometimes lead to inaccuracy. One issue we have faced in the past, particularly on VA panels where black depth can go quite deep, is that there are sometimes rounding errors introduced because the black point was only measured to two decimal places. In the darker brightness settings this can skew the contrast ratio figures a bit. The Calman software actually measures contrast ratio to 3 decimal places which improves this accuracy nicely. We will take these contrast ratio measurements using this software in the future wherever applicable in the reviews.

Colour Volume Measurements for HDR Displays

For a long time a display’s colour range or “gamut” was typically represented with a 2D diagram such as the 1931 or 1976 CIE chromaticity diagram. A triangle overlaid on the chromaticity diagram indicates the bounds of the color gamut reproduced by a given display and can be compared to a given standard like sRGB, DCI-P3 or Adobe RGB for instance. An example CIE-1976 diagram for the sRGB colour space is shown above for instance. Our recent article about colour gamut explains this in more detail including why we have moved to CIE 1976 and adopted new a measurement device and calculation method for our reviews which will provide better accuracy and more useful results.

A color gamut triangle on a 2D chromaticity diagram represents a display’s range of colours at only one luminance level. In the world of SDR, illustrating a display’s color gamut on a 2D color space proved satisfactory, as calibrated displays behaved in a similar manner. Peak luminance was 100 nits and colours were generally well saturated at one luminance level, typically 75 percent of the display’s peak luminance.

But in the HDR era, things are different. Peak luminance can be commonly generally five to ten times higher than SDR, and in some cases even higher. These varying luminance levels can have a big effect on how colours are displayed. For example as the image exceeds the display’s peak luminance capability, colours can be displayed differently from what they were intended to be, sometimes resulting in a wash-out effect. So, while 2D color spaces were sufficient to indicate color gamut in the SDR era, things are different in the world of HDR, where luminance levels differ greatly.

To accommodate the variations of different displays, luminance was integrated as a third dimension into the traditional 2D color gamut diagram to create color volume. This 3D measurement illustrates how a display reproduces colours at all of the luminance levels of its luminance range.

Adding luminance levels to the 2D color gamut diagram results in a 3D color volume that represents the combined ranges of hue, saturation, and luminance that a display can produce. The CalMAN Color Volume Analysis section uses the CIE L*a*b* Color Volume metric and measures 140 points and calculates the CIE L* a*b* volume as a percentage of BT.709 (sRGB), DCI-P3, and BT.2020. Put simply, the higher the colour volume measurement, the better display can express a vast range of vivid, accurate colours. In this example you can see a 141.4% BT.709, 95.0% DCI-P3 and 64.1% BT.2020 colour volume.

It is also possible to provide a visual plot representation of this colour volume, with the outline mapped to DCI-P3 coverage for reference.

We will look in to including these colour volume figures for true HDR displays in the future potentially if we feel it will add value beyond the 2D colour gamut calculations. There is no need to provide them for SDR displays though.

We are exploring further the HDR analysis sections of the Calman software for potential usage in the future, but that is a topic for another day.

Updated Reviews

Taking all this new data in to consideration we will be updating our future reviews in various places as discussed below. We will provide these measurements and results for the screen out of the box at default settings, and in various preset and emulation modes where necessary. We will also use these for our “optimal settings” tests and calibrated results.

Part 1 will focus on the greyscale, colour temperature, gamma and contrast of the display. You can see at the top in the middle a confirmation of the screen’s default OSD settings for reference. To the right of that at the top is the default luminance (brightness), black depth and resulting contrast ratio. You can see that the black depth measurement accuracy is now improved to 3 decimal places which improves the accuracy of the contrast ratio figure. Elsewhere in the review we will continue to provide brightness and contrast measurements in more detail across the backlight adjustment range, again with this improved accuracy.

In the lower left area is the greyscale performance. This measures 14 different grey shades from black to white and confirms how close they are to their target temperature of 6500k. Ideally dE should be <1 which would be indistinguishable to the user and would mean that the grey shades are displayed as intended, and relative to this 6500k temperature. Between dE 1 and dE 3 is a small difference, and anything above dE 3 is an issue and needs correcting through calibration, as the grey shades are too far away from their desired appearance. The graph showing the dE of the greyscale is included, and beneath that confirms the average dE, the white point dE and the max greyscale dE. These are colour coded relative to their score in the same way as our colour accuracy ratings are (shown later) with green being good, amber being moderate and red being bad. The white point dE is particularly useful when considering usage of common white backgrounds for office use and reading text.

The middle lower section provides a visual representation of the RGB (red, green, blue channel) balance across each grey shade relative to the 100% target line in the middle. Ideally all 3 RGB lines should be flat and at the 100% level which would provide a balanced 6500k across all grey shades. You can see here that the blue channel deviates a lot from the target line, particularly as you get closer to white (100). Beneath this graph confirms the average correlated colour temperature across all grey shades and it’s deviance from the 6500k target as a percentage. This is colour coded based on how far away from the target it is, and in the same scale as our screen comparisons charts we provide in the reviews. There is also confirmation of the white point colour temperature which is particularly relevant again for office and text work.

In the lower right area is a graph showing the gamma point. The grey line tracks the monitors gamma at different grey shades and plots how close this is to the target 2.2 flat yellow line at each point. This graph is useful as it can identify where the gamma deviates from the target, which has an impact on visual performance in a number of ways. We provide the average gamma measurement and deviance from the 2.2 target here as before, but the graph provides useful additional information:

If a displays gamma is too low at any point in means its luminance is too high for that grey level. If this occurs in darker shades (lower left quadrant of the graph) then it means the display is washing out shadow details and can produce overly bright, muddy blacks. If the gamma is too low for brighter shades (lower right quadrant) it means the display is compressing or crushing bright greys and losing detail there (e.g. cloud detail).

On the other hand is gamma is too high at any point it means the luminance is too low for that grey level. If this happens in darker shades (upper left quadrant) then this results in compressing or crushing of dark greys which again loses shadow detail but for a different reason than low gamma. Dark shades are too dark and lose clarity. If the high gamma occurs for lighter grey shades (upper right quadrant) results in the washing out of bright grey details, lowering the contrast. So this additional gamma graph provides useful detail and analysis beyond just including the average gamma figure. We will also provide the average overall gamma, along with the average gamma for dark shades (0-50) and for light shades (50-100).

The second part of the results focuses on the colours of the display. For a wide gamut screen such as that shown in this example we will measure the colour gamut and colour accuracy relative to standard gamut (sRGB) and wide gamut (DCI-P3 in this case) references.

The top section focuses on sRGB gamut. The monitors gamut (colour space) is measured and plotted on the CIE 1976 diagram relative to the very common and widely used sRGB standard gamut reference. You can see in this example that the monitors white triangle extends a considerable way beyond the sRGB maroon coloured triangle. Beneath the CIE diagram is the absolute and relative coverage of the monitors gamut compared with sRGB. We have talked in our previous article about moving to CIE 1976 measurements instead of CIE 1931, and making use of a new measurement device and calculation tool for improved accuracy with colour gamut.

To the right of this there is the familiar dE measurements for colour accuracy. This is now based on a much wider 47 colour sample set instead of our previous 18 samples. dE is plotted relative to sRGB in this first section, showing how accurate the colours are relative to the sRGB standard. This is based on the improved dE 2000 figures instead of the older dE 1994 as well. An average and maximum dE figure is provided beneath the graph along with a rating for the colour accuracy. As explained earlier, when measuring a wide gamut screen the colours will look more vivid and saturated compared to standard sRGB gamut screens and so it is not surprising that this accuracy is not as good.

To help overcome this we also consider the target colour space of the screen and measure the gamut and colour accuracy relative to that as well. For wide gamut screens we will plot the CIE 1976 diagram as well relative to a common colour space, in this example DCI-P3 is shown but we could also use Adobe RGB or Rec.2020 in some cases. The gamut absolute and relative coverage calculations are again included below this in the table for various reference spaces. To the right we then include the dE colour accuracy of the screen, but because it’s a wide gamut display, this is now relative to DCI-P3 in this example. You can see that if you are comparing the accuracy of the colours produced compared with this wide gamut reference, it is significantly improved compared with the sRGB measurements above. The average dE and max dE are provided again along with a screen rating.

We will also use the Calman software to complete the calibration and profiling of the screen for improved accuracy and analysis, and we can then provide the same results discussed above after this is completed.