mkk

Firstly I really like how readable the graphs are. Today there's a very good mix between style and function.
Secondly I've had some thoughts on what framerate data that is best to have there. The Medium and Lowest values are great to have there, with the Maximum trailing as a kindof "throw it in if there's space left to fill" as it doesn't hold much value. But I was thinking that perhaps the lowest score might be a bit blunt of a tool, as a few hitches over a period of testing could result in the lowest score giving a skewed presentation of the situation as a whole. I don't know exactly how you gather the results but likely you get a lot of raw numbers over the time of testing, so I was hoping that it could be possible to have another level between Lowest and Medium consisting of around something like 10 or 15% of the lowest scores(unfortunately I'm not one to suggest the exact formula).
A small difference between this Low-balanced and the standard Lowest value would indicate that this level of framerate can be expected to be encountered fairly frequenty. If the difference between the two end up being relatively large then we have an indication that the lowest result might not appear too frequently. Either result could be quite interesting when comparing two products or set of setttings, or whatever. In order to not get too many values in a graph I'd suggest scrapping the Maximum in general since not only is it largely uninteresting but its precense might even confuse visitors not too customed to evaluating these results.

Veridian3

HI Mkk, generally when we do reviews any FPS below 25 have some sort of explanation. Along the lines of "it only happened once during the level" or "it happened multiple times" which gives the reader a better idea about the experience than the FPS alone. We shall consider your points though and see if they can be used at some point in the future.

SeraphicSorcerer

Just out of curiousity, because I can't ever remember, are the graphics options in the control panel for nvidia and ati set for High quality? have we ever messed with the mipmap settings here to see differences in quality/performance? I know everything's fresh install, so default settings then?

__________________

mkk

Good. I was going to write that perhaps there was already some method for this in your testing. And perhaps if you carry out a few checks of data along the suggested line you might end up with the conclusion that the two values doesn't vary often enough in between products to be worth including that kind of data. Could possibly be more worthwhile than the bar for Maximum value though.

Zardon

I actually played about with horizontal line graphs for the frame rate figures for a while last year, which I know is a similar system to HARDOCP. I personally feel the way they handle the graphs isn't ideal however as when you overlay 5 or more products it becomes one big mess of coloured lines and becomes a little tricky to ascertain anything worthwhile from it. Just my view however...

I do agree with you though, there is some merit in monitoring the min frame rates across various sections of game play. I will discuss ideas with V3, thanks for the valuable input MKK.

IvanV

From the "Testing Methodology" page (if that's what you meant):i doubt that many people ever move the mipmap setting from "High Quality" (best and default) and whatever is the appropriate value in Nvidia's control panel. A long time ago, I played with it on my old 9800Pro and I haven't gained even a single extra fps by going from highest to lowest quality while the visual quality degradation was just awful. I wonder why they even keep the option. On the other hand (this is off topic), I'm a bit curious about our tester's personal opinion on Catalyst AI. I usually set it to "Advanced" and, while I haven't benched much in awhile, from previous experience, I'd say that it usually provides a touch higher performance without any bad side effects.

__________________
Feed the porkers and don't touch the buttons!

Veridian3

We leave the driver options (other than vsync) at default because it suits the majority of readers (and I mean vast, vast majority). Saw some stats from one of the manufacturers once and in that there was a section which said well over 90% of people never even access the control panel. (well over 90 but i cant remember the exact stat).

If we were doing an article where we looked at the IQ under various scenarios we might play about a bit more but the fact is the settings we test at are the ones NV/ATI recommend.

SeraphicSorcerer

That makes perfect sense, but I've been curious if changing settings would result in net performance gains without losing image quality, such as IvanV's question as to standard and advanced AI.

__________________

Zardon

Unfortunately for us to satisfy this curiousity means a hell of a lot more work, and as Veridian3 said. im confident 99% of the audience dont even touch those settings. I know I don't (I did for a while and pretty much found what Ivan said earlier).

SeraphicSorcerer

Good point Zardon, I think the newer architectures don't really give a crap about the settings,

I was wondering about the AI same as Ivan though, However I do have a suggestion for reviewing could we maybe bold or italic in the reviews when framerates dip?

For example: (just a bs line)

..As you can see Crysis framerates remain steady throughout our review, however upon entering "insert level here" performance took a hit due to "" However game play was not really affected. As the hit was barely noticeable.

Sometimes I skim reviews, because I am in a hurry, but want to read it. Which of course is my fault, was just wondering if we could have momentary explanations or even a * note.

Just suggestions to try and improve em sorry if they're stupid.

@ Zardon maybe we should twitter CM To ask If those settings do anything and make him do the work

I don't know if there are any Nvidia equivalents on these forums, or I'd ask them too

__________________

Judas

i wonder what about min-average-max frame rate graph bars for each video card overlapping on themselves, only decernable by either the remaining color between each..

just an idea to trim up on the mess of color and lines

__________________

DJ BIG T

Z im one of them that dont touch them...

__________________

Zardon

well im not sure if any of you noticed, but the latest review on Nvidia's 3D vision/Gigabyte GTX 275 SOC used a new min frame rate graph we came up with. Feedback would be good. Helpful? waste of time?

niceguyrichy

I like it.
gives a good overall impression of the cards general performance with just a quickish glance

__________________

comp_ali

The new graph is useful , I liked it.

__________________

"It is doubtful how DRM could reduce piracy and there no way it can boost games sales either"

ET3D

How about some control over graphs? All Flash is used for right now in them is for animation, but it could easily let users sort by min/avg/max and display just the data they want. That way you could cram all the data you want into the graphs, and perhaps by default not show everything, but people who do want the most info will be able to get it.

Zardon

Its a nice idea, but it is vastly more complex and some people would probably end up missing half of it anyway. We still get daily emails from people unable to work out how to join the forum, if we complicate the flash with various buttons and options it might just put people off even reading them in the first place.

Dark Lord

the new low performance graph is nice.


but now that i know you guys were toying with hardocp style graphs, which i like the idea of, ive got a suggestion on that.

since everything is done in flash anyways, what about just mousing over the card of your choice on the graph, and having that cards performance line highlighted. there would be no lack of info, no real button pushing, and you could see the graphs of your choice much easier.

__________________

Zardon

Well they aren't really 'hardocp' style graphs. line graphs have been in existance long before hardocp even started, but yes they are one of the most well known for doing them. I personally dislike their particular style however as they end up so hard to make out especially with a lot of cards.

Thats an interesting idea Dark Lord, something to chew on.

ET3D

You don't have to complicate the view. A single button opening the options would be enough, and will be easy to ignore for people who just want the normal graph.

You also have enough visible components already that could be used for control. For example, the "Avg, Max, Min" legends could be used as buttons, to hide that part of the graph when clicked (and then be greyed out, another click returns them). The bars themselves could be dragged to change sorting.

I think it's okay for some people to miss it completely. There are always different levels of users, and regular DH readers will eventually learn they can control the graphs. You can mention this ability to control the graphs in the article.

Even without options, it's possible to make graphs more interactive. For example, overlaying min,avg,max on a single bar saves space, but makes reading the numbers hard. But hovering over the bar could pop up something clearer.

Zardon

I came up with a new min FPS graph which will be in our next review. its not interactive but it has quite a lot of information in it. we can fine tune it as we go.

All these ideas are great guys, but we spend literally 4-6 or more hours just designing a review for this site, most other sites spent 20 mins uploading a few pics and keying in black text, I need to get some kind of equilibrium or we will be producing far less reviews due to design time. We will get something to keep most people happy. I do stress most, because I think even if we could get leonardo di vinci to do the website design people would still bitch that he 'sucked'. Such is life

Bolletje

The Min FPS graph is really nice, but maybe you can add a button so you can select "Min - Avg - Max"?

You can use the same color scheme: red (min) , blue (max) and green (avg).

__________________
It has come to my attention from sources apart from yourself, that you enjoy the water pokemon mudkip in some manner or form.

comp_ali

I don't hate the new Frames style , just thought it is has much details that keeps me confused. The past one was simple and clearer.

Don't take my feedback as a hostile , I just like to post my personal feedback

But you are right , you will never satisfy all people even whatever you did.

__________________

"It is doubtful how DRM could reduce piracy and there no way it can boost games sales either"

Zardon

Nope. only min is important for those - its the deciding factor for playability which is the reason they are there in the first place. No one is interested really in seeing a constant stream of max frame rates at say 80-120. its useless.

Zardon

The earlier version only had 12 readings from specific sections and why it was probably 'easier' was due to the fact each section was detailed with a representation frame rate figure.

The new graph is actually much more informative because it details a constant stream of minimum frame rates over a demanding section of game play , so as the flash is 'drawn' you are seeing the frame rates almost as it would happen - although slightly slower to be more representative of a longer area of game time.

For instance if you feel that you don't want a specific first person shooter game to drop below 30fps. you find 30 on the graph and look from left to right and see how often it dips under the figure. Then you can make an educated assumption on whether that card would be doing the job for you at that chosen resolution and setting.

Its actually much more useful if you look at it. the frame rates are on the left marked clearly, you run your eye horizontally across the graph to see how the game plays out in regards to the frame rate.

[hobo]eclipse

totally love the new graph, it is very informative. great addition to the standard fps graph and will help to show the minimum performance expected. a lil off topic, but what are the pics behind the graph in the recent AMD review in the l4d2 section of? looks to me to be a new take on the l4d player models. (they look great) is this something from the upcoming sequel i don't know about? some big secret? lol, or just the way it is?

__________________

Zardon

Good to hear, i had a few emails tonight saying it was very useful to a few readers, so hopefully we can make it a mainstay in upcoming reviews. This isn't something we just decided was a nice addition a few days ago, it was actually based on community feedback. Working out the best way to present it proves tricky at times and i think with more than 2 cards in a roundup we may need to further adjust it. Its an ongoing process to adapt and enhance everything we can.

If we are helping you guys work out performance in a worst case scenario it has to be useful. I think we are doing most things right, its all real world testing, its at the correct resolutions and settings for the hardware, everything is patched, newest drivers are used, newest bioses are used, the OSes are always fully updated and we end up working through days of benchmarking, even missing launches now to make sure we deliver. We could have had that AMD review up for instance over a day ago for the Vapor X launch but we wanted to add more testing, even if we lose hits over it. Quality is important, rather than always being 'first up'.

Its just artwork Valve gave us a while ago, we have literally gigs of artwork from publishers in folders. I can send you it if you want. PM your email address.

Left 4 Dead is such a great game, I love the mindlessness of it at times, absolute stress buster for me. im loving the sequel too, awesome games.

Dark Lord

those would be the original l4d models that got scrapped for the current models.

Z, just a bit of nit picking. is it possible to add a horizontal line through the fps graph at the 30 mark? its a little hard to see where the 30 line is on some of the tests, like crysis.

__________________

Zardon

Actually that might be a damn fine idea. I think most people treat 30 fps as the sweet spot in most games, right?

Chaos

I think that is a good idea, +1 for the thought. I don't consider it a sweet spot but a minimum excepted frame rate for smooth video

__________________