nVidia cheats at 3DMark03?

[Strawbs]

It's looking like nVidia may have been caught with their hands in the till. This could be explosive stuff (http://www.geek.com/news/geeknews/2003May/gee20030516020031.htm). :eek:

[chubtub]

Yep, they will do anything to keep ahead. It is amazing the race for speed. I would rather have the race for stability rather than speed.

[AllGamer]

Clearly a cheat, if they drivers were really calculating as it should have been, then that wouldn't have happend

[Kruzin]

This is nothing new for them. They've been 'cheating' with their drivers since the days of the TnT/TnT2 cards in many various ways.

And there is already a discussion about this topic:
http://www.sysopt.com/forum/showthread.php?s=&threadid=137069

[Bigjakkstaffa]

Yeah theyve bene doing it for quite some time indeed, they did it upon the release of the first set of drivers following the FX launch to make em seem better than they actually were

--Jakk:t

[Strawbs]

well I for one am sorely tempted to become a "mountie" for future video card purchases (until ATI does something to p*ss me off). I have read many times in these very forums that ATI's graphics are of a better quality than nVidia's, this is not just icing the cake, if it proves to be true it will be the cherry on top.

[Rugor]

I posted a thread about this very thing a few days ago, I guess it dropped off the bottom of the radar.

Yes, they appear to be cheating.

And if that's not enough there are a number of people who think this is fine, because "It's only 3DMark."

I really don't want to see people heading down the path of the identity of the victim being the determinant of the morality of the act.

[Strawbs]

Originally posted by Rugor
I posted a thread about this very thing a few days ago, I guess it dropped off the bottom of the radar.

Yes, they appear to be cheating.

And if that's not enough there are a number of people who think this is fine, because "It's only 3DMark."

I really don't want to see people heading down the path of the identity of the victim being the determinant of the morality of the act. My Appologies Rugor, If I'd seen your thread earlier I would have posted there!

3DMark is the users choice of benchmark and nVidia knows this, therefore (if true) it is the user that they are trying to fool. I do not take kindly to being made a fool of!

Unlike Microsoft ...who's current monopoly in the OS market makes any alternative unacceptable in my "home working" situation ...nVidia has a completely legitimate and viable competitor in ATI, afterall many of us already use the alternative to Intel processors. If these allegations prove to be true, it would be tantermount to a deliberate breach of trust between nVidia and it's main core of customers. They would lose my support until a full and uncompromising appology is published and nVidia bashing would abound at every opportunity.

I would think that Futuremark would also be a fair bit angry with anyone that attempted to devalued it's benchmarking software.

:t

[GamerBoyFranco]

They seem like real cheaters

[$1500-P4 gamer]

hehe-well ya know I learned in my past a valueable leason with girlfriends.;) If they accuse you of cheating/ dont trust you. It usually means they are the ones with somthing to hide/feel guilty about. Nvidia did the same thing here. had a tissy about ati's drivers being tweaked for 3dmark when 8500 dusted the floor with nvidia. Now Nvidia has done it in the past and continues to. So calling the kettle black eh? :rolleyes: At this point Im so fed up with nvidia's buisness practice Ill turn down everyone over ati! I still havent forgiven them for the hostile take over tactics on 3dfx!:( Get in a law suite over ownership of tech. THen when the guys are broke from pushing next card release/ battling court fee's. You buy them out while they are worthless! THats what nvidia did and we all lost a great co. as a result. I hope nvidia stays in buisness as like with 3dfx we will all lose. BUT I also hope they become the second rate video card co. they are trying to become.:D

[Dracas]

Seriously...if someone managed to get it all set up, round up a good fab facility and gave it a new name, do you think all the old 3DFX programmers, engineers, etc.. would come together again to work under one roof?

Lets just take the people with the all time best concepts and give them a second chance with all the new fab capabilities.

That'd be awesome, but is it even possible?

hehe and what the heck would you call it since you can't use 3DFX?

[$1500-P4 gamer]

"hehe and what the heck would you call it since you can't use 3DFX?"

Easy- 3DEX:x

[Dracas]

Maybe simpler, 3DX?

I mean, look at 3DO's mass following at its apparently three letter acronym

if not, 3DXtreme? PixelFX? Maybe just rename the company based on their voodoo series, hehe

Actually, some of that sounds rather hokey, but I think the old 3DFX patrons should band together again and start crankin out another new videocard revolution, these days, theres no originality

just forty manufacturers making nVidia cards that all look and perform about the same, and forty manufacturers making ATi cards that all look and perform about the same (with the exception of Sapphire. Sapphire is an exploding computer waiting to happen).

Gone are the days where you could peruse through the local computer store, weighing the available processors and video chipsets

"Hmm...for my new system, should I go with 9FX? Maybe 3DFX, I hear they make some of the best 3D cards on the market...whose this nVidia company? Hey, they don't look all that bad, but then, here's one made by . . ."

Here's something I learned about any business: with no competition, prices become inflated, bottom lines are overlooked, and companies start doing stupid things like jackin their drivers to get better benchmarks just to spite one other company.

"Overspecialize and you breed in weakness" thank you Major Kusanagi for that wonderful comment.

I WANT to be able to choose between AMD, Via-Cyrix, Intel, or even Evergreen, but all that survived was AMD and Intel. I wish Evergreen was still around, they really were the pioneers of Overclocking :)

That and I liked having a wide array of chipsets to choose from for video graphics (I'm talking real contenders folks). I remember when my 9FX Reality 332 almost stood head to head with a Voodoo2 or nVidia Vanta/M64/TNT/etc...

Buying and tinkering with computer parts just isn't as fun anymore, because you can either buy this or that, and nothing inbetween, well, at least not if you intend to have a pretty cutting edge machine.

****, its like the whole computer worlds gone into a multi-partition Bipartisan Semi-religious faction of drone microtechnology shared and stolen on a day to day basis with no real new innovations being destined because all of the competition is in one place, so you don't have to plan a defense from multiple angles.

Whoops, ranting LOL, sorry folks :) :t

[$1500-P4 gamer]

Unfortuantly these events were inevitable. As pc world is a jungle. It goes by jungle law. Only the strongest survive. this will cut out some good co.'s tech.

[stix_kua]

This makes me wonder if other companies could be doing this same exact thing...(Intel )...:eek:

The other thing to this is the EA nVidia alliance...Could this be why C&C Generals work on very few video cards...I hope they both crash and burn! Both of them EA and nVidia.

I'm sticking with ATI

[$1500-P4 gamer]

I totally agree!;) When I first posted about them uniting against ati sis and others everyone thought i was nuts. But I posted lots of links including ones to Nvidia claiming just this. From their own site. So it is true-they are doing this! No lies there, this is just plain stupid of them and I too hope they crash and burn for it!:mad:

[Strawbs]

Originally posted by stix_kua
This makes me wonder if other companies could be doing this same exact thing...(Intel )...:eek: AMD has claimed recently that the optimisation code in a popular benchmarking suite (the name escapes me right now ...possibly Sisoftware or Sysmark) has been tweaked to favour Intel Processors and any AMD optimisation has been removed in the current version, so yes ...I would think nVidia may not be the only one employing "Corporate tactics" to get ahead of the opposition. :(

[Rugor]

Futuremark has indicated that Nvidia IS cheating.

Not good.

[Beeblequix]

Unfortuantly these events were inevitable. As pc world is a jungle. It goes by jungle law. Only the strongest survive. this will cut out some good co.'s tech.

So essetially we've now witnessed Compu-Techno-Darwinism (CTD for short)(hehehehe).

[Strawbs]

found This (http://discuss.futuremark.com/forum/showflat.pl?Board=techdisplayadapters&Number=2289289) link in the Futuremark forums. The new patch disables the cheat method and drops the GeForce FX5900 score by 24.1% (almost 1200 3DMarks).

It gets worse: ATI is also being investigated after the new build dropped the Catalyst 3.4 drivers' score by [edit]3%.

The thing is: not everyone will apply the patch and results posted away from futuremarks board may not be reliable as the build number will not be visible.

[edit] After reading This PDF (http://www.futuremark.com/companyinfo/3dmark03_audit_report.pdf) it seems that ATI's benchmark is within the error margin of 3% and they are therfore NOT cheating!!

[Rugor]

ATI did do something,

Their 1.9% Performance drop is entirely due to an 8.2% drop in Game 4 with the new patch engaged.

They were questioned on this, and stated what they'd done was reshuffle the instruction order to take better advantage of their hardware. There was no effect on visual quality.

Most would class this as an optimization since it doesn't change the output. ATI has also come forwards with a statement of what they did and why.

This is in stark contrast to Nvidia.

[$1500-P4 gamer]

I filled in some friends on the current nvidia situation! Since the start of it with 3dfx and then all the **** them complaining about cheating. THen they deliberatly do it themselves. Then they also have the huge flop fx. And top it off with EA alliance. Well lets just say most are turning away from nvidia now like its a bad trafic accident. Wonder how this will effect the hardcore gamers opinions. I know for me its down with nvidia!;) Gotta remember when it comes to cpu oclocking the # of enthusiests are small. BUT these cards are $ and only intended for one thing. Games. So therfore enthusiests are their biggest $. They arent pleasing us this way and it will hurt their sales as a result. We made'em well break'em if they dont streighten out.

[Bigjakkstaffa]

Remember though us enthusiasts only make up a small sales percentage. Indeed the 'ti' (performance) Nvidia cards only make up less than 10% od Nvidias sales (or so i read somewhere), they really dont have to be worried abotu us minority who know what were doing because, liek Intel, their so well in the sack with mass retailers such as PC worlds that they have a sure, mass market consumer base, they could loose all their 'enthusiast' customers tommorrow and still make massive profits in a year from the 'mx' (budget) type cards

--Jakk:t

[Rugor]

Yes the Ti cards make up a small proportion of their sales. But one of the reasons the MX did as well as it didi was because OEMs could say Gf4 Graphics card and make people think they were getting something fast.

The idea was to do the same with GfFX cards. Unfortunately if they aren't the one on all the magazine covers as the fastest card then the name won't help as much.

It's funny you mention Intel, since these days they sell as many or more graphics solutions than anyone else. Intel Extreme Slideshow is more common than anything else in new computers.

[$1500-P4 gamer]

My point was that the only ones that buy high end GPu's are the enthusiest like ourselves. So if they burn us-they might aswell NOT make any fast video cards anymore....Which is what leads the who's fastest race. Which in turn acts like advertising for the co. name like rugor said. So it all leads to Video cards being a whole dif ball game then CPU's.;) Take this quote for instance....

"Dean McCarron, an analyst with Mercury Research, said benchmark results are valuable for graphics chip makers if they want to claim general bragging rights for the fastest chip. But the results hold little sway with the hard-core PC game players who make up the bulk of the customer base for high-end graphics chips."

"What happens is that in the gamer market, the buyers are fairly sophisticated," McCarron said. "They're looking at real-world results--how the product performs on the games they're interested in playing. The overall benchmarks, like what Futuremark does, are more influential for the less sophisticated part of the market."

McCarron said manipulation of benchmark testing is common in the PC industry. "The best advice is to pick an application you're going to use a lot and find someone who's done testing using that," he said. "

From here.
http://news.com.com/2100-1046_3-1009574.html?tag=fd_top

[CrazyCrusher]

I just emailed nVIDIA to find out what their exact stance is on the 3D Mark 03 patch/So called cheat. A few sites already have some responses. Let's look at it:

Since nVidia is not part in the FutureMark beta program (a program which costs of hundreds of thousands of dollars to participate in) we do not get a chance to work with Futuremark on writing the shaders like we would with a real applications developer. We don't know what they did but it looks like they have intentionally tried to create a scenario that makes our products look bad. This is obvious since our relative performance on games like Unreal Tournament 2003 and Doom3 shows that The GeForce FX 5900 is by far the fastest graphics on the market today.

This is taken off of the read me that comes with 3.3.0

3DMark03 v3.3.0 is an updated version of 3DMark03. Hardware review sites discovered deliberate cheats in some drivers. These drivers identify 3DMark03 build 3.2.0, and render the tests differently than 3DMark03 instructs, in order to gain additional performance. Build 3.3.0 has been changed so that the test results remain the same, but the questionable drivers do not identify 3DMark03 anymore. The drivers now think 3DMark is a 3D application among others, and render the tests like 3DMark instructs. This produces a result that is genuinely comparable to other hardware.
--------------------------------------------------------------------------------

Warning: It is possible that ATI and NVIDIA drivers are designed to cheat and produce higher scores. So Both partys are in this cheat.

What exactley is wrong with optomizing drivers to run things better. I wish ATI would optimize their drivers to run games faster. I don't care if its "cheating".

Who is to say that if nvidia has a different/faster way to render 3dmark03, or any game for that matter, with no visable difference, why shouldnt they do it? I really don't understand. Im my opnion, if they could optomize things to run my games better, and were intentionaly not doing it. I would feel cheated.

Sure, if you swing the cam around to an off angle you can see things are messed up, but you know.. why do I care about an off angle? I will never be seeing it that way.

This benchmark as far as I am concerned is completley useless. I don't blame nvidia for pulling out and taking a stance against it, nor do I blame either company for optomizing drivers for it. Difference is, ATI is still in the beta program, so basicly, they are better at cheating it than nvidia is. You'll notice Futuremark called out nivida several times in the read me, while there is no mention of ATI.

BTW - I love the fact that nvidia not only admited they had done this, but stated that they were planning on doing so in the first place. ATI Apparently thought their optimizations would either go un-noticed due to all the attention focused on nvidia, or that since it only raised it "a little" it wasent as important. but yet again ATI have also been called aside.

[Rugor]

I don't think you understand what Nvidia is accused of doing.

First: Their cheats don't give the same visual quality.

Second: The clip plane and back buffer issues would never benefit any game. It's like producing a driver for Doom III, that only worked if you took one path through the level, but not if you decided to go a different way. What possible benefit could you get from a driver that would render properly if you went down the corridor to the left and improperly down the one to the right?

Third: What Nvidia has done is replace Futuremark's code with their own, what ATI did was re-order Futuremarks code. ATI's action is more an optimization than a cheat. Nvidia's action was a cheat and not something you could do to improve game performance. ATI's action would improve game performance without sacrificing visual quality.

Fourth: ATI has publicly stated what they did and why. Nvidia has stated "it's a bug." The sheer number of rendering errors, all of which serve to inflate the score is too great for a simple "bug."

To be honest CrazyCrusher I don't know why I just typed this out as your own post indicates you have no desire to believe anyone but Nvidia.

[CrazyCrusher]

if your looking to be on the nail as to what they did, below is what they have played round with, I know what they did, and im not taking any sides, It just makes you think, really what the hell are we buying. trust me I like Nvidia, always have but if it comes down to it ill end up moving onto ATI, if the only thing Nvidai can come up with is "its a bug", im not taking sides, nore am I a total Die Hard Nvidia Fan, Trust me, but im only posting what I read is all :)


Pixel shaders are functions, taking textures, constants, and texture coordinates as inputs, and producing colors as outputs. Computer scientists have this notion of extensional equality that says, if you have two functions, and they return the same results for all combinations of parameters, then they represent the same function -- even if their implementaions differ, for example in instruction usage or performance.

Therefore, any code optimization performed on a function that does not change the resulting value of the function for any argument, is uncontroversially considered a valid optimization. Therefore, techniques such as instruction selection, instruction scheduling, dead code elimination, and load/store reordering are all acceptable. These techniques change the performance profile of the function, without affecting its extensional meaning.

Optimization techniques which change your function into a function that extensionally differs from what you specified are generally not considered valid optimizations. These sorts of optimizations have occasionally been exposed, for example, in C++ compilers as features that programmers can optionally enable when they want the extra performance and are willing to accept that the meaning of their function is being changed but hopefully to a reasonable numeric approximation. One example of this is Visual C++'s "improve float consistency" option. Such non-extensional optimizations, in all sane programming systems, default to off.

3D hardware is still at a point in its infancy that there are still lots of nondeterministic issues in the API's and the hardware itself, such as undefined amounts of precision, undefined exact order of filtering, etc. This gives IHV's some cover for performing additional optimizations that change the semantics of pixel shaders, though only because the semantics aren't well-defined in the base case anyway. In time, this will all go away, leaving us with a well-defined computing layer. We have to look back and realize that, if CPU's operated as unpredictably as 3D hardware, it would be impossible to write serious software. Nvidia optimized these drivers and setting to reach higher FPS.


this is also a press release from ATI:

Despite still being a full Futuremark Beta member, ATI did not make it out of the report entirely unscathed either. There is a performance difference of about 8% in Game Test 4, that accounts for about a 2% difference in the final 3DMark03 score, between the new and old versions, indicating that although not visually different something was occurring on this particular test. ATI’s Director of PR, Chris Evenden, has since issued the following statement:
"The 1.9% performance gain comes from optimization of the two DX9 shaders (water and sky) in Game Test 4 . We render the scene exactly as intended by Futuremark, in full-precision floating point. Our shaders are mathematically and functionally identical to Futuremark's and there are no visual artifacts; we simply shuffle instructions to take advantage of our architecture. These are exactly the sort of optimizations that work in games to improve frame rates without reducing image quality and as such, are a realistic approach to a benchmark intended to measure in-game performance. However, we recognize that these can be used by some people to call into question the legitimacy of benchmark results, and so we are removing them from our driver as soon as is physically possible. We expect them to be gone by the next release of CATALYST."

[AllGamer]

Man Did you even bother reading the Audit made by Futuremarks?

http://www.futuremark.com/companyinfo/3dmark03_audit_report.pdf

(Page 4) they said 1.9% DROP! in the ATI, they did not say an inclease of 1.9% :rolleyes:

also you can see Samples of the nVidia ("bugs") Cheat and what problem it causes

:t

[CrazyCrusher]

yep I did read them,

Im not backing Nvidia guys, calm down, dont kill me here :), Im only saying, do we even know what we are getting, let it be ATI or Nvidia, If Nvidia can do it, who says there all not doing it? all im saying is Nvidia got cought simple as that, I like my Nvidia, but now im starting to doupt the power thats behind my TI4600,

and alos there is alot of tension between Nvidia and ExtremeTech, as you all know Nvidia did not Invite them to the opening of the FX family, so there is people out there that think ExtremTech is taking a Dig at Nvidia, but who knows, has Nvidia done anything things else to try clear this up? apart from the its a "bug thing" I guess we are going to have to wait and see what Nvidia do to try clear this mess up eh, It should be interesting to see it unfold, dont you think? :)

[bob05]

I am not going to belive (and always havent) ANY benchmarks. I only pay attention to acaual FPS ratings in games. Games are what I am going to be using on a graphics card, not benchmarks. If NVIDA gets 23 FSP's better on UT2k than ATI, then I know that the Nvida is better. 60 points or 60,000 points doesnt mean anything to me, in testing anything. Wheater it be CPU's, memmory, or motherboards. Benchmarks can lie, real life testing doesn't.

[CrazyCrusher]

I totaly agree, FPS in games is what I would go on, not a benchmark program, my posts may indicate that Im all for benchmarks and what FPS they give out, but its not ment to come of like that, I would have to say when it boils down to it, its how my games run, and how well the graphics looks.

[AllGamer]

i'm not taking sides...

but when it comes to Speed Test (what this boils down to) is to test it with what you use.

in this case, 3DMarks 2003 is for the moment the only accurate program to test the NEW video cards

when you test a 5900 or 9800 with a 5, 2, or 1 year old game, you are not really testing the video card, you are just testing how fast the game runs.

specially those older games

it's like testing a DX 7 game with a DX 9 GPU, of course is gonna run faster

so that's why 3D Marks 2003 is necessary, to test the new video cards with DX 9 codes.

by the time DX 9 games come out, video card for DX 10 is already out and so on

right now, we are still playing games from DX 8, no DX 9 games out right now, are using any of the new features set provided in DX 9

so your assumption that using old and current games to test a new video card is flawed

:t

[bob05]

Originally posted by AllGamer
i'm not taking sides...

so your assumption that using old and current games to test a new video card is flawed

:t

Maybe I didnt state what i meant clear enough. I'm sorry for that.

First off I want to say that if Nvidia truly cheated, then that was worng. I am not way taking sides, I'm telling you may take on things.

I know that we dont have DX 9 games out, but I really dont think that a lot people will go buy a Nvidia FX or a Radeon 9800 because of what they see on a benchmark reguarding technology that isnt avalible yet on games today. We dont even know how all the fetures are going to be used in the games. Maybe developers will see unessasary fetures in DX 9 and not add them in the games. Or maybe they see limitations in the DX 9 and have to have new features software rendered, which takes up a lot of system resoures. So the 3dmark gauge wouldnt be accuerate. And which every card you get, Nvidia or Radeon, you probaly can still use it (without terrible performance) through DX 11, let alone DX9. I know this becuase I use my Geforce 2 MX 200 PCI for DX 8 games, and I am not getting unplayablity. I dont get less than 30 fps (with the exception of Splinter Cell). And the MX 200 wasent even the top of the line for MX's (or Geforce 2 for that matter).

Another huge factor is price. If you go to a store and u see a Radeon 9800 for $100 or a Geforce FX 5900 for $200, u'll pick up the Radeon 9800. You know that the performance is almost equal, unless you are will to pay $100 more dollars for a few FPS difference.

Just my .02 :t

[Rugor]

I have a Radeon 9500 Pro. My brother has a Ti4600.

His card is a little faster without AA and AF enabled. Mine is faster with. He has more raw FPS. I have better image quality.

He got what he wants. I got what I want. All the screenshots I've seen show ATI's max image quality to be higher at the moment. I don't play fast games, so I want the IQ more.

Buy the card that gives you what you want.