Some thoughts on Windows 1903 + AMD's chipset drivers
Added 2019-06-19 03:17:26 +0000 UTCHi all,
We've been doing a lot of CPU benchmarking over the past two months. Unfortunately, we keep throwing away data because updates come out that seem important to test. We always try to pre-test as much as we can before new CPUs show up, but sometimes, that does mean we have to do the work twice if important updates launch.
We recently retested Windows 1903 without the new chipset driver, finding that all changes in performance were within margin of error and usual test variance. Because of this, after validating on 3 AMD CPUs and 1 Intel CPU, we decided to keep our existing test data.
After that, we saw that the new chipset drivers had been posted, and so again re-ran the tests with 1903 and the new chipset driver. AMD claimed major gains (upwards of 15% in a very specific scenario) as a result of improved clock responsiveness, but in our testing, we have been unable to reproduce these results in any of our normal games or production applications we test. All results are within margin of error and variance.
We are now evaluating how we want to approach the upcoming reviews, because sticking with 1903 means we'll have fewer CPUs on our charts (we'll have to throw a lot of data away and won't have time to update it all), but moving entirely to 1903 means it's updated. If we saw meaningful gains, we'd definitely make the move. For now, we are considering keeping our existing data (+/-1FPS in all games tested between 1903+chipset and the previous results), but then migrating the new CPUs to 1903+chipset drivers and validating their performance on the old test setup. This would allow the most CPUs on the chart while still getting the updates loaded for the CPUs that may be more likely affected. It's always a challenge of how many CPUs we can test versus how many changes happen between that pre-testing and launch.
We'll figure it out, but just wanted to let you all know that we've done the testing and have seen no benefit as of now. Intel is about +/-2FPS AVG, AMD is about +/-1FPS AVG in our testing. Production results are within normal error margins.
Edit: One more important thing - we will validate this again once the chipset driver is re-pushed by AMD. For now, only a few partners have uploaded it, so we do want to make sure it's the right one and not some mistake by partners.
Comments
"The best Windows keeps getting better".... ~Microsoft Windows Features page tag line... π©
PCS ProShop
2019-07-21 03:17:36 +0000 UTCNot really. However, I believe they've worked out enough of the bugs so I don't see perpetual USB "power irregular" and the other vague errors (preinstall). I'm so glad you (Steve Et Al) have been putting so much time, helping us plebes get a better handle on this update, you've saved me alone a TON of hours. I'm serious, if it weren't for the (what we used to call "Wire-Heads") on these alternative tech sources (YouTube, Reddit, GitHub, and of course Patreon), I don't think we'd have a chance! Anyway, "Reader's Digest" version.... THANKS! For ALL your outstanding tenacity and diligence! KUDOS!
PCS ProShop
2019-07-21 03:14:58 +0000 UTCWe've been wrestling with 1903 here too, actually believe (from all the semi-manic) MSDN & Microsoft "Migration Support" queue posts, i think the entire Microsoft Support/Development staff is filibustering for a global retrograde back to Windows 7!!! Ok, well maybe Vista Team will be awakened from cryogenic sleep pods to supplement support staff.... DOH!
PCS ProShop
2019-07-21 03:05:32 +0000 UTCi agree i feel that they must be holding back the advantage for some reason... to be honest thogh it dosn't quite make sence as you would think they would wan't to get it out asap so reviewers could have all advantages over intell as possible
Caleb Johnson
2019-06-28 14:02:51 +0000 UTCthat would be a nasty thing to do... oh well though... i mean why would AMD NOT want to have this out as soon as posible. you would think they would wan't the reviewers to be able to test it with all the improvements possible...
Caleb Johnson
2019-06-28 13:59:41 +0000 UTCOh was that Windows 7? I remembered there being a restriction but thought it was 10, my mistake. Yeah fair point, the amount of work required and the result from it ends up being pretty imbalanced, they have to trim sometimes. End of the day if people want older stuff, there's always some sites that have big databases of stats and they can do comparisons. It's not the same I know, but it serves a certain purpose.
James Holland-Hart (Bryn)
2019-06-20 08:49:42 +0000 UTCand the driver update venders uploaded, may have that one part disabled in it, a requirement for being able to upload it before the embargo lifts, maybe.
ZarconDeeGrissom
2019-06-20 00:56:51 +0000 UTCgood question. I know win7 updates wouldn't work (due to end of life activation limitations on M$'s end) with anything after the 7700k and FX stuff. I Don't think win10 has a similar block against older stuff. Also, I think the simple fact that there is only one Patrick and a lot of processors with each needing a lot of different tests done to have it on the charts, makes having everything that everyone would want a tad impossible, lol. So it's as much a lack of time thing as much as a Roi thing. B) I would love the FX8370 and R7 1700 and R5 2600 and R7 2700 to stay, however, I'm not expecting them to, and I'm sure others would want a 2500k, 4700k, 6700k, 7700k, 7740x, 8700k, 9900k, 9600k, 9900KeepSpending etc. lol. Only so much time in a day, and only so many Patricks, lol.
ZarconDeeGrissom
2019-06-20 00:51:13 +0000 UTCRegardless of how it shakes out always nice to see transparency and the thought process behind decisions.
SilkMonkey
2019-06-19 13:54:16 +0000 UTCThank you for your hard work.
2019-06-19 12:48:50 +0000 UTCFrom your description, 1903 and updated drivers look like a wash, with no gain, yet AMD is claiming 15%. What the heck are AMD talking about then? I could understand if 15% turned out to be closer to 10%, but 0% is odd. I think we're missing something.
2019-06-19 10:29:45 +0000 UTCWhy will some processors no longer be able to be used? Is that because of the thing a couple of years ago that only certain Intel parts would work with Windows 10 or versions of 10?
James Holland-Hart (Bryn)
2019-06-19 07:30:23 +0000 UTCIn an ideal world, each test result in a chart would have a link attached leading to a small sheet of meta data showing exactly under which circumstances a result has been achieved. Spectre/Meltdown mitigation would be a candiate for such meta info. For now I'd go with Blaank's suggestion. I would like to improve on it by suggesting to cross-validate exemplary results from older CPUs, let's say only the top/i7 ones sort of representative for a generation's architecture, on the newer software platform.
2019-06-19 07:14:21 +0000 UTCLike the others have said, put an asterisk& explanation and keep checking updates, retest later when new updates start to move out from error margins. Do you test with all vulnerability mitigations?
2019-06-19 03:49:48 +0000 UTCThat might be true. We'll keep an eye out and retest once it's available!
GamersNexus
2019-06-19 03:36:24 +0000 UTCI heard speculation that bios revision that is not out in the wild was also necessary to see the perf AMD is claiming? What say tech Jesus?
2019-06-19 03:34:33 +0000 UTCIf it's within margin of error, just upload all your data and put an asterisk next to 1807 chip tests and an explanation at the bottom that they were tested on windows 1807 and others were tested on 1903. It gives full disclosure which were tested in what way so people can factcheck it.
2019-06-19 03:29:07 +0000 UTCI miss snowflake as well. I miss my Lucy but I'm allergic to cats so she was my one and only. If there is no change then you can keep the old data but you must continue to check for changes in performance as time progresses.
2019-06-19 03:28:48 +0000 UTCFirst. Where is Snowflake?
Nory R
2019-06-19 03:20:53 +0000 UTC
