The State of Tech Journalism in 2024

Introduction

The Broader Issue/Problem

To understand the depth of this issue, we focused on five key areas: the overall decline in reviewer quality, the fake review industry on Google, the significant role of corporate-owned media in disseminating misinformation, the trust gap between corporate and independent publishers, and the issues with various product categories.

While corporate giants are often more manipulative, independents also struggle with credibility.

This report underscores the urgent need for a renewed commitment to transparent and honest journalism, ensuring that tech reviews genuinely serve consumers and restore public trust.

Trusted Publishers are Faking Product Tests

Add Introduction on the audience and captured by the corporate profit motive..Why are they included

Key Takeaway: When looking at monthly traffic numbers, four out of five of the highest traffic sites in our dataset are classified as fake reviewers. Together, these four publications alone bring in almost 176M monthly views – that’s about 18% of the total traffic in our entire dataset.

Key Takeaways

1. Consumer Reports
2. Good Housekeeping
3. Popular Mechanics
4. WIRED
5. Forbes

The top five publications by trust and traffic paint a very troubling picture. Here’s some fast information on them:

Notice a trend? Four of the top five publications by traffic have Fake Reviewer as a classification, meaning they are faking their testing claims. What does that mean? Simply put, 30% of the traffic going to these top five publications is going directly to fake reviewers.

For reference, that’s 138M monthly visitors out of the 974M total traffic.

Let’s dig into these fakers and find out why they were labeled this to begin with:

Consumer Reports

Many of you can remember a time when Consumer Reports was the trusted name in product reviews. Back then, if Consumer Reports gave a product the thumbs-up, you could buy with confidence. But these days, their reviews aren’t what they used to be.

As a nonprofit, they generate over $200 million annually, supported by nearly 3 million print magazine members and more than a dozen special-interest print titles covering autos, home, health, and food.

With over 14 million monthly online visitors and 2.9 million paying members, they’ve built a massive audience—and they’re taking advantage of it.

Their content is distributed across multiple platforms, including mobile apps and social media channels.

Nowadays, there’s so much circumstantial evidence indicating that Consumer Reports hides their test results and duplicates their reviews across different products. And their disappointing 45.49% Publication Trust Rating reflects that.

While their car reviews are still pretty reliable, other product reviews and buying guides in other categories lack the in-depth test evidence that once distinguished Consumer Reports.

We reached out to Consumer Reports in December 2023, and we learned that they’ll give the actual test results if you contact them for the data. So the testing is happening, but getting that information is very inconvenient.

Many reviews are templatized, repeating the same sentences across different products’ reviews. And with little to no visible test results to back up their claims, their reviews are unreliable.

Subscribers shouldn’t be receiving these basic reviews nor have to jump through hoops to see the test results—especially when Consumer Reports used to set the standard for transparency and detailed product reviews.

18 of the 24 categories we investigated earned the Fake Reviewer class. There’s repeated circumstantial evidence across categories that Consumer Reports is concealing their test results and duplicating their reviews.

Despite their clear claims of testing and a long-running reputation for “tested” product reviews, Consumer Reports earns a spot in the Fake Five because of a total lack of transparency and many of their reviews are duplications of one another.

The most glaring issue this lack of transparency creates is a complete absence of test results to support the claims they make in their reviews.

Additionally, their reviews often follow a cookie-cutter format, with the same sentences repeated nearly identical paragraphs used across different products.

Let’s take a look at their TV reviews for example, a category which earned a failing 47.60% Trust Rating.

TVs

TVs are the most difficult product category to test. It’s also expensive to afford all the proper test equipment, which probably isn’t an issue for Consumer Reports. However, why did they get that terrible 35.40% Trust Rating? 

Immediately in the subheadline, you see the author James K. Wilcox state that Consumer Reports tests a huge amount of TVs every year.

By seeing that subheadline, a reader expects to see test results in this TV buying guide from the best product testers in the world.

However, that’s exactly not the case as you can see in the screenshot below.

The Results section shows that CR rates TVs based on multiple criteria, many of which are very important, like picture quality, sound quality, and viewing angle.

At first glance, you’d think that this review of this LG TV looks pretty thorough and reliable. But a score out of 5 is pretty basic.

You may want to find out more info. How did this LG TV get a perfect score on Picture Quality? Luckily, there’s a tool tip that should go further in-depth and show the actual test results, right?

Unfortunately, when you mouse over the tooltips to get more information on their bizarrely simple scoring, there isn’t much beyond additional claims about the various criteria they tested.

Testing produces actual hard data, but there isn’t any here, just an explanation about what they were testing for.

Where’s the test data? You may try scrolling down the review to find those results. Then you’ll spot a Detailed Test Results section that must surely contain the test results to back up their scores. Right?

Even in the section that would seem most likely to provide “detailed results” there’s just qualitative language with no quantitative data. There are no color gamut graphs, no contrast screens, and no brightness measurements. Instead, we get statements like “brightness is very good” or “contrast is excellent.” 

If they were really testing contrast ratio (which they say is excellent) they’d provide a quantitative test result in an x:y format (Indicator 8.4 where we look for units of measurement). Consumer Reports never got points for Indicator 8.4 due to hiding their test results.

There’s also no mention of testing equipment like a luminance meter to measure the contrast or a colorimeter to measure color accuracy.

Even after you pay for a membership, you still don’t get any actual test results–you only get vague scores out of 5 for various criteria. You have to contact Consumer Reports to see the test results.

This is why they lost so many points in the Testing & Data Science category. Since we couldn’t see any of their quantitative test results, we had to mark their claim to test as “untruthful”. Their poor performance in that scoring category is what ultimately brought their Trust Score down below 60% in TVs. It’s the same story in many of their other categories. 

And even worse, their reviews are presented in this cookie cutter format that use templatized language.

Blenders was the worst case of these tokenized reviews that we came across. Check out the reviews for the Vitamix Professional Series 750 and the Wolf Gourmet High Performance WBGL100S below. The “Detailed Test results” Sections? Exact duplicates, as you can see by the highlighted parts.

It seems whoever put these reviews together didn’t even try to change the wording up between the reviews. 

Still don’t believe us? Take a look at these other sections reviews and see for yourself.

Let’s also take a look at two different reviews for an LG OLED77G4WUA TV and a Samsung QN77S90C, both 77 inch OLED TVs.

The two different TV reviews’ Detailed Test Results paragraphs above are the exact same except for one sentence. Both reviews highlight identical statements like “picture quality was excellent” and “color accuracy was excellent,” without offering any distinctive insights or data points for each model. These cookie-cutter reviews embody generic product praise rather than meaningful analysis.

And again, it’s not just TVs that’s the problem area in CR. Let’s take a look at some other categories where they claim to test despite publishing vague product reviews with no test results. The claim that they test products stands out most on pages where they ask for donations or memberships.

Routers

Routers is another problematic category with hidden test results and duplicated reviews. This category earned a pretty bad 45.20% Trust Rating.

Let’s take a look at the top of a single product review this time.

There aren’t any bold testing claims at the top of the review page for CR, unlike other sites.

But as you scroll, you’ll see the same Ratings Scorecard with the basic 5-point scoring system that they call “test results”.

Like we saw with how CR handled televisions, the test results section for their router reviews follows a very similar structure. Lots of different criteria are examined and supposedly tested and that’s how an item receives scores out of five per criteria. Unfortunately, the tooltips contain no useful information – just further explanation on what makes up any given test criteria without actually providing test results.

If you keep scrolling, you’ll see that the detailed test results for routers are even more anemic than they were for televisions.

There isn’t much to go off of here beyond qualitative explanations of how the router performed. There’s no information about actual download speeds, upload speeds, latency, or range testing.

Membership and Donation Pages

To top it all off, Consumer Reports promotes their product testing across their website, including pages where they solicit donations or memberships.

This emphasis on rigorous testing is out of sync with their anemic review content and lack of test data.

The Correction We Demand

For a brand that built its name on transparency, having to jump through hoops to get actual test results is frustrating and bizarre.

People expect real, tested insights—not vague claims or recycled templates. And when that trust cracks, it’s hard to rebuild.

If Consumer Reports doesn’t change course, they risk losing what made them different: the confidence readers felt knowing they were getting honest, thorough advice. Without that trust, what’s left? Just another review site in a crowded field.

The corrections we demand? Show the test results and stop copying and pasting the same basic paragraphs across different product reviews. Give users access to real numbers, side-by-side product comparisons, and actually helpful reviews.

That’s how Consumer Reports can reclaim its place as a reliable source and boost their Trust Ratings. Because trust comes from transparency, not just from a good reputation.

Good Housekeeping

If you’ve ever grabbed a product off the shelf with the Good Housekeeping Seal on it, you know the feeling. That seal wasn’t just a logo—it was a promise.

It meant the product had been rigorously tested by the experts at the Good Housekeeping Institute, giving you peace of mind, right there in the store aisle. But things aren’t the same anymore.

What used to be a symbol of trust now feels like it’s losing its edge. Since 1885, Good Housekeeping has been a trusted name in home appliances, beauty products, and more.

With 4.3 million print subscribers and 28.80 million online visitors every month, they’ve built a reputation that millions have relied on. And now they’re taking advantage of that trust and cutting corners in their reviews.

They barely passed in vacuums (61.40%), which is unexpected since they’re know for testing appliances – it should be way higher.

They also barely passed in e-bikes (61.53%), and routers (61.40%) thanks primarily to their actual use of the products, though there aren’t any test results.

In the majority of their categories, however, it’s hard to tell if the products were actually put to the test due to a lack of quantitative test results.

Their worst categories are air purifiers (13.40%) and drones (20.20%).

Good Housekeeping has a similar story to Consumer Reports–it seems they’re hiding the test results, which is a big reason for their awful 38.62% Trust Rating.

For a brand that once set the gold standard in product testing, this shift hits hard.

Without the transparency they were known for, it’s hard to trust their recommendations. And that’s a tough pill to swallow for a name that’s been synonymous with reliability for over a century.

The way that Good Housekeeping (54.8M monthly views) handled their TV reviews is part of what spurred the intense analysis we started performing on product review testing: their testing claims didn’t reflect in the text they were publishing. But the problems don’t stop at TVs – out of the 16 categories they claim to test, 11 were found to have faked testing:

Soundbars

Good Housekeeping makes an immediate claim about their testing in the title of the post, and has an additional blurb about it down below the featured image. The expectation is clear: these 9 soundbars have been tested.

Another paragraph dedicates itself to assuring the reader that testing is being performed by dedicated tech analysts who cover a variety of home entertainment equipment. The implication, of course, is obvious: soundbars are also covered.

The actual review portion, however, leaves a lot to be desired. There’s no data despite clear mention of testers giving feedback, including direct quotes. No maximum sound levels are recorded, frequency response isn’t noted, and there’s no indication they tried to measure total harmonic distortion. Ultimately, the testing claims fall flat without anything to back them up – instead, we just have qualitative language assuring us that the soundbar sounds good and gets loud. A small spec box accompanies the text, but specifications data isn’t testing data.

The dedicated “how we test” blurb isn’t any better. There’s plenty of mention of important performance criteria that’s supposedly being tested, but there’s no data or mention of tools being used to help facilitate this testing. It’s just a bunch of claims with no supporting data to show they did what they claimed to do.

TVs

The title doesn’t make any claims, so there’s nothing particularly out of place or unusual here.

The “How We Test” blurb makes a mountain of promises. Everything from measuring brightness with industry-standard patterns to investigating sound quality to look for “cinema-like” sound is mentioned. GH also notes they care a lot about qualitative performance criteria, in addition to the hard and fast numbers of things like brightness. Ease of use in day-to-day interactions with the TV is also part of their testing process. This is nice, but a TV that is great to use and extremely dim is not a particularly good television.

There isn’t much of use when you get to the actual review text, though. Beyond explanations of how good the TV looks (which is purely qualitative) there’s no data that suggests they actually tested. Mentioning how wide the color space is indicates they tested the gamut – but there’s nothing to suggest they did, because there’s no percentages given or gamuts mentioned. Bright whites, deep blacks – there’s no data to support this and no images either.

The Correction We Demand

The trust Good Housekeeping has spent generations building is at risk here. With a 38.62% Trust Rating, there’s a clear gap between the testing they claim to do and the evidence they provide.

If they can’t start showing their work—they need to get real about where thorough testing happens and where it doesn’t.

In some categories, like e-bikes and routers, their testing holds up. But in others—like air purifiers and Bluetooth speakers—it’s hard to tell if the products were actually put to the test.

If they can’t provide hard data, they need to state that their review is based on “research” instead of “testing.”

Like Consumer Reports, Good Housekeeping has spent decades earning consumer trust. But leaning on that trust without delivering transparency is a risky move.

They could jeopardize the reputation they’ve built over the last century. And once trust is broken, it’s hard to win back.

Recommend the Use of our true score by suggesting what Good Housekeeping is to be trusted for and when to use GR for the true score…

Popular Mechanics

Popular Mechanics has been a staple in science and tech since 1902, known for its no-nonsense, hands-on advice and practical take on how things work.

With a total reach of 17.5 million readers in 2023—split between 11.9 million digital readers and 5.69 million print subscribers—it’s clear that they’ve got a loyal following. Every month, their website pulls in 15.23 million visitors, all eager for insights on the latest tech, from 3D printers and gaming gear to electric bikes and home gadgets.

But lately, there’s been a shift.

Despite their history and resources, many of Popular Mechanics’ product reviews don’t quite measure up. They often skip the in-depth testing data that today’s readers are looking for, leaving a gap between their testing claims and the proof behind them.

This has landed them a disappointing Publication Trust Rating of 28.27%, raising doubts about the depth of their reviews. For a brand with over a century of credibility, this shift makes you wonder if they’re still delivering the level of rigor that their readers expect.

Popular Mechanics claims to test products across 13 categories, but there is strong evidence suggesting that thorough testing may not have been conducted in 11 of those categories:

What does this mean?

A pretty severe breach of trust. Coasting on your reputation as a trusted source and just letting your reviews begin to decay is exploitative, and means you’re just cashing in public trust and goodwill for an easy paycheck.

Claims to test need to be backed up, but it’s rare for Popular Mechanics to even offer you units of measurement in their reviews to indicate they’ve done actual testing.

Fixing the problem doesn’t have to be difficult: Popular Mechanics could simply change their reviews so they don’t claim to test when they aren’t. It won’t dramatically improve their scores, but they won’t be misleading the public anymore at least.

Alternatively, they could live up to their history and start producing good reviews with thoroughly tested products, offering the kinds of insights and data that made them famous to begin with. The choice is theirs.

Air Conditioners

Lets take a look at the reviews and product tests for air conditioners. To test this category well, it’s important to measure how long a unit takes to cool a space in seconds or minutes. 

Popular Mechanics makes claims to test right in the subheadline of their air conditioner buying guide and has a dedicated “Why Trust Us?” blurb that covers their commitment to testing.

The buying guide itself even has a small “How We Tested Segment” that promises that the air conditioners covered we tested in a real-life setting and involved taking multiple important measurements, like cooling throw and temperature drops. There’s a minor red flag in their blurb, however: they note that some models weren’t tested, and to compensate, they “consulted engineers” and “scoured the specs”. The former is interesting – the general public usually can’t speak to engineers, but what an engineer says and what a product does aren’t necessarily aligned. The latter point, “scouring” specs, is something anyone can do and doesn’t involve testing: it’s just reading a spec sheet, often included with the air conditioner itself.

Unfortunately, the segments dedicated to talking about how the product performed don’t have much quantitative measurements to support their testing claims. There’s no data showing the temperature readings they supposedly took, nor any information about how long it takes for an A/C to cool a room of a certain size. This is the reason they received a No for our AC testing question 8.5: Does the publication run quantitative tests for cooling and/or dehumidification speed? (seconds/minutes)? There aren’t even noise level measurements, just qualitative language saying, in effect, “Yeah, it’s decently quiet.”

For all the claims of measurement, there aren’t any measurements to be found such as. The spec data is the one place we can find hard numbers, but spec data is freely available to anyone and doesn’t involve any testing.

Vacuums

Popular Mechanics doesn’t make a testing claim this time, but their dedicated “Why Trust Us?” blurb leads to a page about their commitment to testing.

This time around, Popular Mechanics has dedicated space to the expert that is reviewing the vacuum cleaners, who is also making claims about testing a wide variety of vacuum cleaners. Models that were included on the list include “several” vacuums that the expert personally tested. “Several” is an important word here, because it means that not every vacuum on the list has been personally tested. In fact, “several” don’t even mean that most of them were tested. Including review data to help make picks isn’t bad.

There’s a lot in this image, but all of comes down to one thing: there’s no test data. Despite claims of personally testing the vacuum, there’s nothing in the actual guide to suggest they did. Where’s the data on how much noise the vacuum makes? How much debris it picks up or leave behind? Even the battery life, something that requires nothing more than a stopwatch, isn’t given an exact measurement, and instead is given a rough approximation. Sure, the battery can be variable, but tests can be run multiple times to get average battery life, and we don’t see that here.

So what’s next for Pop Mech?

Pop Mech does a poor job of living up to their testing claims because there isn’t anything on the page that really hammers home that they did their homework and tested everything they said they tested. This ultimately damages Pop Mech’s authority in practical, hands-on advice.

To regain trust, they should commit to sharing clear, quantitative results from their product testing, making sure that their readers know exactly how a product performs. Otherwise, we ask them to change their “testing” claims to “researched”.

WIRED

If you’ve searched for the latest tech trends, you’ve probably run into Wired. Since 1993, Wired has made a name for itself as a go-to source for everything from the newest gadgets to deep dives into culture and science.

With 21.81 million visitors a month, Wired reaches a huge audience, shaping opinions on everything from the latest gadgets to where artificial intelligence is headed.

They cover everything from laptops and gaming gear to electric bikes and smart home devices, making them a go-to for curious tech fans.

The problem is, despite their influence, a lot of Wired’s reviews don’t dig as deep as today’s readers expect. With a 32.36% Trust Rating across 29 categories, their recommendations don’t exactly inspire confidence.

Their reviews often skip key testing data or even real-world images of the products they claim to test, leaving us wondering just how thoroughly these products were reviewed.

It’s not uncommon to see their reviews skip out on hard testing data or even real-world images of the products they claim to put through their paces.

Over time, Wired’s focus has changed. Their writing is still engaging, but their product reviews have become more surface-level, leaning heavily on impressions instead of detailed metrics.

That’s why we included them in the Fake Five—their massive online reach gives their reviews weight, but without solid testing data to back them up, it’s hard for readers to fully trust what they recommend.

For a brand that once set the bar in tech journalism, this shift has been frustrating for readers who expect more.

To reclaim the reputation they built their name on is the same as it is for many of the Fake Five: show your work. When you claim to test, provide proof. Photos, units of measurement, real-world undeniable proof you’re actually testing products.

It’s difficult to test thoroughly, but everyone wins when proper tests are performed. Alternatively, they could simply stop claiming to test. Their Trust Rating won’t improve much, but it’s better than lying, and that’s worth something.

Gaming Mouse

Wired makes a testing claim right in the subheadline of their gaming mouse buying guide.

These are specs, which are always great to include, but specs don’t indicate you tested. You can get them from a product description or the back of the box, so we can’t consider this testing.

This is close to a test – after all, battery life is extremely important for wireless mice. However, the fact that the language around the time is so evasive leads us to have doubts. If you’re measuring battery life as a proper test, you’d have not only the time in hours but also minutes. This is what constitutes a good, rigorous test: getting good data that is concrete, and not so approximate. Unfortunately, this is also the battery life claimed by the manufacturer, which is commonly a “best case” estimate under specific conditions that the user might not be able to replicate. Including information like the response time but no proof you tested it also doesn’t help build a case for having actually performed any real testing.

The problem with Wired’s buying guide is that there’s nothing to support their testing claims. If these mice were indeed tested, they’d have concrete battery life, click latency, DPI tests, and software customization covered, but we don’t get test data for these criteria, just specs. Mentioning the wireless receiver boosts the polling rate just doesn’t cut it, because there’s no explanation of how they got this data. Buying guides tend to be brief, but they can link out to reviews or test data to support the claims without sacrificing brevity.

Webcams

The top of the page doesn’t show us any testing claims, just a basic subheadline, and a nice opening image.

The testing claims roll in once we hit the opening paragraph, and they’re even comparative in nature, with the testing claims suggesting the camera is among the nicest of the ones they’ve tested. The pros section suggests testing too, with key performance criteria mentioned. Despite what the “test webcams” link might have you think in context, it’s actually a link to a “best webcam” guide, not to a testing methodology.

Multiple highlighted portions in this paragraph all suggest not only usage but clear testing. The autofocus in particular, one of the most important aspects of a webcam, is put through its paces and does an excellent job of maintaining focus up to 4 inch away – or so the reviewer claims. But there’s no imagery to showcase this.

Instead of imagery that gives us an idea of how the camera performs (for example, showing off the excellent autofocus), we instead just get stock images from the manufacturer. This doesn’t build a convincing case for the testing claims that Wired is making in this review, and it casts serious doubt over the whole review. While the text suggests that the webcam may have been used, the total lack of any kind of testing imagery and data makes it very hard to believe.

Conclusion

For a brand that’s all about tech deep-dives, Wired has some serious issues in how it backs up its product reviews, and with its 32.36% Trust Rating, the gaps are starting to show. To get back on track, Wired should focus on bringing more transparency into their testing—think real photos, hard data, and detailed results. This would help them reconnect with readers who want the facts to make smarter buying decisions.

Forbes

Forbes, founded in 1917 by B.C. Forbes, has been a cornerstone of financial journalism for over a century, renowned for its trustworthy coverage of finance, industry, and business.

The magazine earned credibility with business professionals through initiatives like the “Forbes Richest 400” and by maintaining high standards of factual accuracy.

However, over the past five years, Forbes made a calculated move to leverage its brand equity and trust to gain prominent placement in Google’s search results, expanding into areas like “best CBD gummies” and “best gaming TVs.”

This strategy, known as “parasite SEO,” has led to reviews that lack the depth and expertise Forbes is known for, earning them a terrible Publication Trust Rating of 34.96% that raises concerns about the credibility of their non-financial content.

With 65M monthly visitors, Forbes lands a Fake Reviewer Classification with a shocking 10 categories featuring faked testing out of the 21 they claimed to test in:

Their Trust Rating reflects this poor performance: at 34.62%, even if they cleaned up their act, they’d still be deep in the red and would have a long way to go to get themselves to a more respectable classification.

Too often is Forbes claiming to test without actually backing up their test claims, and if they want their product reviews to have the same authority as their financial coverage, there’s a lot of work to be done.

Testing, providing imagery to prove testing, real measurements, solid analysis – it’s a hard road to legitimacy, but worth walking it when everyone stands to gain from it.

Forbes has a lot of ground to cover though – not only do they have a substantial number of categories they’re faking, they’re simply not in the majority of the categories they cover with their reviews, which means they have to ramp up testing across the board.

Alternatively, they could simply stop claiming to test altogether and instead note that they’re publishing “first impressions” reviews or simply research reviews based on data from around the web. They’ll have stopped lying, but they won’t earn substantial a Trust Rating for doing so.

TVs

This testing claim is right in the title, front and center, so everyone can see it.

The text here is worrying. Testing is said multiple times, but in the same breath, Forbes also says that LG doesn’t share what the actual nits measurement is. Part of testing is to do that yourself – grab a colorimeter and measure the actual brightness. Publications that perform rigorous testing have the equipment to test, but it is clear from this text that Forbes isn’t actually testing – they’re just using the TV.

Methodology is huge for testing – it’s how you get consistent results and provide meaningful information to the reader. The lack of numbers offered here is the first red flag. While the inclusion of a photo of the distortion is extremely helpful and absolutely a step in the right direction, the important thing about viewing angles is the angle. Showing that distortion can occur is good – but not giving the actual angle this happens at is a major misstep.

We get more claims about testing and expertise here, as well as a few notes on what the reviewer finds important: great picture, solid refresh rate and console compatibility, along with good audio quality. The issue is that “great picture” is a multi-faceted thing. It’s an entire category of performance, with multiple criteria under it that all have an impact on the overall picture quality. Brightness, color gamut, contrast… the list goes on, and it’s a very important list for getting a complete “picture” on picture quality.

The highlighted segments above are huge red flags. There’s just no data in the previous text of the review to really warrant to support the claims being made. Things like color gamut, brightness, contrast ratio, and EOTF are all testable, measurable and important, but none of them are mentioned in the review. Instead, we get confirmation that specs were cross-checked and warranties were looked at, neither of which constitute an actual test. Arguably, the most concerning part is the claim that not only is this testing, but this testing is enough to recommend that the televisions will last. Longevity tests can’t be done in an afternoon – they take time; months at the minimum, and years ideally. There’s no doubt Forbes actually used the TVs they got – there’s plenty of photo evidence of them being used. But they didn’t test them at all.

Gaming Headsets

The testing claim from Forbes is (once again) right in the title of the post. Forbes even makes the claim that they’ve put these headsets through their paces. Unfortunately, using a headset for hours isjn’t quite the same as testing it – at least not entirely. There’s plenty you can learn about from a headset by using it, especially given how much things like sound are a matter of taste, but other things like maximum volume and latency are firmly in the realm of measured testing, not simply using a headset while you play a few matches of CoD.

The first red flag comes up with the battery life. After stating how important battery life is, we simply get estimates on battery life, instead of actual testing data that shows how long the battery lasted. And for how much the reviewer talked about how important so many factors of the headset were, to get not only an approximation of the battery life, but also no further concrete numbers (like maximum volume) is disheartening.

Ultimately, the biggest issue is the lack of measurements from Forbes. For all the subjective qualities a headset has (microphone quality, sound quality, comfort, feel) there are multiple objective qualities that require testing to determine. Battery life, latency, and range are all objectively measurable, but no data is provided. How far can you go with the headset before it cuts out? Without the data, it’s hard to say that Forbes actually tested any of the headsets it claims to have spent so much time on – though they definitely did use them. Even when Forbes seems to be providing an actual measurement (like charge time) the numbers match up one to one with what’s listed in a spec sheet from HyperX.

Conclusion

Simply put, they’re not worth trusting until they can get real testing results in front of readers and actually deliver on their claims of testing. They could also simply remove references to testing from categories where they were found to be faking it, the less you test, the less you can climb.

Most Trusted Publications

The most trusted corporate-owned publication is still less trusted than the fifth most trusted Independent publication.

There is an interesting trend among the most trusted publications: they cover very few categories. Eight of the ten covered just one category, which suggests that focusing on a single category can improve overall Trust Rating. However, it is also possible that covering a few categories simply gives a publication fewer chances to “mess up.”

Key Takeaways

Based on this graph, however:

And on this one:

The data suggests that this isn’t the case. Low coverage appears to be more strongly correlated to poor Trust Rating over strong Trust Rating. In fact, the strongest overall number of categories to cover is suggested to be 16. However, that is how many categories RTings covered, which is notable for having an average Trust Rating of nearly 100%, and is one of only five publications that covered that many categories. The outsized footprint it has on the average Trust Rating for publications covering this many categories is therefore obvious.

When it comes specifically to independent publications, the five most trusted are noted below:

Meanwhile, the five most trusted corporate publications are:

On the whole, the five most trusted independent publications have higher publication Trust Ratings (which we derive from average together their performance across all the categories they cover) than the five most trusted corporate-owned publications. In fact, the most trusted corporate publication is still less trusted than the fifth most trusted independent publication.

Least Trusted Publications

As stated in the previous section, the number of categories covered does not seem to strongly align with higher average Trust Ratings. As the graph above illustrates, the least trusted publications in our dataset with significant traffic cannot break even 10% Average Trust Rating across the categories they cover. Many of these publications cover just one category, too.

The Independent Testers

Intro and thesis of a mixed bag It’s possible to achieve high trust without raising capital. The independent publishers like Rtings.com

Key Takeaways

1. You have a better chance of getting useful information from an independent publication – but not by much. 6.8% of the independent sites we researched are trustworthy or highly trustworthy, while just 4.9% of corporate sites are trustworthy (and not a single one is highly trusted.)

2. The only Highly Trusted publications are independent, and there are not a lot of them. There are just 4 publications that managed a “Highly Trusted” classification – 0.8%.

3. Among the aspects of trust ratings, Data Science contributes most significantly to independent publishers’ scores, followed by Visual Evidence. This suggests that independent publishers may excel in providing data-driven, evidence-based reviews.
   
4. The wide range of trust ratings among independent publishers (0.00% to 102.20%) indicates substantial variation in trustworthiness. This variation might be explained by factors such as resources, expertise, niche focus, and individual publication practices.

Definitions

For this analysis, independent publishers are defined as tech review publications that have not raised external capital, have not acquired other sites, are not a division of a larger conglomerate, and are not publicly traded companies. In our dataset, these publishers are marked with a “Yes” in the independent column.

Note: these folks are probably faking to survive and compete. Corp might be the ones to blame.

• Takeaway: The chances of independent publications being fake reviewers are distressingly high. 39% of the independents in our dataset are fake reviewers, with testing claims that are not supported by testing data and custom imagery.

We have around a 60/40 split when it comes to Independent and Corporate publishers, skewed slightly positively towards indies. Here are some fast facts:

• Of the 294 independent publishers we have in our dataset, 115 of them are Fake Reviewers. That means 39.1% of indies are fake testers.

By contrast, of the 202 corporate publishers in our dataset, 109 of them are Fake Reviewers. That means 53.9% of corporate-owned publishers are fake testers.

Key Takeaway: The only Highly Trusted publications are independent, and there are not a lot of them. There are just 4 publications that managed a “Highly Trusted” classification – 0.8%.

They perform exceptionally well, however, with an average Trust Rating of 98.8%.

Takeaway: Corporate publications dominate web traffic despite being extremely unhelpful. Of the 973 million total monthly visits every site in our dataset sees combined, corporate publications receive 85% of that traffic.

The monthly traffic for the 202 corporate sites we covered is 827M – by contrast, indies have 294 sites and only managed monthly traffic of 145M. Corporate sites see, on average, 5.7x more traffic.

Example 

Limitations:
No video reviewers included in this analysis. 

Product Categories in Focus

Trying to do research on products has become more difficult over the last several years. In addition to the ever-changing landscape of search results on the part of Google’s search empire, there has been a continuous shift (mostly a decline) in the quality of reviews being published by outlets. This isn’t to say they’re being written more poorly, but rather that many reviews now make testing claims that are either poorly supported or entirely unsupported.

Assumptions: we assume that the 30 categories we assessed are a good representation of the tech journalism. We recognize that there are many other categories to cover but given the time tables and cost, we felt these 30 are an accurate representation of what is occuring in the industry. 

Implications of our findings: almost half the time you’ll find a fake review when you’re on the web. Be it via Google or another source. However, despite this, we also found that even though pubs are faking their tests, there is always the potential of a publisher copying another website’s tests. The problem with this, though, is it could very well be the “blind following the blind,” and it’s very difficult to quanity who is doing this. And thus undermines the entire ecosytem. 

Impetus and why fake reviews exist: the economics of the matter dictate that some pubs. Meaning, testing is pricey and to get an ROI as a publication, or a better ROI on one’s investment, it’s cheapers to copy or fake tests. 

Gaming 

For this analysis, we focus on gaming hardware and peripherals, specifically gaming chairs, gaming headsets, gaming monitors, and gaming mice. These products form a crucial part of the gaming experience and are frequently reviewed by tech publications.

Key Takeaways

1. Gaming-focused products have serious problems with untrustworthy reviews.  Nearly 60% of gaming tech reviews are untrustworthy, including the 18% of gaming tech reviews that show signs of faked testing. Biggest Culprits are IGN, GamesRadar and PCGamer earning low trust ratings.

2. Trusted publications that cover gaming products are only decent. On average, a trusted reviewer earns a Trust Rating of 76%, which puts it in the bottom half of the range Trusted publications can sit in (70 – 89%.)

3. Gaming monitors are disproportionately plagued with fake testing and untrustworthy publications. Nearly 69% of tech reviewers who covered gaming monitors were either guilty of faking their testing or were simply not trustworthy and did not offer useful information.

Home Office

The home office segment of our analysis focuses on a few important home office products. These include computer monitors, keyboards, mice, office chairs, printers, routers and webcams. We feel these products are the cornerstone of any home office when it comes to tech, and they’re the most likely products to be reviewed by any tech publication.

1. Computer monitors and keyboards have huge problems with fake reviews and fake testing. 42% of the reviewers covering computer monitors and keyboards showed evidence of faked testing.
2. There is an alarming amount of untrustworthiness surrounding keyboard reviews. 92% of all of the reviewers in our dataset that covered keyboards were either faking their testing or were untrustworthy. Only a single publication earned a trust rating greater than 90%.
3. Almost half of router reviewers are worth trusting to some extent. While reviewers with more mixed Trust Ratings make up more than half of these reviewers (34 of the 57) this still leaves routers as the category a consumer is most likely to find reviews and reviewers worth trusting on some level. 

Small Appliances

This product category is pretty wide as far as the types of products it can cover, but we focused on a small set that are frequently covered by tech publications. The categories include air conditioners, air purifiers, blenders, coffee makers, fans, robot vacuums and vacuum cleaners.

1. It’s difficult to trust publications for useful information about health-sensitive devices like air purifiers. Three out of every four publications covering air purifiers produce fake reviews or untrustworthy ones, either because they fake their tests or do not perform any at all. 23% of the sites we analyzed that covered air purifiers faked their testing, meaning you have a nearly 1 in 4 chance of being outright misled.
   
2. A little over one in three vacuum reviewers have serious issues with faking testing. 36% of the reviewers analyzed exhibited clear signs of faked testing and fake reviews.
   
3. Air conditioners are the most likely place to find trustworthy reviewers, but none of them are “Highly Trusted”. Almost 27% of the reviewers covering A/Cs are worth listening to, almost half of those are only “Mixed Trust”.

Audio & Video

The audio and video category is a much more focused category that has far fewer products in it, though it also features some of the most difficult to test products across all the categories we researched. Televisions, headphones, bluetooth speakers, projectors, soundbars and speakers all live in this product category, and many of these require specialized equipment to properly test.

1. Problems with fake testing and fake reviewers run rampant in audio/video tech reviews. Of the tech reviewers that covered audio & video equipment, 42% of them were faking their testing.
   
2. Fake reviews are a major problem when it comes to projector reviews. Over 66% of the tech reviewers who covered projectors showed signs of faked testing.
   
3. You can’t trust 4 out of every 5 TV reviews. Over 82% of the tech reviewers covering televisions have serious trust issues, either because they’re untrustworthy or show clear signs of fake testing and fake reviews.

Emerging Tech

The “Emerging Tech” category covers drones, e-scooters, 3D printers, and e-bikes. As the name of this category suggests, the products living here are “emerging” in nature. Many are new technologies that are still seeing major refinements, like 3D printers, while others are categories that are becoming more accessible and affordable to your average consumer (again, 3D printers would fall into this umbrella, but so do drones, e-bikes and e-scooters.) These categories are also unique in that they have very diverse testing methods, many of which are still being pioneered. Drones and 3D printers, for example, are still be iterated on and see dramatic improvements with each “generation” of the product, be it in flight time and stability for drones, or resolution and complexity with 3D printers.

1. Products considered “emerging” tech have the most trustworthy reviewers, but that still only comes to one in three. Almost 59% of the reviewers covering products like e-bikes and drones aren’t trustworthy or exhibit signs of fake testing. In fact, 15% of the tech reviewers in the dataset are classified as fake reviewers.
   
2. You’re most likely to run into good reviewers and good data when reading reviews about electric bikes. 36% of the reviewers in our dataset were at least “Mixed Trust” or higher.

True Score Insert

Next Steps

Using the True Score to buy the best tested product for your needs.

Is AI Content a Problem?

Can the industry clean itself up?

Definitions
Table Showing Distribution of Trust Ratings that includes Publication Name

Visualization
Assumptions: