Straight up aggregation. Every month I scan Uncut, Mojo, and Paste. Some months, when it's available to me, I include Rolling Stone, Spin, and Q. I take every review that's listed on metacritic.com. I average all of those together, giving extra weight to metacritic according to the number of reviews listed there.
The average recording gets a score of around 71. An artist releases a disk. Say it's a an 80. They get 9 points, equal to the number of points that recording is above the average. Artist 2, say Kid Rock, releases a CD that averages, say, 12. He gets -59 points. I add these points together for all of the artists releases since Jan 1, 2000, and list them in descending order.
Please note, this is REPORTAGE, and not my personal opinion. For instance, while I don't hate Tom Waits, I don't own any of his recordings. So this is like an uber-critic listing of which artists get most favorably reviewed. It's not scientific, authoritative, or even necessary. It's just part of what I do in my part-time gig as OCD man.