Taken a step futher, this questions usually boils down to "is Roger Federer better than Pete Sampras?"
Federer is far and away more consistent than Sampras ever was. Since winning his first Grand Slam title (Wimbledon 2003) he's been in the finals of nearly every Grand Slam event. And even more impressive, Federer has been in the finals in twelve of the last twelve Grand Slam events! If we define Sampras' prime as 1993 thru 2000 when he won seven out of eight Wimbledon titles, he never came close to accomplishing anything like this-- and Federer's streak isn't even over yet. If you look at Sampras' results during his prime, he lost early in the French Open numerous times, had several early round losses in the Australian and US Open, and many late round losses, too. Since Federer started rolling at Wimbledon 2003 he's pretty much dominated like no other player in recent history. If Federer can hold even some of his current form for the next 3 years that will give him the same eight year prime that I've defined for Sampras, and would give us a slightly better view of the two careers side by side, but right now, it's hard to imagine Federer not continuing to dominate in Grand Slam events.
One argument could be made that Sampras had stiffer competition than Federer has. Maybe. Maybe not. This is perhaps an argument for another day, but one thing appears clear-- Federer doesn't lose to lower ranked players in Grand Slam events. Sampras did. If you look at Sampras' losses during his prime, he has many losses to no-name players. Federer doesn't. Sampras didn't have a problems beating the other top players. When he got into the finals, he usually won. It was getting to the finals that was difficult for him. Federer dominates lower ranked players.
So if you define the "greatest" player by overall consistency of their career, I think you'd have to give Federer the edge-- which is even more amazing because his career is no where near over. If Federer never wins another Grand Slam event, than maybe I would have to come back and revise this statement, but as it looks now, he's going to continue to win for the foreseeable future and will probably surpass Sampras as the person with the most Grand Slam singles titles.
Have a quick look a the Wikipedia entry for Sampras and Federer to see their Grand Slam results.
What most people mean when they off handedly say "Sampras was the greatest ever" is that Sampras' "A" game would beat Federer's "A" game if they played during the prime of their careers. Maybe. But maybe not. This is obviously an impossible argument to prove either way. Perhaps someone like Agassi could chime in with his two cents. Since he played both players in their prime it would be about as close to the truth as we could get.
Even though this argument is impossible to prove, let's break the statement down a little further and see if we can find any answers. If this "magical" match were to take place, what surface would they play on? Certainly not clay-- Federer would almost certainly win on Clay. What about grass? They both have exceptional records on grass. So let's say they play on center court at Wimbledon? I read an article years ago about Borg's five Wimbledon titles and how there was some speculation that during his run because he was such a rock star, Wimbledon wanted to give him a little extra chance to win. So they grew the grass a little longer than usual and let a little bit of air out of the balls. This slowed the game down a little bit and gave Borg a slightly better chance at winning. So if Federer and Sampras were going to play one "magical" match on center court at Wimbledon during the peak of their careers, what pressure should we put in the balls? And how long should the grass be? And who said it should be played on grass, anyway? It depends on who you want to win.
But if you're going to continue along with the impossible to prove "A" game argument, I'd like to throw in another player-- Boris Becker. Becker's consistency was obviously no where near what either Federer or Sampras' was. But when he was on his game, he dispatched the best in the world with ease. I remember a Davis Cup match where Becker blew Mats Wilander off the court-- and this was when Wilander was still playing good tennis. I remember a French Open quarterfinal match where Becker blew Chang away-- and this was just two years after Change had won the tournament. Becker's game was so large there was simply nothing Change could do, even on the slow red clay. In addition, if you really want to make the "A" game argument, what about Richard Krajicek? He actually did beat Sampras on center court at Wimbeldon when Sampras was in his prime (1996 Wimbledon quarterfinals). Or what about the huge serving Goran Ivanisevic? Krajicek and Ivanisevic had such huge serves, it's quite possible that there was one day sometime during their lives where they actually could have beaten both Sampras and Federer on their best days. Obviously they couldn't keep it up day in and day out, but if you're just going to say who would beat who on their very best day, it's hard to imagine anyone beating either Krajicek or Ivanisevic if they could sit out there and serve aces for fives hours. And it's also hard to imagine that there wasn't at least one day during their lives where they could have actually done it.
The point is that tennis is not a sport about one "magical" match and it's not a sport played on one surface. So clearly the element of overall consistency is a huge part of being the "greatest" which is why Becker, Krajicek and Ivanisevic are never mentioned in a discussion of the "greatest" of all time.
But lets take this one step further. Suppose we had both Sampras and Federer in their primes and we decided to play 5 matches-- the 4 grand slam matches on their respective surfaces (rebound-ace, clay, grass, and hard) and then a fast indoor court to round it out. As mentioned, it's hard to imagine Federer losing to Sampras on clay. So that's one that we can pretty safely give to Federer 1-0 Federer. But Federer has proven equally adept on all other surfaces so I don't think there's any "for sure" matches for Sampras. So Sampras has to win three out of 4 matches on the other surfaces. Maybe he would, but that's an up hill battle and I'd put my money on Federer to split the grass, rebound ace, hard, and indoor surface and then win on clay. 3-2 Federer.
But let's imagine that Sampras and Federer were the same age and peaked at roughly the same time. In my opinion at the end of their careers Federer would have more Grand Slam titles than Sampras. Maybe Sampras would end up with a better head-to-head record against Federer, but every time Sampras would make it to a Grand Slam final, Federer would be there waiting for him. There would be many Grand Slam finals during this time where Federer would simply beat someone else because Sampras had an off day and didn't make it to the finals. So even if Sampras had a slight edge in head-to-head matches against Federer, Federer would end up with many more Grand Slam titles. Which is what I think will end up happening in reality when Federer breaks Sampras' record of most Grand Slam singles titles.
Ultimately, since Federer's career is far from over, it's probably too early to declare him the "greatest" ever. In the next few years it will become more clear. But the fact that this argument comes up so much when Federer isn't even close to being done, simply points to the fact that he probably will ultimately be the undisputed "greatest" player of all time. Until some one even better comes along.