I know you and I know you don't have anything better to do this weekend than read an eight-page article that gives the blow-by-blow behind the scenes action of a computer match that isn't going to happen. Coincidentally, just the other day IM David Levy sent me just such a document! It details the negotiations behind a planned match between the programs Rybka, by Vasik Rajlich and Junior by Amir Ban and Shay Bushinsky. (Full b-ball disclosure: I know exactly how many free throws out of 10 Shay can hit and I once went to a WNBA game with his sister.) It was intended to take place alongside the Mexico City world championship in September. Pics of all the programmers here.
You'll remember Rybka offered a challenge to FIDE and the winner of their "Ultimate" computer match last June. This match wasn't really that one. No handicap or winner-take-all. The gist of this epic, which is just begging to be set to music, I might add, is that everyone on all sides supported a match. FIDE and the Mexico organizers were on board. It would be between the 2006 (Junior) and 2007 (Rybka) world computer champions. Rybka dominates the computer vs computer rating lists and Junior just beat Fritz in a match in Elista. The company that distributes Rybka, Convekta, was offering to guarantee a $100,000 prize fund. Convekta's Sergey Abramov represented Rajlich in the negotiations. The major issues in the negotiations were 1) Time frame, 2) Tiebreaks, 3) Financial guarantees, 4) Hardware. The first three were settled rather quickly, the first two conceded by the Junior team, but the fourth became complex. The main issue was whether or not the computers the programs ran on would have to be on-site or if remote was acceptable.
To summarize Abramov's position, such a high-stakes match cannot allow any possibility of cheating. As such, the machines must be on site and available for inspection. The Junior team responded that there is a long tradition of using remote terminals and that this was the only way they could be sure to be playing on the strongest hardware possible. The expense of bringing/arranging a top-level machine in Mexico would be prohibitive. Eventually Abramov accepted that position but wanted a set of controls to be put in place to assure the probity of the contest. This included providing the log files and copies of the program to the arbiter after each game, something that at least at some level has been done many times in the past. Abramov then asked that the log files and engines also be provided to the opposing team after the match was completed, something that Levy describes as out of bounds.
That turned out to be the sticking point. It was either on-site hardware or handing over executables and logs. Eventually they overstepped the July 30 deadline set by Abramov without a resolution. You can read the full Levy article in all its original formatting glory here in PDF format.
I'm not really torn up about this. More chess is always good, but as I said about the Junior-Fritz match, this isn't high-stakes chess. Both engines are commercially available, if not in their absolutely latest versions perhaps. It's a fun academic exercise to play an annual championship between the comps, but it's not really much of a spectator sport. You might make the argument that with computers being as strong as they are this is the best chess ever played on the planet. But making that argument would make you a silly person. Compare it a potter making a vase. A machine can crank out countless perfect replicas that have fewer flaws and that possess perfect symmetry. But is it art?
The slight imperfections are what make human chess a sport. Computers make mistakes of course, even big ones, but it's more of a science in finding out how and why the algorithm coughed up a hairball. That's not sport, that's debugging. As a programmer I'm all for the argument that great coding has artistic merits and this is even more the case with chess programming. I've oohed and aahed over elegant code the way we coo over a brilliant mate. That's art. Still, when it comes to chess matches, the principles of competitive tension, winning when it counts, and results being what matter are alien to machines. I watch, I'm interested in the games, but it's hard to care. Yes, the programmers get worked up plenty, but this isn't the same. Watching NASCAR is bad, although millions apparently do so. Imagine watching NASCAR with the cars being remote-controlled. Or driven by robots as the robot designers cheer from the sidelines. Not so much.
As an aside, unless the machines are of roughly similar processing power there is even less of a point in holding competitive events. I'm aware of all the reasons this can be difficult, but since most progs are running on standard PC architecture these days it's not hard. (I.e. a program that can only run on a Cray would require some serious equivalence calculations. And a hardware-based program like Hydra is another thing altogether.) It also makes marketing sense to want them on the fastest machines available. Faster machine, better chess. But if one is on 16 cores and the other 8, it's a bit silly to talk about the programs and sport. It's like having a Ferrari vs a Hyundai to find the best driver.
You've got a stray http in the link to that PDF file. Yes, I actually clicked on it. No, I don't know why either.
With all respect to other engines, Rybka is the first widely available program which as a rule gives very sensible evaluations, not just counting pieces and calculating tactics. All GMs whose opinions are known to me can say more or less the same. Normally I would prefer to remain silent, but after FIDE supported the computer match in Elista without participation of Rybka, it is only fair not to hide facts, which are known to professional players. As FIDE (which is not a privat firm) was behind the match, they should have invited a really strongest program, IMHO.
Instead of reading the boring document this weekend you could perhaps follow the live transmissions from the "arctic chess challenge" tournament which has become a decently strong event now that Magnus Carlsen has decided to participate,
Very nice analogies, Mig. More than made up for the poor piece earlier in the day about the American sharing first and losing in play-off, etc.. Carlsen was first on tiebreaks even without the tiebreak games and should be given credit as sole champion just as a Kramnik is given when he wins an event on a tiebreak. Viva Brooklyn and Viva America do not make very neutral reading material..
Thanks. I was probably overdoing it with the comparisons but they're like potato chips, you can't have just one. See? As for neutral, go read a Swiss blog! I'm interested in chess, not neutrality. And I'm usually very good about not giving much credit to system tiebreaks, which are usually worthless and don't decide any money anyway. I always talk about shared firsts in round-robins. Rapid is bad enough, but an armageddon game? Please. It's a necessary evil in KO events but let's not call it anything more than that. Carlsen winning their head-to-head game, or even just winning more games overall, would be better tiebreaks in his favor.
Given that Rybka would have been the favorite in this match, I'm really surprised by their apparent shirking. Assuming that they didn't really expect the Junior team to cheat, as Abramov himself says, what could have been going on behind the scenes?
Or do they just not trust Intel? I'd love to get this match hosted in the AMD Developer Center. Would Convekta be amenable to that?
Why doesn't the Rybka team just send a rep over to the location where Junior's hardware is located at Rybka's expense? Or is that cost-prohibitive too?
My interest in engine vs. engine chess matches decreases when anything other than a commercially available version of either program is used.
My interest drops further when they use hardware beyond a high-end PC (two CPU's per computer is okay).
- - - - - -
I presume Rybka could publicly stage this match without Junior's permission. However, suspicion of inherent bias, and a less dramatic atmosphere, would make the exercise pointless.
Rybka will compete in Mainz Germany in mid-August, at the Chess Tigers' chess960 computer tournament. Fritz9 & Fritz10 are chess960 enabled, but they choose to not compete in Mainz.
I don't watch American football so I can see a ball move from one end of the field to the other. I watch it for the human sports drama and action. Watching one man trying to avoid 11 others can be exhilarating, and yes even artful.
Chess too has human drama, and action, but like football I have no interest in watching a mere end to a means. Watching a human find a way to maneuver an army of chess pieces to capture the enemy king is far more exhilarating, and yes even artful, than watching two computer programs calculate math problems for hours.
Great simile/analogy! Not only is human chess competitions more akin to our own experiences in the game, but computer chess is simply alien and remote. I find my road bike more interesting than someone elses ferari.
I believe that a sponsor, e.g. Intel, AMD, Dell, etc. should provide two identical machines for such a match. I agree that it's pretty useless to compare the programs that run on different hardware.
On the other hand, how much interest is there in this when you can run a match between the two engines on your laptop anytime you want?
Ivanchuk rolling !!!! http://reports.chessdom.com/villarrobledo-day-1
Ivanchuk never fails to amaze! This guy is so good none can describe! Go Chuko!
I don't really see how more imperfections make for more artistic chess, unless it has reached the point where top level computers produce an abhorrent number of draws in head to head play. But it hasn't has it?
Why don't they resolve the hardware problem in this way: Play two matches concurrently, one on the Junior team's preferred hardware and one on Rybka's preference. Add up the number of points from these two matches to determine the winner. Set reasonable limits on the number of CPUs/cores and RAM size.
Am I wrong, or does team Junior's claim of prohibitive expense to bring the machines on site sound dubious? Maybe they realize they need a sizeable hardware edge to compete.
Cynical Gripe, I don't think so. Also, maintaining those servers in Mexico would be 100 times harder than in Intel.
Taking into account that everyone knows that Junior team REALLY would not cheat, I cannot find any explanation other that Convetka didn't really want to fund the match.
Assuming someone REALLY would not cheat and taking provisions to make sure it isn't possible are two different things. Sure, the honour system is great and all, but when reputation (and money) are at stake it's not enough. I don't see why it would be so hard to maintain, say, a maximum of 8 servers, in Mexico. Do you think the Junior team would have agreed to an equal hardware stipulation?
Cynical Gripe,
If I remember correctly Junior team suggested Rybka team to negotiate with Intel and ask for a second similar server. I am almost sure Intel would not refuse ;)
Also, in the initial proposal Rybka was giving draw odds and one point handicap. At CCCL or something like that rating difference between 4CPU Rybka and 2 CPU Rybka is not too large, and is going to be smaller as you add processors. If they really were sure they could win with one-point and draw-odds handicap, slightly inferior hardware would not be a problem.
Then, Junior team gave example of U.S. match, where bringing the servers cost 40000 and it the hardware was changed 3 times in 2 weeks because it was not stable. 2 servers makes it 80000, and don't tell me that it is logical to spend 80000 on bringing hardware for a 100000 prize fund match.
Then, precautions are a very good thing, but they should be reasonable. Handling everything (including executables)to referees and giving them right to make investigations is ok in my opinion. While look at Rybka team's position-they were against that referee is the final instance when there is a cheating suspicion.
They were really-really loud when FIDE staged Junior-Fritz match, but after reading the pdf, I think that either Rybka team didn't have money OR they became less confident. Much less.
Mig, not that I agree that chess and neutrality are mutually exclusive interests, but its nice to be able to express opinions here without the fear of getting the boot from a stupid 'sysop'. Also, I don't see much of a point in publicizing low-quality (compared to European standards) American games and players here especially given that you don't really try to add any distinct American flavor elsewhere on the site (like talk about Park Slope or downtown NYC, etc.) But its your site. Do what you please. I'll enjoy the good posts and take the rest with a pinch of salt.
Anyway, lets do something we both like - FIDE bashing..
http://video.google.com/videoplay?docid=-3563574206685768080
Interesting how much Vishy Anand had to go through to actually 'reach' to Karpov for the WC title in '98 (do read the text alongside the video)..
"Then, Junior team gave example of U.S. match, where bringing the servers cost 40000 and it the hardware was changed 3 times in 2 weeks because it was not stable. 2 servers makes it 80000, and don't tell me that it is logical to spend 80000 on bringing hardware for a 100000 prize fund match."
These figures seem obscenely high to me. And even if the cost was somewhere close to this, it is surely worth it to prove that your software is the best (the match prize fund is not the most important thing).
Levy portrays the Rybka team as selfish and unreasonable during the negotiations, citing the precedent of using remote computers for important matches and the Junior team's unblemished record of fair play. But if you look at it arbitrarily, were they not just trying to remove all reasonable possibilities of cheating? It does seem like they were being a bit paranoid with the logs though, and I fully agree that these things should be left to the referee.
"They were really-really loud when FIDE staged Junior-Fritz match, but after reading the pdf, I think that either Rybka team didn't have money OR they became less confident. Much less."
Why not be really loud about a Chessbase publicity stunt sanctioned (and funded?) by FIDE? Didn't have the money or became much less confident? C'mon man, surely they had the money. And why would they become less confident in such a short time span unless Junior magically gained a hundred rating points?
Well, I agree that we are doing all our conclusions from the single pdf. So, if the person who wrote this was biased-then we do not have objective information. Although, if we are not really paranoic, we can at least trust the citations from e-mails he brought.
30-40 thousand was the number that Junior team tells the U.S. Kasparov-Junior match organizers had spent.
> "They were really-really loud when FIDE staged >Junior-Fritz match, but after reading the pdf, I >think that either Rybka team didn't have money OR >they became less confident. Much less."
>Why not be really loud about a Chessbase >publicity stunt sanctioned (and funded?) by FIDE? >Didn't have the money or became much less >confident? C'mon man, surely they had the money. >And why would they become less confident in such >a short time span unless Junior magically gained >a hundred rating points?
Well, I am not saying that FIDE staging Junior vs Fritz match was fair. And that is not my point-my point was that first Rybka publishes a document where they suggest a match with huge odds (which itself seems a little bit strange to me, I would have expected more respect towards their colleagues), but then they do not come to a simple agreement. They want to take such precautions as if they are 100% sure that the opponent is going to cheat.
Now you tell me the reason why this happened. For example, why Rybka can play WCC with remote computers, and cannot play this match? Is it realistic that there is no trusted third party who could be a referee for this match?
And I think they might not have 100.000 for that match. Look at their forum, they are collecting donations from their community to stage some matches against GMs. This is cool indeed-having such a community which participates in staging the matches. As a side effect, this indicates that they really do not have large financial resources. Not so rich, simply to say...
After reading the pdf, I got few questions:
1) Was Mr. Levy permitted by all parties to disclose all those details of negotiations? I don't think so, because both Convekta and Vasik are still silent about any details of the process.
2) Doesn't it look like Mr. Levy, approached to be an independent arbiter, clearly took one side instead of being neutral? He might have his own opinion, of course, but during the reading I found that he had much more personalized contacts with Junior team. His reflection on contacts with Mr. Abramov shows these were pretty formal, and the most frequent form used by Mr. Levy for Junior side and for himself, was "we". Were they one team, or what?
I ignore all that BS about inflation of computer equipment prices which is truly opposite in real life, and numerous attempts to point to Kasparov-Junior match, totally ignoring the much more recent Kramnik-Fritz match, where the server was on stage, by the way.
Also, it looks like these folks believe Mexico is a middle of nowhere, without 24/7/365 data centers, without local Intel office, etc.
C'mon, guy, Mexicans know what is computer very well, ask Mr.Radjabov if you don't believe me!
I really don't follow computer chess very much, so pardon this question. Where can I find stats showing Rybka'a dominating play?
on the internet, so google you lazy person!
stendec,
I don't follow the computer chess ratings that much, but here are a few lists that seem meaningful since the authors have considered a large number of games on various machines:
CCRL:
http://computerchess.org.uk/ccrl/4040/rating_list_all.html
CEGT:
http://www.husvankempen.de/nunn/40_40%20Rating%20List/40_40%20BestVersion/rangliste.html
SSDF:
http://ssdf.bosjo.net/list.htm
These sites are linked by the Wiki entry on computer chess (see 2nd paragraph):
http://en.wikipedia.org/wiki/Computer_chess
Thanks, Cynical Gripe, for helping this lazy person out!
The SSDF website seems most convincing as the hardware is not the difference between the "teams". It has often been a curiosity of mine how much of a program's success is due to better programming (heuristics as well as processing allocations) as opposed to better hardware.
@Stendec, the comparison at CCRL and CEGT is also on the same hardware levels, with the exception that they combine single, 2 and 4 CPU results in the same list (but keep them apart). For better overview, CCRL offers custom selections where you can for example see only the 32 bit single cpu ratings:
http://tinyurl.com/ysdvwt
You can also let display all 64 bit ratings, all freeware etc., with the selection options below the main list. - At CEGT and CCRL, you will find ratings for some strong engines which are missing in the SSDF list, yet (Naum,Loop,Toga...).
The analogy with drivers is indeed very relevant. And I'd like to add to this analogy:
In order to know who is the best driver, everyone will agree that the best way would be to make competitions with all drivers using the same car. Note that here there is already a question about the car to choose, since the best Hyundai driver may not be the best Ferrari driver...
But even when such competitions exist, the most famous one, the world F1 championship, allows each driver to run a different car. And at the end, everybody agrees that the winner may not be the best driver, it's just the best combination car+driver.
IMHO, it's here that the comparison fails!!! Because, common people consider/see only the program to win a competition. For example, in the Fritz-Junior match, I've never seen something like a report saying that Junior/PentiumX/4GBram/8CPU/(...) won against Fritz/AMDY/16GBram/2CPU/(...). People remember/know only the program. This makes the difference with the car championship where the car and the driver are well known!
More, maybe this is a reason why Rybka team is reluctant to organize this match: they think/believe/know Rybka is probably stronger than Junior. But they also know that the hardware is important, too. As they are favourite, they may fear to loose because of this hardware difference. In this case, as I stated, people won't see the defeat of the hardware, but only the defeat of the program.
The RYBKA camp has replied at www.chesstoday.net, or more accessibly, at http://susanpolgar.blogspot.com/, 8th August 12.09
Clearly they are not happy with David Levy, who they now see as not in the neutral zone.
There's plenty of tech around to sort the security for this.
Levy erupts on Chessbase every so often. What he lacks in fairness and overall good judgment he more than makes up for in long-windedness.