For those of you who are into such things -- and it's my impression that fewer and fewer people are -- the 17th World Computer Chess Championship is underway in Pamplona, Spain. Just ten competitors, including defending champ Rybka, which has dominated the computer chess world for what seems like forever. Perhaps that dominance is one reason there's less interest in these events these days, hard to say. Or maybe it's just the continuation of the decline that set in once computers became so dominant over humans. Other perennial contenders Shredder, Junior, and Hiarcs are also there.
One thing I'm happy to see is a limitation on the number of cores the competitors can use this year (to eight). Turning a programming competition into a race to see who could acquire the most outlandish hardware never made any sense to me. (Various rants via the computer tag.) Nor did the argument "but you want to see the best possible chess," since you can just play longer games if that's what matters to you. Anyone who lets his quad-core home machine think an hour per move overnight will produce better chess than just about anything running at tournament controls.
Mig, there was a discussion in Computer chess forum and almost everyone was very critical of the hardware limitation rule (I was arguing FOR it, as you do here). In particular, their principal argument was that making a chess engine work effectively on a big cluster is something different than making it work on 2 or 4 cores. There are a lot of programmers who invest a lot of time in this; this rule cripples them down. The rule not only limits the number of cores, it effectively forces you to use a fixed architecture. For instance, it leaves out dedicated computers like Diep out.
Another argument was that there are these Computer rating lists, which run thousands of games on identical machines, identical books and so on, e.g. "fair conditions".
Good point playjunior. I think the argument is basically a) Limiting the spec of the hardware allows theoretically the guy in his garage developing a competitive machine and b) Unlimiting the spec of the hardware allows progress in the software development field harnessing the power of clustering. I'll sit on the fence and say both arguments seem valid.......
Do they also inhibit over-clocking? The Junior team has some ties to the Intel fab plant in Israel that could be useful under a strict core rule.
I think the rules are that everyone has to use strictly limited hardware except Junior team who gets badass over-clocked Intel chips directly from Intel research lab.
So 8 cores is the only limitation? What's to keep someone from using a W5580 dual socket and overclocking it insanely far? Or using specialized hardware as long as it doesn't use more than 8 cores?
It doesn't make sense to place hardware restrictions unless they are, super strict. That is including the software, because if the hardware was identical, programmers could optimize it for that particular CPU which would also jeapordize the supposed "fairness" you were trying to gain by imposing hardware restrictions in the first place.
So in order to have this special tournament that is only a measure of the chess programs themselves you'd need identical hardware, identical OS and CPU overhead, identical opening books, identical tablebases, and identical hardware-specific optimization of the chess program.
If that isn't going to be done I don't see why there are hardware restrictions at all.
How can optimizing the book so that you get positions which fit the playing style of your engine be possibly be unfair? You can argue that not everyone can afford a 52-node a-la-Rybka cluster, but sure everyone can do a book tuning.
It's a good question. The idea would be to make it so that the competition wasn't decided by opening books. Perhaps only very early lines could be chosen so that the engine would have a choice to go into lines that would suit its particular playing style.
It's a frustrating dilemma though, and dilemmas like these are what make me opposed to these restrictions completely.
At least when the rules remained the same we could discuss engine strengths relative to a particular style of event. Now we can't do that "after" the 8-core rule already.
It's just wrong to think that with this 8-core rule is actually getting closer to some sort of "pure" benchmark of computer chess program software; Unless all the factors I described earlier can be adopted. There isn't a "pure" chess competition and folks who think this need to just get over it.
If we could get some supercomputers to play I would welcome it. It would be very interesting.
I understand Mig's comment that letting a computer run at very long time controls on common hardware would produce the results we are looking for by running a chess computer on super hardware at tournament time controls, but you can't really watch the long time controls match live now can you?
Plus the results of these tournaments more or less coincide with the results of CCRL and CEGT, something that indicates that the tournaments are in fact not in danger of being determined by hardware alone.
Hardware restrictions are superficial and incorrect. I do not agree with them! With the exception of if there's going to be a separate type of tournament with the super restrictions I listed earlier, and that's probably not going to happen.
What would happen if they remove all books and table bases? All draws ?
Or if the same software is presented with random initial positions?
Btw how do human stands in Fisher random against computers? I don´t recall any important match with that setup.
If books were removed from serious engine matches completely we would see certain opening lines hard coded by programmers to get the edge. They could disguise this in any number of ways.
Chess 960 would more or less prevent this. The problem with this is that most folks are interested not in a chess engine that can play chess 960 strongly, but one that can play regular chess strongly, and there is probably a serious concern that an engine's fischer random strength does not correlate well with its regular chess strength. I like this idea though, and it should be seriously considered if the ultra strict engine testing was ever to be done. For now we have the CCRL FRC (Fischer Random Chess) results:
http://www.computerchess.org.uk/ccrl/404FRC/
>Btw how do human stands in Fisher random against >computers?
Poorly, since this basically negates the human's key strength, the opening. Computers play the middle game much better than humans, and I suspect raw unfamiliar openings as well. I haven't seen such a match either but I strongly suspect that it wouldn't go well for the human.
{ from Finnegan:
>Btw how do human stands in Fisher random against >computers?
Poorly, since this basically negates the human's key strength, the opening.
}
It would be interesting to see what kind of book titled...
"Modern Chess Openings According to Unaided Computer Chess Engines"
...would be produced by Rybka et al for the traditional setup. Is there any reasonable amount of time in which a room of maybe 5-10 computers would discover and recommend the Ruy Lopez closed or the BenOni or the QGDeclined or the French Defense, to a depth that matches perhaps humanity as of 1970?
A comparison of such a computer generated MCO to the human-written MCO would tell us a lot about how well Rybka et al could produce an MCO for any one FRC-chess960 setup that we told the computers to focus on; like RNBBKNQR.
If handed a computer generated MCO for RNBBKNQR, I presume that human grandmasters could quickly make improvements.
It would be facinating to watch a tiny branch of opening theory, for RNBBKNQR, grow into a second trunk. The new ideas would be like biting the thick part of a barbeque rib, instead of nibbling loose strands at the end of the bone.
Can a computer resist this one http://photo.chessdom.com/thumbnails.php?album=223
;)
Btw that same Chessdom are the only site that seem to have pgn from the World Computer Chess Championship.
"It's just wrong to think that with this 8-core rule is actually getting closer to some sort of "pure" benchmark of computer chess program software; Unless all the factors I described earlier can be adopted. There isn't a "pure" chess competition and folks who think this need to just get over it."
This reminds me a lot of car racing, especially F1.
You can go check the bible of rules in F1 relating to mechanics and HP and whatever but the outcome always comes with very different cars for different teams.
I'm cool with this high tech escalation. Just as in cars, where F1 and other top competitions are good for the general car industry, High tech chess will be good for chess as well.
Conclusion for me is:
Like in car racing, it's not about the car, or the pilot. It's about the team. Remember that the chess program did not create itself and also does not control itself.
I recommend you check out Junior's game of today against Hiarcs - round 4. It's Tal on steroids. You don't see such firework often, between human or engines.
The Junior team maybe cooking a new version of it?
Mig you must know these guys from your days at Club Kasparov. Can you get some Daily Dirt for us on what they've been up to? We read/see a lot of Rybka, would be interesting to hear what the Junior team has done.
So far both Rybka and Junior are at 4/4, not having played each other yet.
As for the Junior-Hiarcs game, it would be especially interesting how much of it came from the opening book. Both opponents were on Xeon W5580 CPUs, 8 cores at 3.2 GHz.
Another spectacular Junior win happened in the 3rd round, against Jonny. - After 4 rounds, only Rybka and Junior have 4.0/4. Their game against each other will be in the 9th, last round on May 17th.
Overclocking is not very important in terms of results, and the participants need to care for system stability, more than for a relatively small speed gain. Something like +30% speed is almost nothing, in computer chess. The differences of engines and openings have MUCH more impact than what overclocking can give. I guess though that some overclocking IS being done, but it won't be decisive.
Some interesting stuff, thanks for keeping us up to date, guys. Good to see you slumming in the Dirt, PB!
My point is fairly simple. Either standardize the hardware within reason so it's about the program or standardize the software and everyone can have fun putting together better and better machines. Otherwise it's a competition without any defining standard so you don't know if it's the car or the driver, as it were.
That's why in car and motorcycle racing (the best analogy I can come up with) they standardize the engines and just about everything else in each competition. Yes you can have a good car and a good technical team, etc., but you don't have a situation in which one car is simply 20% faster than the others due to having a much larger engine. That's what we've had in these chess computer competitions for years.
Yes, we have the rating lists and their standardized hardware. That's why many suggest these championship tournaments are superfluous to begin with. Isn't the argument about the lists admitting these tournaments are only about acquiring the best hardware? We have a rating list in human chess and we play the games and the WCh events anyway to see who can perform on the spot and under pressure. This is fairly meaningless with computer chess. If Rybka loses, or, say, Jonny, wins this 9-round event, is anyone going to say it's a better program than the ones hundreds of points higher on the list?
The reason these tourneys can be interesting is because they should reflect the work of the programmers over the past year to improve their creations. You bring a new version, you test it out against the other guys' new versions in open combat. You can't do that at home. Plus it's fun and draws some attention and somebody gets to put "2009 computer world champion!" on the box.
I'm also in favor of balloted openings, since book work has become as important to the results of these events as anything else. Over a few hundred or thousand games as on the lists, not so much, but just as in human chess, a novelty or mistake in the opening can cost you the tournament. This is also part of the discussion "best chess" and "team" versus "best program," I know. So I'm just speaking for myself. I'd be far more interested in the results if they used standardized machines and balloted openings with 2-game matches with colors reversed. E.g. starting on move 12 of a randomly selected opening (all pre-approved, as in checkers, or even every match playing the same opening, as in contract bridge) and playing the same opponent from both sides.
Then you'd have a good idea of the best program and even why it's the best program. You could see where they diverged in their analysis and which one did a better job. Programmers would have to work on playing better chess, not only on getting their books to sync with the styles of their programs. Of course that's the way humans do it, tailoring openings to their styles, and they hire seconds the way programmers hire books specialists. But I feel the goals of computer chess aren't the same, that there should be more to it than winning.
To sum up, as much as I still enjoy it, I've lost a lot of interest in these comp events due to the focus on books and hardware. I still like to play with new engines, run them through a few custom test suites, etc. But I don't really care who wins a nine-round tournament when they rarely show anything new or interesting about the engines. They could!
I know it's completely off-topic ... sorry in advance if anyone is offended! Putting it here so I don't interrupt or disrupt the now livly US championship discussions - and at least it concerns another world championship ,:) :
FIDE press release:
"FIDE announces that the negotiations with Universal Event Promotion (UEP), the original bidder for organising the final stages of the World Championship cycle 2009-2011 (Candidates Tournament and Final Match), did not reach a final agreement. FIDE is already in contact with other organisers and sponsors interested in holding these events."
Three criteria:
1) Book
2) Hardware
3) Software
Rybka team, with Noomen, could be 100 ELO or more above next best. But major problem is that amateur teams really stink here, losing even another 200 ELO perhaps.
Most wins versus them derive just from book. Like mobile Nokia in Beijing.
Rybka team, with Cimotti 52-72 cluster, would be probably again 100-150 ELO beyond the 8 cpu machines. Overclocking by 30% is probably 25-30 ELO. Useful payoff over 9 rounds.
Rybka team, with Rajlich programming and Kaufman intellect, is about 150-200 ELO beyond 2nd place for Pamplona, but only maybe 100 beyond Naum 4.
For participant accounting:
Beijing also only 10, counting the mobile Nokia to avoid byes. Last big contigent was almost a decade ago. One problem possibly is Europe-centrism, though it might be that suitable hobbyists don't exist elsewhere anymore. Also, holding it every year, amongst other games (9x9 Go again has as many competitors, and check out KriegSpiel and even PhantomGo), could reduce attendance. ICGA (with G for Games) replacing ICCA (with C for Chess) was big fault line too, possibly. Last North American event was almost 2 decades ago. Largely only Crafty and Zappa have recently traveled (look - no Naum, no Thinker). 2002, Maastricht, had Insomniac, Noonian, and even the 0/9 Sharky from US, but those days appear passed.
i prefer the big hardware. i quite liked the deep rybka 52 vs deep sjeng 56 game. it's definitely a bit more formula-1 than nascar. to me it's far more interesting to see the best that the programs can produce, rather than something i could create on my own comp.
just my opinion, but, my impression of the hardware limitation is that is has more to do with certain teams not having competent cluster versions and/or their engines not scaling well, than it does with difficulties getting hardware together. you can deduce who those teams are by their lack of participation in the unlimited competition (hint: hunior and jiarcs).
AFAIK the hardware limitation thing was proposed and pushed by the Hiarcs team. Junior don't have a cluster version, but usually they scale quite well and they have access to the best possible machines from Intel.
The contention that HIARCS proposed the limitation rule is more conjecture than reality, I think. Simply because their Forum first published the Levy letter.
http://hiarcs.net/forums/viewtopic.php?t=2008
Just a few points for Mig to consider:
1. Computer WCH games are far better and much more interesting than the ones played in the US ch for example.
There are no pre-agreed draws and the players aren't afraid of taking risks which usually result in boring games. More novelties are produced by the computers.
2. The computer WCH is not "randomly won". in fact for the past 14 years only 3 programs won it Shredder 4 times, Junior 5 times and Rybka twice.
3. Unlike the rating list benchmarks, which consists of pretty much random positions played by older program versions taken from human games, the WCH shows the engines at their best. The programmers tailor the openings to the engine's ability just like humans do to their own style. This produces far better chess than any rating list benchmark.
4. There are many other ways to measure computers - suite solving could be one - However - they portray suite solving not overall playing ability. There is no substitute to a chess tournament where all teams bring their best versions tailored to the playing hardware; In fact this constitutes the incredible constant progress made in computer chess.
When computers play too good ignore them.
If you're looking for those gross blunders, yes the US open in the place for you...
Jay
"in fact for the past 14 years only 3 programs won it Shredder 4 times, Junior 5 times and Rybka twice."
Last winners:
2008 Rybka
2007 Rybka
2006 Junior
2005 Zappa
2004 Junior
2003 Shredder
2002 Junior
1999 Shredder
1995 Fritz
Jay, if you put something in quotation marks that means somebody else said it. I didn't say (did anyone?) the WCCC was won randomly. My point was to question the object of the competition and to say what I would like that object to be.
As for the computer games being better and more interesting than human ones, to each his own. If computers drove F1 cars maybe they could also go around the track faster than humans do. Still wouldn't be interesting to me (beyond the initial "can they do it?"). The pressure, the psychological choices, the risks, and yes, the mistakes are what make chess human and what make chess interesting. Perhaps a correspondence player might agree with you, however. Or perhaps if you stripped the player identities away from all games and looked at them in isolation. But then who would care? Maybe Huebner.
I'm more interested in how the programs play chess than in how well their book programmers do their job. Getting a program to play, for example, the black side of the King's Indian or a Hedgehog would be fascinating. Telling the book programmer not to allow the King's Indian or Hedgehog is trivial and is little more than a patch for the inability to teach programs to play certain types of positions competently.
Of course it's a competition and they can and should do what it takes to win under the rules. But that doesn't mean I have to enjoy it or sympathize with it. I'm just another selfish spectator who wants what he wants. And if that means 'worse' chess because the comps aren't playing exactly the positions they like best out of their customized books, I'm fine with that. We wouldn't force humans to do that (though balloted openings might be inevitable there, too) but then, they are humans. Computer chess is a science and I'm more interested in that side of it than how well a human can program its book.
I don't agree that the "best possible chess" definition, even should that be the object, is fulfilled only by allowing the program to start thinking deep into its book. Saying so is a disservice to the programmers. The programs are strong enough and even smart enough to play some very interesting ideas in all sorts of positions, even theoretical ones. 10.Re3! might ring a bell. Of course if the balloted positions are too deep we would lose this element, which would be a shame. So a balance must be sought, as in most things.
"But then who would care? Maybe Huebner.'
Why is that ? I didn't get it.
Just an inside-inside joke. He was famously objective and clinical in his analyses.
Last winners:
2008 Rybka
2007 Rybka
2006 Junior
2005 Zappa
2004 Junior
2003 Shredder
2002 Junior
2001 Junior / Shredder
1999 Shredder
1997 Junior
1996 Shredder
1995 Fritz
Mig,
You mentioned in your first post that Jonny might win the championships indicating it is a random event - point was that their is a high correlation between the winner of the world comp ch, and the best comp in the world as reflected by Rybkas' standing at the moment. It is highly unlikely that Jonny or any other weak program would win this event. (not the case btw for many human tournaments)
As for the liking, you're absolutely right - it is a matter of taste, but know that their are many (correspondence) and others that follow the world comp ch. which means more to them than the same people playing in all the top tournaments. I was just objecting your "who cares" attitude...
As for the issue of playing deep out of the book we all know that humans now days play deep out of their (home comp prepared book) - comps are more and more responsible for most opening novelties these days - in fact comp books tend to replace human moves with their own simulations. In fact comps revolutionized chess. And their progress is important for the game whic could be followed closely in the world ch.
So perhaps the humans are the ones who should start playing from random positions... remember Bobby?
"
2001 Junior / Shredder
1997 Junior
1996 Shredder
"
These three are from the World Micro Computer Chess Championship, not the World Computer Chess Championship (the events coincided in 1999).
If desiring those, please add:
2001 Junior
2000 Shredder
1995 MChess-Pro 5.0
Also note: the Junior team was banned from entering Indonesia in 1996.
This makes 6 winners in the last 14 years, rather than the originally facted 3.
How do they verify the 8-core rule?
Back in the early WCCC days, they had 3 types of competitors:
*) dedicated hardware
*) commercial software (on Apple II, for instance)
*) academic bigiron (CrayBlitz)
The last was allowed to compete "remotely", but it was sort of a gentleman's agreement, hat-tipping to academia. With the commercial software, the machine it ran on typically had to be right there. Obviously the dedicated units were onsite.
But now, with business interests at stake, I don't see why one should assume that the modern "remotely-played" games are always legit. Especially with 8-core limit. Rajlich even says something like that the slaves don't produce their own output, and he just multiplies the node numbers, via a formula, for them. Who wouldn't be able to dodge the system?
How many clusters were in the Internet CCT? Only Rybka, I think. Not even Crafty, though presumably in the future. Maybe Sjeng or Diep, but hard to tell. Someone of them were even on notebooks. Cluster Toga was in the last WCCC, and Jonny was a cluster also in Beijing. So maybe 5 or 6 exist by now.
I said the last 14 years, simple arithmetic doesn't include 1995. Apologies for leaving out Zappa a well deserving winner. In summary only 4 programs Shredder, Junior, Zappa, Rybka got the title during this period which says that the WCH title is significant and it looks like Rybka is going to win it again this year.
May 15 2009 minus 14 years includes October 1995 (WMCCC) and May 29 1995 (Fritz over StarSocrates)... smiley... Now that the 2009 edition is over, fourteen years doesn't include 1995 anymore.
In the end, Rybka scored 8/9 to win. But Rybka had White against all three 2nd place finishers, Shredder, Sjeng, and Junior, and in the last-round Junior had to play for a win with Black, with 6.5/8 versus 7/8.
I cannot say how much the core limit helped or hurt. Those with spendy $5000 octocore machines were helped. If you deleted the books and reduced them to a fixed machine one-core, it could be a lot different. If one-core made the attendance go up, an unscaled Rybka would probably no longer be a complete favorite. Also Hippolytus would have a chance.
Congratulations to Rybka. Great games. I liked the way she smashed Shredder.
You mean to the Rybka team ...
Hard to congratulate software , they are very moody and often find compliments sexist or deprecatory.
Have no enough cash to buy a house? Worry no more, because that is achievable to get the mortgage loans to work out such kind of problems. Thence get a small business loan to buy everything you require.