Solving mate problems is counter-productive for improving engine strength, because mate solving depends on pure tactics and discourages any positional knowledge. Although it's OK to solve mates just for fun.

Here is one much more interesting test suite: Strategic Test Suite.

Excellent! Naturally I am only interested in a UCI version.

With or without opening book code is both fine. Most chess interfaces provide book handling, as well as polyglot of course. If you'll decide to implement opening book, please provide a way to disable it, because many tournaments accept only engines that can play without own opening book.

Exactly. Yes, bitbases only have "win/loss/draw" information, but it's often enough. On the upside, bitbases are small and fast, so you can probe them long before the exchange, to know which endgame to convert to. Although winning some won endings (like some kqpkq positions) with just bitbases may be tough or impossible.

Yes, and it should play them out properly after finding them. It's also useful to check if the engine will correctly detect deep combinations that may show up outside of the principle variation that it builds up. I haven't tried to write an efficient mate-solver though, mainly for the reason you mentioned: it's a waste of time in terms of actual playing strength.
That said, I've probably seen better improvements on test scores from tuning based on actual games than from tuning based on test score results...

I think the UCI specification demands that you do that.
Not that I believe all engines implement the UCI specification to the letter.

In that case the position will not be scored as mate-in-N+a-bit in your favour.

Oh, most computers play a horrible kind of chess, and perform very poorly in closed strategic positions. They've become a lot better and maybe the top engines these days can't be defeated by "anti-computer" tactics so easily anymore, but all of the weaker ones almost certainly can be.

EDIT: Latest source attached.
It should now compile cleanly on both OS X and Linux (I haven't tried Windows, but since I haven't done anything for Windows specifically, it's likely to be as hard to compile there as the previous version was; MinGW might work though).
There are various improvements and enhancements, and the engine should play a better game than the one I posted here earlier (but maybe not as good overall as some of my earlier builds; I've been adding new functionality to the evaluation function and I don't think everything has been properly tuned yet - and of course, there's still a lot missing there too).
To build the UCI engine (for running using a generic GUI rather than the build-in Allegro5 one), run "make ucijazz". The UCI implementation is a bit rough at the moment, but it works well enough to play a game.
The engine's time management isn't that great at the moment (again, something I would need to tune); I find it plays best when given a fixed time per move...

EDIT2: Oh, before I forget: to build the Allegro 5-based GUI, you'll need the latest (SVN) version of Allegro...

Amazing!

Cool. It'll be interesting to know how things run.

Makes sense.

Right. At the moment, it's only added on Linux. I should probably add it for MinGW as well, but better would be to detect when it's needed and when it isn't.
Or just always use it, even if a native version is available...

That's odd. It should, and does for me when I run it under a UCI interface here (but it doesn't when I run it through polyglot in XBoard).

Right, as I said, the UCI implementation is somewhat incomplete. Not sending a ponder move is correct: at the moment, the engine cannot ponder. This is actually relatively easy to add using UCI (just make the next move from the principle variation and enter an infinite search), but I haven't got round to doing this.

It'll have a proper version when it's properly released.
I'll stick something temporary in there now, so you can always identify different versions (this being the only one not to report any version information).

Considerably. The 64 bit version is about twice as fast as the 32 bit version on my test machine. Note that I've done all of my development in 64 bit and haven't (yet) made any effort to speed up 32 bit.
On the other hand, a factor 2 is "only" one ply of search... which makes a difference, but the difference between 10 and 11 ply shouldn't be so shocking, especially considering that the principle variation will probably be searched deeper and that this excludes the quiescence search.

Good, it shouldn't. That's not saying it never will, of course, but I hope I've been careful about fixing things like that.

That does sound similar to what I'm seeing. What it should do is divide the remaining time by the number of moves until the next time control and add half the increment (if there is one) and use that amount of time for each move. It's not particularly clever, but it should work nevertheless.
I suppose it's possible that I'm not checking the clock often enough during the search, but I wouldn't have thought it would cause that many problems. Certainly something I'll have to experiment with a bit. I keep a log of all UCI commands received by the engine, so I can at least look at that and try to reproduce what it's doing during the game (oh, you may want to remove that log, since it's appended to and never deleted. I'll make that a debug option so it doesn't get written by default).

I don't suppose you keep records of the individual games (PGN)? I'd be interested in checking some of them to work out where the problem lies. Major shortcomings I've noticed are in its idea of king safety (it'll castle and keep its pawn shield in tact after that, but it doesn't take much to have it wreck either before that, so it'll often either castle into a wide-open king side, or move the king on its own) and playing endgames. In particular rook endings, which is probably due to the very poorly tuned evaluation in the end game. And possibly the lack of bitbases for those cases as well.

Yup. The new 64-bit GCC released for Windows the other day^[1] looks interesting for that.

Interesting. I wonder what the main reason for that is, although I suppose most commercial engines being only available for Windows and most hobbyist programmers also (historically at least) using Windows has something to do with it. Crafty and GNUchess being obvious examples of engines that were born on different platforms.

Ok, great!

Probably not; unless I know what I'm looking for exactly it's probably too large and unwieldy to work out which line in the file corresponds to which position in which game.

That's actually just a matter of un-commenting one line in the source and commenting another, but what I'd do (probably will do, when I get round to it) is just start ucijazz in a terminal and send it position and go commands from the log. That's also how I plan to debug the time management.

EDIT
In fact, sending

(The last two lines of the logfile on my machine) I get

which seems right to me.
However, it doesn't send a principle variation. I thought I made it do that, but maybe it only prints it in its "native" output (which is suppressed when using UCI). There are issues with it anyway, since now and then the principle variation will be truncated by the transposition table. I know how to fix that (if the new PV is shorter than the old one, but the moves that are there are the same, keep the old one - or alternatively and in addition, try to reconstruct the missing part from the transposition table) but I haven't got round to it yet...

Thanks!
I had a quick look at four of the games (one win and one loss against each of the opponents); it's a bit confusing. For the most part, Jazz seems to play decently well in the middle game, although it doesn't quite centralise its pieces as efficiently as it could and it seems to like leaving a pawn vulnerable and then stepping in to defend it at the last moment, which isn't so good. The way it plays endings looks pretty bad, and I've seen it drop a piece in both the losses I looked at in a position where I wouldn't have thought it necessary to do so. That could be the broken time management ruining things; I should check whether these blunders correlate with move number.

It's the same everywhere.
I neglected to mention Glaurung as another example of an engine that originated on another platform above.

Right, I checked the code, and it indeed doesn't send the PV in UCI mode. I'll fix that once I have the code to "keep" the PV (what I described earlier) in place.

By the way, I fixed a rather nasty bug late last night: when generating check evasions, Jazz will only generate moves that can eliminate the check (as opposed to generating all moves and discard the ones that don't eliminate check afterwards). It will generate captures (of the checking piece, if there is only one), legal king moves and (finally) moves that interrupt the line of sight of the checking piece. However, there was a bug where it would not generate these if the check was across a file. By ill luck, this didn't show up in any of the test positions I tested the new code on, but I found it yesterday when I used the same lookup tables for something else.
At worst, this bug could have lead Jazz to see a mate where there wasn't one, but the most probably effect is that it miscalculated the best response to check, maybe dropping a piece or walking into a forced mate in the process. I'll check to what extent this may have damaged match results locally, then attach a new version later on.

It crashed on me Evert, I attached the console output so maybe you can figure out why.

Edit:

Just happened again in a similar way...

There seems to be a fairly active community for it, although I can't really judge how large the influx of new people is.
I guess writing a chess program is a fairly daunting task and requires some dedication and attention to details (in other words, you need to write reasonably bug-free code), so I can imagine people getting started on it and then abandoning the prject before it gets anywhere.

Cool. My first chess program was from 1996 or so. I can't run it now though, because it was written in Turbo Pascal mixed with assembly language (actually, almost everything was written in assembly). It played horrible though.

That would be fun to do - but I'm afraid finding the time for something like that will be hard...

I'll give it ago! I decided to implement the UCI protocol (it's a bit simpler), but I can run through XBoard using a translator.

Damn! Sorry about that...
Any chance you could send me your moves so I can try to reproduce the sequence of moves here?
Alternatively (or perhaps even better), if you're willing to look into this, could you send me a gdb backtrace?

Heh. I'll do my best.

It was late, but I think I read everything: it crashed, you attached the console output, then it crashed again in the same (or a similar) way. In the log you showed, there is a segfault after a 7 ply search... but that knowledge alone isn't enough to debug the problem, I probably need at least the sequence of moves from the initial position if I'm going to try to reproduce it...

Hmm. Do you see the same thing with the A5 examples or demo?

Woops! Again, sorry about that... it doesn't do that for me, but that doesn't help you. If you have the core dump, you should also be able to get a backtrace from that (gdb jazz core).

Perhaps... still, would be good to sort out.

Against humans, quite possibly. Against other computers, I'm not so sure... I've seen it play a pretty lousy end game as well.
The early middle game is probably where the search is shallowest (well, the opening will be worse, but it's not as important there) and where the lack of certain evaluation features (most notably, proper king safety) hurts most. If it gets through this stage without wrecking its king's position, it probably won't unless pressured, so it should do ok in that case and reach the end game ok. The end game evaluation is crude and inaccurate, but the search depth at least partly makes up for it. Again maybe not against other computers, but against humans it probably does.

I'm not able to save with F6, getting a zero length file.