SAT-Race Logo

SAT-Race
2006

Overview
Rules
Important Dates
Organizers
Downloads
Qualification
Solvers
Results
Result Analysis

Rules

These are the official rules for the SAT-Race 2006 (version 1.0, last updated on 08/08/2006).

Entrants

An entrant to the SAT-Race 2006 is a SAT solver submitted in either source code or binary format using the Web registration form. Binary format submissions must be compatible with the competition hardware and operating system. Each SAT-Race 2006 entrant must include a README file explaining how to (compile and) install the solver. Installation and execution of solvers must not require root access. Binaries should be statically linked. The organizers will make reasonable efforts to install each system, including communication with the submitters of the system in case of difficulties. Nevertheless, the organizers reserve the right to reject an entrant if its compilation or installation process proves overly difficult. In order to obtain reproducible results, SAT solvers should refrain from using non-deterministic program constructs.

Each entrant to the SAT-Race 2006 must include a short (1-2 pages) description of the system. This should include a list of all authors of the system and their present institutional affiliations. It should also describe any algorithms or data structures that are not standardly used in such systems. System descriptions will be posted on the SAT-Race 2006 website.

In contrast to former SAT Competitions, it is not required to make source code or binaries publicly available after SAT-Race 2006.

We have set up a discussion forum for the SAT-Race and strongly encourage participants to use this forum, e.g. to find answers to frequently asked questions or to discuss with co-competitors.

Submitters are encouraged to be physically present at SAT-Race 2006, but are not required to be so to participate or win.

Input and Output Format

The input and output format requirements are the same as those used for the SAT Competitions.

Execution Environment

During SAT-Race 2006 all programs will run on machines with the following specification:
  • Operating System: GNU/Debian Linux 2.6.8 (no SMP)
  • Processor: Intel Pentium 4, 3 GHz (hyperthreading switched off)
  • Memory: 2 GB (1.5 GB memory limit for solver processes enforced)
  • Cache: 1024 KB
  • Compilers: GCC 3.3.5 (might change to 4.0.x), javac 1.5.0_04

Benchmark Selection

The benchmarks for the SAT-Race 2006 (as well as for the qualification rounds) will be selected randomly out of a pool of SAT instances. This pool mainly consists of instances from the industrial category of previous SAT Competitions, but may also contain a smaller fraction of new instances stemming from applications with industrial relevance.

To obtain benchmarks of suitable difficulty we select only such instances that can be solved by at least one of the top ten SAT-Solvers of last year's SAT Competition (industrial category) in a sensible time-frame around the Race's threshold run-time.

Qualification Rounds

Before the race there will be two qualification rounds. Successful participation (i.e. no wrong results; compliance with I/O format requirements; sufficient performance) in at least one of them is mandatory to qualify for the SAT-Race.

The Race

The Race itself will take place during or shortly before the SAT'06 conference. Each solver will have to process 100 SAT instances. Per SAT instance and solver a run-time limit of 15 minutes will be imposed.

Assessment of Solvers

Solvers will be assessed based on a score that takes into account (1) the number of instances solved within the run-time limit and (2) the total time needed to solve all instances.

The exact assessment scheme is as follows:

  • For each solved instance a prover gets 1 solution point.

  • For each solved instance—depending on the time the solver needed—additional speed points are distributed: For each instance x points are equally split up among all successful solvers. The resulting number is the maximal speed score (pmax) a solver can obtain, i.e.

    pmax = x / #successful_solvers .

    Then, in a second step, the number of speed points ps each successful solver s obtains is computed by

    ps = pmax · (1 – ts / T) .

    Here ts is the time that solver s needs to solve the instance, and T is the run-time threshold (i.e. 15 minutes). Thus, a solver gets pmax speed points if it solves the instance immediately and 0 points if it solves it at or above the run-time threshold, with linear interpolation in between.

    We will set x in such a way that at most one speed point per solver and instance can be obtained.

    Speed points are motivated by and related to last year's SAT Competition's speed purses.

The total score a solver obtains is the sum of its solution and speed points summed over all instances.

Prizes

The solver that achieves the highest score in the race obtains the title "SAT-Race World Champion 2006". Moreover, there will be prizes for the three solvers with the highest scores. There will also be a special prize for the best solver submitted by a (team of) (PhD) student(s).

Restrictions

Due to limited computational resources, the organizers reserve the right not to accept more than one version (defined as sharing 50% or more of its source code) of the same solver. The organizers also reserve the right to submit their own systems—or other systems of interest—to the competition.