v1.6.3

What's Changed

• Install apple-codesign as locked and use input ref for checkout by @ayazhafiz in #75
• Surface native runner information in the presence of errors, and unconditionally write to reporters. by @ayazhafiz in #76
• Prepare ABQ 1.6.3 by @ayazhafiz in #77

Full Changelog: v1.6.2...v1.6.3

ABQ 1.6.2

ABQ 1.6.2 is a patch release fixing an issue that could result in denial-of-service of an ABQ queue due to large
test results.

ABQ will also now log a warning when a worker for a given run ID is using a test
command different from any other worker, either in the same run or during
retries. ABQ may not function properly if the test command for a run ID changes
between workers executing tests for that run ID.

What's Changed

• Stream test results without batched loads into server memory by @ayazhafiz in #73
• Log warnings when workers have different commands by @ayazhafiz in #74

Full Changelog: v1.6.1...v1.6.2

ABQ 1.6.1

ABQ 1.6.1 is a patch release fixing an issue that would not continue offloading manifest and results files if one offload attempt failed.

What's Changed

• RWX access token in local tests in #68
• Lower the log level of failed connections in #70
• Do not quit results and manifests offloading loop if one file offload fails in #71

Full Changelog: v1.6.0...v1.6.1

ABQ 1.6.0

ABQ 1.6.0 adds the abq report list-tests command, which lists the tests run remotely by a worker and runner.

More documentation is here

What's Changed

• minor tweaks from reading the code in #65
• Drop support for ABQ statefiles in #33
• add abq report list-tests in #67

ABQ 1.5.0

ABQ 1.5.0 adds the ability to replay tests locally as well as a new strategy for test distribution.

Replaying Tests Locally

You can now replay exactly the tests run on a worker in CI in cloud mode. To achieve this, you should create a read-only personal access token and hook it into ABQ locally using abq login.

More docs here

Test Distribution Strategy

Previously, ABQ assigns the next available test to any worker. That behavior is still the default, but now, you can opt-in to a worker getting an entire file's tests. This ensures that expensive per-file shared setups or teardowns will run only once, on one worker, however it may cause tests to be less evenly distributed.

CLI Docs are here

What's Changed

• fix one potential deadlock in #43
• adds a --startup-timeout-seconds flag in #42
• adds per-file queueing strategy in #41
• adds personal access token & abq login support in #38

ABQ 1.4.0

ABQ 1.4.0 adds support for loading test runs from other queue instances via remote storage.

Until 1.4.0, a test run that executed on one instance of ABQ could not be loaded
and re-executed on another instance. Executing a test suite with the same run ID
on two different ABQ instances would run the test suite twice, with no sharing
of test results.

ABQ now writes run_state files to remote persistence locations, when remote
persistence is configured via the ABQ_REMOTE_PERSISTENCE_STRATEGY environment
variable, and its accompanying environment variables. If the same remote
persistence strategy is configured between multiple ABQ queue instances, queue
instances will have the capability to load test runs that initially executed on
previous queue instances.

At this time, loading test runs from other queue instances includes the
following restrictions:

• run_state files are schema-versioned, and no schema-version compatibility
  guarantees across versions of ABQ queues are provided at this time.
  run_state files are guaranteed to be compatible if shared between ABQ
  queues of the same version.
  If an ABQ queue loads a run_state file that it is incompatible with, the
  remote test run state will not be loaded. Executing a test suite whose
  run state file failed to be loaded will fall back on executing the test
  suite as a fresh run, similar to the pre-1.4.0 behavior.

• The same run ID may not be executed, in parallel, on two different ABQ queue
  instances sharing the same remote persistence. For a given run ID, an ABQ
  queue will assume exclusive ownership of the test suite run associated
  with that run ID.
  At this time, ABQ does not verify whether it indeed has exclusive
  ownership of a run ID. If you are self-hosting ABQ, you must ensure that
  run IDs are routed to a unique ABQ instance for the duration of a test run;
  however, once a test run is complete, retries of the test run may be routed
  to another ABQ instance, so long as the exclusive ownership constraint
  continues to apply for the duration of the retry.
  If you would like to avoid self-hosting, RWX's managed hosting of ABQ
  supports routing test runs under these constraints.

See the ABQ documentation for more details on persistence.

What's Changed

• Add methods to check it remote manifests for a run ID exist by @ayazhafiz in #30
• Add a model for persisting run state when a run completes by @ayazhafiz in #32
• Load run states from remote by @ayazhafiz in #31
• Persist run states when a run reaches the end of a manifest by @ayazhafiz in #34
• Load results from a remote persistence location on first open by @ayazhafiz in #35
• Add an integration test for running tests between queue instances by @ayazhafiz in #36
• Prepare 1.4.0 by @ayazhafiz in #37

Full Changelog: v1.3.5...v1.4.0

ABQ 1.3.5

This patch release fixes a bug related to a race condition in how test results are stored.

What's Changed

• Drop use of an LRU cache for results files by @ayazhafiz in #29

Full Changelog: v1.3.4...v1.3.5

ABQ 1.3.4

This release adds a feature preview of remote persisted storage of test runs and results.

A remote storage source is used to

• synchronize ABQ manifests and test results stored on local disk to the remote
  source
• offload manifests and test results at a configured frequency, synchronizing
  the data to the remote source and removing it from local disk
• download offloaded manifests and test results on-demand

When configured, files will be synced to the remote persistence target with
the following behavior:

• Manifests are synced to the remote persistence target after all entries in
  a test run's manifest have been assigned to a worker. The manifest is not
  modified again.
• Results are synced to the remote persistence target when there are no writes of
  results to the local persistence target in-flight. That is, writes to the
  remote target are batched and executed when the local state is stable.

The following environment configuration variables are available for configuring
remote storage on abq start:

• ABQ_REMOTE_PERSISTENCE_STRATEGY: What remote persistence strategy should be
  used. If unset, no remote persistence will be configured. The options for
  remote persistence are:

  • s3: manifests and results will be persisted to an AWS S3 bucket. This
    strategy requires the following additional environment variables to be
    set:

    • ABQ_REMOTE_PERSISTENCE_S3_BUCKET: The S3 bucket files should be written
      to.
    • ABQ_REMOTE_PERSISTENCE_S3_KEY_PREFIX: The prefix to use for keys written
      to the configured S3 bucket.

    AWS credentials and region information are read from the environment, using
    the standard AWS environment variable support.

  • custom: Files are remotely persisted by calling a provided executable.
    This strategy requires the following additional environment variables to be
    set:

    • ABQ_REMOTE_PERSISTENCE_COMMAND: A comma-delimited string of the executable
      and the head arguments that should be called to perform an operation on the
      remote persistence target. The executable will be called in the following form:

      <executable> <...arguments> <mode> <file-type> <run-id> <local-path>
      

      Where

      • <mode> is either "store" or "load", depending on whether the file should be stored
        into the remote location, or loaded from the remote location.
      • <file-type> is either "manifest" or "results".
      • <run-id> is the run ID of the test suite run.
      • <local-path> is the path to the file on the local filesystem. If the mode is "store",
        the content to upload should be read from this path. If the mode is "load", the
        downloaded content should be written to this path.

      The provided executable must be discoverable in the PATH that abq start
      is executed with.

      For example, if you have a Node.js script abq-remote-persister.js that
      performs remote persistence operations, the configured environment
      variable would be

      ABQ_REMOTE_PERSISTENCE_COMMAND="node,abq-remote-persister.js"
      

      Supposing that node is in the PATH.

• ABQ_OFFLOAD_MANIFESTS_CRON: If configured, the cron schedule
  by which manifest files stored locally should be offloaded to the remote
  storage location.
  A manifest file is only eligible for offload if it is stale; see
  ABQ_OFFLOAD_STALE_FILE_THRESHOLD_HOURS below.
  All time in the cron expression will be interpreted as UTC.

• ABQ_OFFLOAD_RESULTS_CRON: If configured, the cron schedule
  by which results files stored locally should be offloaded to the remote
  storage location.
  A results file is only eligible for offload if it is stale; see
  ABQ_OFFLOAD_STALE_FILE_THRESHOLD_HOURS below.
  All time in the cron expression will be interpreted as UTC.

• ABQ_OFFLOAD_STALE_FILE_THRESHOLD_HOURS: The threshold, in hours, since the
  last time a while was accessed before it is eligible for offloading to the
  configured remote persistence location, if any is configured.
  The default is 6 hours.

A future release of ABQ may support synchronizing manifests and test results
between ABQ queue instances, by way of a configured remote persistence target.

As this is a feature preview, the interface presented here may have a breaking
change in a future ABQ release.

Downloads

• Linux, x86-64: https://abq.build/api/downloads/1.3.4?os=linux&arch=x86_64
• Linux, aarch64: https://abq.build/api/downloads/1.3.4?os=linux&arch=aarch64
• macOS, x86-64: https://abq.build/api/downloads/1.3.4?os=darwin&arch=x86_64
• macOS, aarch64: https://abq.build/api/downloads/1.3.4?os=darwin&arch=aarch64

ABQ 1.3.3

This patch release improves the stability of the queue implementation.

• ABQ now uses the default musl libc memory allocator on x86 Linux targets, replacing use of the mimalloc memory allocator. For larger workloads, the default musl allocator is expected to provide better memory utilization characteristics. (#16)
• ABQ will now cancel active runs for which no progress has been made in an hour. (#12)

ABQ v1.3.2

This release improves the output for abq test and abq report. It also includes several bug fixes.

• Failure detail is provided in abq report output.
• Retried tests are listed in summary output.
• Runner number is included in failed/retried test lists during abq test.
• Worker number is included in failed/retried test lists during abq report.
• Fix reporting of native runner failures that are later retried.
• rwx-v1-json reporter counts native runner failures as otherErrors.
• Manifest generation output suppressed when it's empty.
• Document support for pytest.