GH-46609: [Release][CI] Use System GTest for macos verification

[raulcd]

Rationale for this change

Our system GoogleTest (installed by Homebrew) and the bundled GoogleTest are mixed and we get some CI failures.

What changes are included in this PR?

Use the system GoogleTest on our verification CI jobs for macOS.
The PR also contains a minor fix on gendata due to a small bug on a Python version.

Are these changes tested?

Yes, via archery.

Are there any user-facing changes?

No

• GitHub Issue: [CI][Release] Release verification jobs for macOS AMD64 fail to build GMock #46609

[github-actions]

⚠️ GitHub issue #46609 has been automatically assigned in GitHub to PR creator.

[raulcd]

@github-actions crossbow submit verify-rc-source-cpp-macos-amd64 verify-rc-source-cpp-linux-ubuntu-24.04-amd64

[github-actions]

Revision: a295e17

Submitted crossbow builds: ursacomputing/crossbow @ actions-a736b5ee65

Task	Status
verify-rc-source-cpp-linux-ubuntu-24.04-amd64
verify-rc-source-cpp-macos-amd64

[raulcd]

@github-actions crossbow submit verify-rc-source-cpp-macos-amd64 verify-rc-source-cpp-linux-ubuntu-24.04-amd64

[github-actions]

Revision: e2d6347

Submitted crossbow builds: ursacomputing/crossbow @ actions-d4e7beb963

Task	Status
verify-rc-source-cpp-linux-ubuntu-24.04-amd64
verify-rc-source-cpp-macos-amd64

[raulcd]

@github-actions crossbow submit verify-rc-source-cpp-macos-amd64 verify-rc-source-cpp-linux-ubuntu-24.04-amd64

[github-actions]

Revision: da3e1b0

Submitted crossbow builds: ursacomputing/crossbow @ actions-868e9b032d

Task	Status
verify-rc-source-cpp-linux-ubuntu-24.04-amd64
verify-rc-source-cpp-macos-amd64

[raulcd]

@github-actions crossbow submit verify-rc-source--macos-

[github-actions]

Revision: da3e1b0

Submitted crossbow builds: ursacomputing/crossbow @ actions-01e35ef461

Task	Status
verify-rc-source-cpp-macos-amd64
verify-rc-source-cpp-macos-arm64
verify-rc-source-cpp-macos-conda-amd64
verify-rc-source-csharp-macos-amd64
verify-rc-source-csharp-macos-arm64
verify-rc-source-integration-macos-amd64
verify-rc-source-integration-macos-arm64
verify-rc-source-integration-macos-conda-amd64
verify-rc-source-python-macos-amd64
verify-rc-source-python-macos-arm64
verify-rc-source-python-macos-conda-amd64
verify-rc-source-ruby-macos-amd64
verify-rc-source-ruby-macos-arm64

[raulcd]

The previous failure (Gtest build) is fixed but the integration jobs seem to be failing to generate the integration data, see:

 Successfully installed archery-0.1.0 cffi-1.17.1 click-8.2.1 pycparser-2.22
Traceback (most recent call last):
  File "/var/folders/y6/nj790rtn62lfktb1sh__79hc0000gn/T/arrow-HEAD.XXXXX.jinLaDxeh3/venv-source/bin/archery", line 8, in <module>
    sys.exit(archery())
             ~~~~~~~^^
  File "/private/var/folders/y6/nj790rtn62lfktb1sh__79hc0000gn/T/arrow-HEAD.XXXXX.jinLaDxeh3/venv-source/lib/python3.13/site-packages/click/core.py", line 1442, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/private/var/folders/y6/nj790rtn62lfktb1sh__79hc0000gn/T/arrow-HEAD.XXXXX.jinLaDxeh3/venv-source/lib/python3.13/site-packages/click/core.py", line 1363, in main
    rv = self.invoke(ctx)
  File "/private/var/folders/y6/nj790rtn62lfktb1sh__79hc0000gn/T/arrow-HEAD.XXXXX.jinLaDxeh3/venv-source/lib/python3.13/site-packages/click/core.py", line 1830, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/private/var/folders/y6/nj790rtn62lfktb1sh__79hc0000gn/T/arrow-HEAD.XXXXX.jinLaDxeh3/venv-source/lib/python3.13/site-packages/click/core.py", line 1226, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/var/folders/y6/nj790rtn62lfktb1sh__79hc0000gn/T/arrow-HEAD.XXXXX.jinLaDxeh3/venv-source/lib/python3.13/site-packages/click/core.py", line 794, in invoke
    return callback(*args, **kwargs)
  File "/Users/runner/work/crossbow/crossbow/arrow/dev/archery/archery/cli.py", line 873, in integration
    run_all_tests(**args)
    ~~~~~~~~~~~~~^^^^^^^^
  File "/Users/runner/work/crossbow/crossbow/arrow/dev/archery/archery/integration/runner.py", line 639, in run_all_tests
    generated_json_files = datagen.get_generated_json_files(tempdir=tempdir)
  File "/Users/runner/work/crossbow/crossbow/arrow/dev/archery/archery/integration/datagen.py", line 1890, in get_generated_json_files
    generate_primitive_case([17, 20], name='primitive'),
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/runner/work/crossbow/crossbow/arrow/dev/archery/archery/integration/datagen.py", line 1557, in generate_primitive_case
    return _generate_file(name, fields, batch_sizes)
  File "/Users/runner/work/crossbow/crossbow/arrow/dev/archery/archery/integration/datagen.py", line 1497, in _generate_file
    col = field.generate_column(size)
  File "/Users/runner/work/crossbow/crossbow/arrow/dev/archery/archery/integration/datagen.py", line 614, in generate_column
    values.append(random_bytes(nbytes))
                  ~~~~~~~~~~~~^^^^^^^^
  File "/Users/runner/work/crossbow/crossbow/arrow/dev/archery/archery/integration/util.py", line 112, in random_bytes
    return random.getrandbits(nbytes * 8).to_bytes(nbytes,
           ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
TypeError: an integer is required

I am unsure why and I can't seem to find any related commit about integration tests apart from moving the JS integration tests to use the remote repository. @kou do you know if something has changed about the data generation that could cause this failure?

[kou]

Hmm. It seems that nbytes is np.int32 here. How about the following?

diff --git a/dev/archery/archery/integration/datagen.py b/dev/archery/archery/integration/datagen.py
index c8cc36ddc5..18f1d4803d 100644
--- a/dev/archery/archery/integration/datagen.py
+++ b/dev/archery/archery/integration/datagen.py
@@ -609,9 +609,9 @@ class BinaryField(PrimitiveField):
 
         sizes = self._random_sizes(size)
 
-        for i, nbytes in enumerate(sizes):
+        for i, np_nbytes in enumerate(sizes):
             if is_valid[i]:
-                values.append(random_bytes(nbytes))
+                values.append(random_bytes(int(np_nbytes)))
             else:
                 values.append(b"")

[kou]

Ah, this may be related to Python version:

Python 3.12:

import numpy as np
list(enumerate(np.random.exponential(scale=4, size=10).astype(np.int32)))
# => [(0, 0), (1, 0), (2, 4), (3, 7), (4, 1), (5, 2), (6, 7), (7, 7), (8, 1), (9, 0)]

Python 3.13:

import numpy as np
list(enumerate(np.random.exponential(scale=4, size=10).astype(np.int32)))
# => [(0, np.int32(0)), (1, np.int32(7)), (2, np.int32(5)), (3, np.int32(1)), (4, np.int32(1)), (5, np.int32(4)), (6, np.int32(0)), (7, np.int32(4)), (8, np.int32(3)), (9, np.int32(1))]

[raulcd]

@github-actions crossbow submit verify-rc-source-integration-*

[github-actions]

Revision: a1e44fc

Submitted crossbow builds: ursacomputing/crossbow @ actions-7f6847a981

Task	Status
verify-rc-source-integration-linux-almalinux-8-amd64
verify-rc-source-integration-linux-conda-latest-amd64
verify-rc-source-integration-linux-ubuntu-22.04-amd64
verify-rc-source-integration-linux-ubuntu-24.04-amd64
verify-rc-source-integration-macos-amd64
verify-rc-source-integration-macos-arm64
verify-rc-source-integration-macos-conda-amd64

[raulcd]

oh! This seems to be a bug on Python 3.13.4 that is getting fixed on Python 3.13.5, see:

• regression in 3.13.4 Random.getrandbits when an int like object is passed python/cpython#135326

[raulcd]

@kou this should be ready for review now.