Error in test command: Both stdout and stderr are not empty for \"xcodebuild\" comman

[funzin]

Description:

While using the sweetpad extension in VSCode, an error occurs during the execution of the test command. The issue is caused by multiple matching destinations being detected, leading to ambiguous selection and failure in fetching the build settings.

The error occurs automatically when the sweetpad extension tries to run a test build. Here's the error message
Feature is a target defined in the Swift Package Manager.

{
  "message": "Both stdout and stderr are not empty for \"xcodebuild\" command",
  "level": 2,
  "time": "2024-10-19T01:35:52.823Z",
  "stdout": "[\n\n]",
  "stderr": "--- xcodebuild: WARNING: Using the first of multiple matching destinations:\n{ platform:macOS, arch:arm64e, id:XXXXXXXX-XXXXXXXXXXXX, name:My Mac }\n{ platform:macOS, arch:arm64, id:XXXXXXXX-XXXXXXXXXXXX, name:My Mac }\n{ platform:iOS, arch:arm64, id:XXXXXXXX-XXXXXXXXXXXX, name:TestDevice1 }\n{ platform:iOS, arch:arm64, id:XXXXXXXX-XXXXXXXXXXXX, name:TestDevice2 }\n",
  "command": "xcodebuild",
  "args": [
    "-showBuildSettings",
    "-scheme",
    "Feature",
    "-workspace",
    "/path/to/project/App.xcworkspace",
    "-configuration",
    "Debug",
    "-json"
  ],
  "cwd": "/path/to/project"
}
{
  "message": "Error fetching build settings",
  "level": 3,
  "time": "2024-10-19T01:35:52.823Z",
  "stackTrace": "Error: Error fetching build settings\n    at new qv (/Users/dummyuser/.vscode/extensions/sweetpad.sweetpad-0.1.37/out/extension.js:233:12440)\n    at /Users/dummyuser/.vscode/extensions/sweetpad.sweetpad-0.1.37/out/extension.js:235:27\n    at Generator.next (<anonymous>)\n    at o (/Users/dummyuser/.vscode/extensions/sweetpad.sweetpad-0.1.37/out/extension.js:1:1350)\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)",
  "command": "sweetpad.build.test",
  "error": {}
}

[hyzyla]

@funzin Could you manually execute this command show me the output?

xcodebuild \
  -showBuildSettings \
  -scheme Feature \
  -workspace /path/to/project/App.xcworkspace \
  -configuration Debug \
  -json

I think you are correct that the problem is due to multiple destinations, but there may also be some additional information in the stdout

[funzin]

@hyzyla

Thank you for you reply.
This is the output log.(with masking id)

--- xcodebuild: WARNING: Using the first of multiple matching destinations:
{ platform:macOS, arch:arm64e, id:XXXXXXXX-XXXXXXXXXXXXXXX, name:My Mac }
{ platform:macOS, arch:arm64, id:XXXXXXXX-XXXXXXXXXXXXXXX, name:My Mac }
{ platform:macOS, arch:x86_64, id:XXXXXXXX-XXXXXXXXXXXXXXX, name:My Mac }
{ platform:macOS, arch:arm64e, variant:Mac Catalyst, id:XXXXXXXX-XXXXXXXXXXXXXXX, name:My Mac }
{ platform:macOS, arch:arm64, variant:Mac Catalyst, id:XXXXXXXX-XXXXXXXXXXXXXXX, name:My Mac }
{ platform:macOS, arch:x86_64, variant:Mac Catalyst, id:XXXXXXXX-XXXXXXXXXXXXXXX, name:My Mac }
{ platform:macOS, arch:arm64, variant:DriverKit, id:XXXXXXXX-XXXXXXXXXXXXXXX, name:My Mac }
{ platform:macOS, arch:arm64, variant:Designed for [iPad,iPhone], id:XXXXXXXX-XXXXXXXXXXXXXXX, name:My Mac }
{ platform:DriverKit, name:Any DriverKit Host }
{ platform:iOS, id:dvtdevice-DVTiPhonePlaceholder-iphoneos:placeholder, name:Any iOS Device }
{ platform:iOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-iphonesimulator:placeholder, name:Any iOS Simulator Device }
{ platform:macOS, name:Any Mac }
{ platform:macOS, variant:Mac Catalyst, name:Any Mac }
{ platform:iOS, arch:arm64e, id:XXXXXXXX-XXXXXXXXXXXXXXX, name:funzin }
{ platform:iOS, arch:arm64, id:XXXXXXXX-XXXXXXXXXXXXXXX, name:funzin }
{ platform:iOS Simulator, id:XXXXXXXX-XXXXXXXXXXXXXXX, OS:17.5, name:iPad (6th generation) }
{ platform:iOS Simulator, id:XXXXXXXX-XXXXXXXXXXXXXXX, OS:17.5, name:iPad (6th generation) }
{ platform:iOS Simulator, id:XXXXXXXX-XXXXXXXXXXXXXXX, OS:17.5, name:iPhone 15 Pro }
{ platform:iOS Simulator, id:XXXXXXXX-XXXXXXXXXXXXXXX, OS:17.5, name:iPhone 15 Pro }
{ platform:iOS Simulator, id:XXXXXXXX-XXXXXXXXXXXXXXX, OS:18.0, name:iPhone 16 Pro }
{ platform:iOS Simulator, id:XXXXXXXX-XXXXXXXXXXXXXXX, OS:18.0, name:iPhone 16 Pro }
[
]

[funzin]

I understand.

It seems that when specifying a module(e.g. Feature ) managed by SPM (Swift Package Manager), the xcodebuild -showBuildSettings command returns an empty output. As a result, if you attempt to run tests for an SPM-managed module through xcodebuild, the current setup does not allow you to retrieve build settings.

For now, you might need to explore alternative methods to handle test execution for SPM modules

https://github.com/sweetpad-dev/sweetpad/blob/main/src/common/cli/scripts.ts#L136-L185

[hyzyla]

Oh, thanks. Information about SPM is super helpful here. Could you make some simple test project with the same structure as your project? Or could you explain how to reproduce this structure?

[funzin]

@hyzyla

Could you check the project?
SwiftExample.zip

When I checked the error again, a different one was output.

{
  "message": "Error fetching build settings",
  "level": 3,
  "time": "2024-10-21T11:41:01.213Z",
  "stackTrace": "Error: Error fetching build settings\n    at new Kv (/Users/dummy/.vscode/extensions/sweetpad.sweetpad-0.1.38/out/extension.js:233:12698)\n    at /Users/dummy/.vscode/extensions/sweetpad.sweetpad-0.1.38/out/extension.js:235:27\n    at Generator.next (<anonymous>)\n    at o (/Users/dummy/.vscode/extensions/sweetpad.sweetpad-0.1.38/out/extension.js:1:1377)\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)",
  "command": "sweetpad.build.test",
  "error": {}
}

[rpedretti]

On the same topic of SPM packages I have an example that fails with

xcodebuild: error: Failed to build workspace TestProject with scheme TestProject.: Tests in the target “TestProject” can’t be run because “TestProject” isn’t a member of the specified test plan or scheme.

when running specific tests e.g.

xcodebuild test-without-building \
    -workspace /Users/rafael.pedretti/sources/TestProject/TestProject.xcodeproj/project.xcworkspace \
    -destination 'platform=iOS Simulator,id=959B32D4-953C-49EB-B093-8A88C0654D92' \
    -scheme TestProject \
    -only-testing\:TestProject/LibOneTests/testExample

But if change the only-testing to -only-testing\:/<Lib name>/<Class name>/<test name> as -only-testing\:/LibOneTests/LibOneTests/testExample it works as expected

TestProject.zip

[wang9262]

@hyzyla

Could you check the project? SwiftExample.zip

When I checked the error again, a different one was output.

{
  "message": "Error fetching build settings",
  "level": 3,
  "time": "2024-10-21T11:41:01.213Z",
  "stackTrace": "Error: Error fetching build settings\n    at new Kv (/Users/dummy/.vscode/extensions/sweetpad.sweetpad-0.1.38/out/extension.js:233:12698)\n    at /Users/dummy/.vscode/extensions/sweetpad.sweetpad-0.1.38/out/extension.js:235:27\n    at Generator.next (<anonymous>)\n    at o (/Users/dummy/.vscode/extensions/sweetpad.sweetpad-0.1.38/out/extension.js:1:1377)\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)",
  "command": "sweetpad.build.test",
  "error": {}
}

Faced the similar problem here. In my case, I use a package.swift to generate my Xcode project，it throws the same error when i build the related scheme

[hyzyla]

Thanks frontage information. I’ll try to check this issue this weekend

[hyzyla]

@funzin, could you help me understand how to run tests directly in Xcode? I don’t have much experience working with projects like this

I understand.

It seems that when specifying a module(e.g. Feature ) managed by SPM (Swift Package Manager), the xcodebuild -showBuildSettings command returns an empty output. As a result, if you attempt to run tests for an SPM-managed module through xcodebuild, the current setup does not allow you to retrieve build settings.

For now, you might need to explore alternative methods to handle test execution for SPM modules

When I select Feature in the scheme and then run tests in the Xcode project, I get the following error:

[hyzyla]

But if change the only-testing to -only-testing:/// as -only-testing:/LibOneTests/LibOneTests/testExample it works as expected

This is a key point! As I understand it, to run tests in an SPM package, we need to specify the target name from the .testTarget in the Package.swift file, rather than using a target from the xcodebuild -list command. In the latest version, I've added support for detecting SPM packages and their test targets. Could someone check this on their projects? I'm not entirely sure if this is the right fix for this issue

[rpedretti]

I'll have some code by eod today for people to test also. I'm scanning the project to add all the tests to test explorer a do some other fixes

…

On Sun 3 Nov 2024, 12:48 Yevhenii Hyzyla, ***@***.***> wrote: But if change the only-testing to -only-testing:/// as -only-testing:/LibOneTests/LibOneTests/testExample it works as expected This is a key point! As I understand it, to run tests in an SPM package, we need to specify the target name from the .testTarget in the Package.swift file, rather than using a target from the xcodebuild -list command. In the latest version, I've added support for detecting SPM packages and their test targets. Could someone check this on their projects? I'm not entirely sure if this is the right fix for this issue ***@***.*** (view on web) <https://github.com/user-attachments/assets/586c240b-bb5e-449e-b904-a6f331abefd4> — Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFE6EILXKKTMSPQSV7PID3Z6YLTPAVCNFSM6AAAAABQG7LKAOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJTGQYTMNBWGM> . You are receiving this because you commented.Message ID: ***@***.***>

[wang9262]

But if change the only-testing to -only-testing:/// as -only-testing:/LibOneTests/LibOneTests/testExample it works as expected

This is a key point! As I understand it, to run tests in an SPM package, we need to specify the target name from the .testTarget in the Package.swift file, rather than using a target from the xcodebuild -list command. In the latest version, I've added support for detecting SPM packages and their test targets. Could someone check this on their projects? I'm not entirely sure if this is the right fix for this issue

In my case, the problem still exists. You can easily reproduce using this repo. The generated folder just using Package.swift to generate project

[rpedretti]

I have a PR to try to get most of the test data from the project itself: #67
To try it on your local just checkout the branch and

• go to project folder
• npm install
• npm run build && npm run vscode:prepublish && vsce package
• remove Sweetpad from the extensions
• install the extension from the generated vsix