Need a mode that will exercise all gem instrumentation without reporting data

[fallwith]

In a discussion in #2869, we discuss the need for running the agent in a non-production mode with all of the usual Ruby library instrumentation active but without streaming any data to New Relic that would incur cost or potentially confuse things on the UI side. I was under the impression that having agent_enabled: true and monitor_mode: false would give us everything we want, but evidently it doesn't.

We should consider supporting such a mode, either by making monitor_mode: false work that way or by introducing support for another usage pattern.

[workato-integration]

https://new-relic.atlassian.net/browse/NR-321248

[tannalynn]

monitor_mode: false should behave this way unless there is some other configuration overriding it.

[ZilvinasKucinskas]

@tannalynn Have there been any changes lately to how monitor_mode: false works?

The issue is that the default config in the test environment (agent_enabled: true & monitor_mode: false) did not catch the error discussed in #2869. The likely code path that was collecting information to send to NewRelic was never called.

This issue is about introducing a mode where all API calls to NewRelic are stubbed, but all code paths (like collecting tracing information) still run. This would greatly enhance reliability and prevent issues that happened in #2869 (impossible to catch in CI and broke production environment after deployment).

[tannalynn]

Hello @ZilvinasKucinskas,

I closed this issue because I did some testing with a rails test app today and monitor_mode does indeed work this way. First I just checked generally on some of our instrumentation, but after seeing your new comment I went back and reproduced the original view component bug and tried it with monitor_mode true and false, and I was still able to reproduce the original bug when I set monitor_mode: false but agent_enabled: true in my newrelic.yml.
As far as I can tell, monitor mode is working properly in general, and also in the context of that specific bug. The instrumentation was running, I could see the error occur, and there was no data being transmitted to new relic.

For the situation described in the previous ticket where monitor_mode was not working as expected:
It's possible the configuration options weren't correctly set, or were being overwritten by other config sources (such as env variables).
It's also possible that there is some specific situation or environment that needs to be present in order for the config not to take effect correctly. If that's the case, I'm not sure what that situation is since our test apps are working as expected, so providing a minimal reproduction would help us in investing that further.

I hope that explanation helps, and I am happy to reopen this issue if there is a reproduction available of monitor_mode not working correctly. Thank you!