Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat/edit: adds the context for the edit command #6924

Merged
merged 5 commits into from
Feb 4, 2025

Conversation

hitesh-1997
Copy link
Contributor

@hitesh-1997 hitesh-1997 commented Feb 3, 2025

Context

  1. The PR adds the logging for the edit feature. The logging is only done for the dotCom users and the who are in the feature flag.
  2. The data will be used offline to experiment with smaller model for instant smart apply feature and evaluation to select models for selection/applying phase. This will be used to create a synthetic dataset used for smart apply.

Test plan

  1. Added unit tests
  2. Add yourself to the cody-edit-context-logs-collection-flag feature flag.
  3. Observe the telemetry events in the output log:
█ telemetry-v2 recordEvent: cody.command.edit/executed: {
    ... other fields
    "privateMetadata": {
      "intent": "edit",
      "mode": "edit",
      "source": "editor",
      "editContext": {
        "userQuery": "remove user message from the function",
        "filePath": "code-matching-eval/edits_experiments/examples/manual-examples/codium-js-rewrite.js",
        "fileContent": "import React, { useState } from 'react';\n\nexport default function Home() {\n  const [message, setMessage] = useState('');\n  const [username, setUsername] = useState('');\n\n  const onMessageChange = (event) => {\n    setMessage(event.target.value);\n  };\n\n  const onUsernameChange = (event) => {\n    setUsername(event.target.value);\n  };\n\n  return (\n    <div>\n      <input \n        type=\"text\" \n        value={username} \n        placeholder=\"Your name\" \n        onInput={onUsernameChange} \n      />\n      <input \n        type=\"text\" \n        value={message} \n        placeholder=\"Your message\" \n        onInput={onMessageChange} \n      />\n    </div>\n  );\n}",
        "selectionRange": [
          42,
          651
        ]
      },
      "model": "anthropic::2024-10-22::claude-3-5-sonnet-latest"
    },
}

const legacyMetadata = {
intent: task.intent,
mode: task.mode,
source: task.source,
...telemetryMetadata,
editContext: editContextData,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @akalia25
Similar to the previous PR, we are logging this additional context for the edit command. The additional context will be wrapped under the editContext field. Can we add this to backend ?
Also, please let me know if you spot anything in the PR related to telemetry :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @hitesh-1997 , thank you for flagging this. I have added editContext as a key that contains sensitive information.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as a reminder all the data within editContext will be redacted on v2_events, so if anything is non sensitive and would be useful to store in v2_events for analytics, feel free to put it outside the editContext key (i.e. selectionRange if needed)

metadata: {
...metadata,
recordsPrivateMetadataTranscript: editContextData === undefined ? 0 : 1,
},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @akalia25
I added recordsPrivateMetadataTranscript field to in the metadata, is this sufficient.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is perfect! 🔥 👌

Copy link
Contributor

@umpox umpox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 👍

Copy link
Contributor

@akalia25 akalia25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Data backend changes completed to mark editContext as a key containing sensitive data.

@hitesh-1997 hitesh-1997 force-pushed the hitesh/edit-context-logging branch from 297c36c to 009f001 Compare February 4, 2025 15:49
@hitesh-1997 hitesh-1997 merged commit 772e30b into main Feb 4, 2025
21 of 22 checks passed
@hitesh-1997 hitesh-1997 deleted the hitesh/edit-context-logging branch February 4, 2025 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants