Skip to content

Commit

Permalink
initial implementation of CodeCommit
Browse files Browse the repository at this point in the history
  • Loading branch information
pzarfos committed Aug 22, 2023
1 parent 4184f81 commit d31b66b
Show file tree
Hide file tree
Showing 9 changed files with 915 additions and 22 deletions.
68 changes: 68 additions & 0 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -255,3 +255,71 @@ docker push codiumai/pr-agent:github_app # Push to your Docker repository
5. Configure the lambda function to have a Function URL.
6. Go back to steps 8-9 of [Method 5](#method-5-run-as-a-github-app) with the function url as your Webhook URL.
The Webhook URL would look like `https://<LAMBDA_FUNCTION_URL>/api/v1/github_webhooks`

---

#### AWS CodeCommit Setup

Not all features have been added to CodeCommit yet. As of right now, CodeCommit has been implemented to run the pr-agent CLI on the command line, using AWS credentials stored in environment variables. (More features will be added in the future.) The following is a set of instructions to have pr-agent do a review of your CodeCommit pull request from the command line:

1. Create an IAM user that you will use to read CodeCommit pull requests and post comments
* Note: That user should have CLI access only, not Console access
2. Add IAM permissions to that user, to allow access to CodeCommit (see IAM Role example below)
3. Generate an Access Key for your IAM user
4. Set the Access Key and Secret using environment variables (see Access Key example below)
5. Set the `git_provider` value to `codecommit` in the `pr_agent/settings/configuration.toml` settings file
6. Set the `PYTHONPATH` to include your `pr-agent` project directory
* Option A: Add `PYTHONPATH="/PATH/TO/PROJECTS/pr-agent` to your `.env` file
* Option B: Set `PYTHONPATH` and run the CLI in one command, for example:
* `PYTHONPATH="/PATH/TO/PROJECTS/pr-agent python pr_agent/cli.py [--ARGS]`

#### AWS CodeCommit IAM Role Example

Example IAM permissions to that user to allow access to CodeCommit:

* Note: The following is a working example of IAM permissions that has read access to the repositories and write access to allow posting comments
* Note: If you only want pr-agent to review your pull requests, you can tighten the IAM permissions further, however this IAM example will work, and allow the pr-agent to post comments to the PR
* Note: You may want to replace the `"Resource": "*"` with your list of repos, to limit access to only those repos

```
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"codecommit:BatchDescribe*",
"codecommit:BatchGet*",
"codecommit:Describe*",
"codecommit:EvaluatePullRequestApprovalRules",
"codecommit:Get*",
"codecommit:List*",
"codecommit:PostComment*",
"codecommit:PutCommentReaction"
],
"Resource": "*"
}
]
}
```

#### AWS CodeCommit Access Key and Secret

Example setting the Access Key and Secret using environment variables

```sh
export AWS_ACCESS_KEY_ID="XXXXXXXXXXXXXXXX"
export AWS_SECRET_ACCESS_KEY="XXXXXXXXXXXXXXXX"
export AWS_DEFAULT_REGION="us-east-1"
```

#### AWS CodeCommit CLI Example

After you set up AWS CodeCommit using the instructions above, here is an example CLI run that tells pr-agent to **review** a given pull request.
(Replace your specific PYTHONPATH and PR URL in the example)

```sh
PYTHONPATH="/PATH/TO/PROJECTS/pr-agent" python pr_agent/cli.py \
--pr_url https://us-east-1.console.aws.amazon.com/codesuite/codecommit/repositories/MY_REPO_NAME/pull-requests/321 \
review
```
42 changes: 21 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,26 +75,26 @@ CodiumAI `PR-Agent` is an open-source tool aiming to help developers review pull

## Overview
`PR-Agent` offers extensive pull request functionalities across various git providers:
| | | GitHub | Gitlab | Bitbucket |
|-------|---------------------------------------------|:------:|:------:|:---------:|
| TOOLS | Review | :white_check_mark: | :white_check_mark: | :white_check_mark: |
| | ⮑ Inline review | :white_check_mark: | :white_check_mark: | |
| | Ask | :white_check_mark: | :white_check_mark: | :white_check_mark:
| | Auto-Description | :white_check_mark: | :white_check_mark: | |
| | Improve Code | :white_check_mark: | :white_check_mark: | |
| | Reflect and Review | :white_check_mark: | | |
| | Update CHANGELOG.md | :white_check_mark: | | |
| | | | | |
| USAGE | CLI | :white_check_mark: | :white_check_mark: | :white_check_mark: |
| | App / webhook | :white_check_mark: | :white_check_mark: | |
| | Tagging bot | :white_check_mark: | | |
| | Actions | :white_check_mark: | | |
| | | | | |
| CORE | PR compression | :white_check_mark: | :white_check_mark: | :white_check_mark: |
| | Repo language prioritization | :white_check_mark: | :white_check_mark: | :white_check_mark: |
| | Adaptive and token-aware<br />file patch fitting | :white_check_mark: | :white_check_mark: | :white_check_mark: |
| | Multiple models support | :white_check_mark: | :white_check_mark: | :white_check_mark: |
| | Incremental PR Review | :white_check_mark: | | |
| | | GitHub | Gitlab | Bitbucket | CodeCommit |
|-------|---------------------------------------------|:------:|:------:|:---------:|:----------:|
| TOOLS | Review | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
| | ⮑ Inline review | :white_check_mark: | :white_check_mark: | | |
| | Ask | :white_check_mark: | :white_check_mark: | :white_check_mark: |
| | Auto-Description | :white_check_mark: | :white_check_mark: | | |
| | Improve Code | :white_check_mark: | :white_check_mark: | | |
| | Reflect and Review | :white_check_mark: | | | |
| | Update CHANGELOG.md | :white_check_mark: | | | |
| | | | | | |
| USAGE | CLI | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
| | App / webhook | :white_check_mark: | :white_check_mark: | | |
| | Tagging bot | :white_check_mark: | | | |
| | Actions | :white_check_mark: | | | |
| | | | | | |
| CORE | PR compression | :white_check_mark: | :white_check_mark: | :white_check_mark: | |
| | Repo language prioritization | :white_check_mark: | :white_check_mark: | :white_check_mark: | |
| | Adaptive and token-aware<br />file patch fitting | :white_check_mark: | :white_check_mark: | :white_check_mark: | |
| | Multiple models support | :white_check_mark: | :white_check_mark: | :white_check_mark: | |
| | Incremental PR Review | :white_check_mark: | | | |

Examples for invoking the different tools via the CLI:
- **Review**: python cli.py --pr_url=<pr_url> review
Expand Down Expand Up @@ -153,7 +153,7 @@ Here are some advantages of PR-Agent:
- We emphasize **real-life practical usage**. Each tool (review, improve, ask, ...) has a single GPT-4 call, no more. We feel that this is critical for realistic team usage - obtaining an answer quickly (~30 seconds) and affordably.
- Our [PR Compression strategy](./PR_COMPRESSION.md) is a core ability that enables to effectively tackle both short and long PRs.
- Our JSON prompting strategy enables to have **modular, customizable tools**. For example, the '/review' tool categories can be controlled via the [configuration](./CONFIGURATION.md) file. Adding additional categories is easy and accessible.
- We support **multiple git providers** (GitHub, Gitlab, Bitbucket), **multiple ways** to use the tool (CLI, GitHub Action, GitHub App, Docker, ...), and **multiple models** (GPT-4, GPT-3.5, Anthropic, Cohere, Llama2).
- We support **multiple git providers** (GitHub, Gitlab, Bitbucket, CodeCommit), **multiple ways** to use the tool (CLI, GitHub Action, GitHub App, Docker, ...), and **multiple models** (GPT-4, GPT-3.5, Anthropic, Cohere, Llama2).
- We are open-source, and welcome contributions from the community.


Expand Down
2 changes: 2 additions & 0 deletions pr_agent/git_providers/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from pr_agent.config_loader import get_settings
from pr_agent.git_providers.bitbucket_provider import BitbucketProvider
from pr_agent.git_providers.codecommit_provider import CodeCommitProvider
from pr_agent.git_providers.github_provider import GithubProvider
from pr_agent.git_providers.gitlab_provider import GitLabProvider
from pr_agent.git_providers.local_git_provider import LocalGitProvider
Expand All @@ -8,6 +9,7 @@
'github': GithubProvider,
'gitlab': GitLabProvider,
'bitbucket': BitbucketProvider,
'codecommit': CodeCommitProvider,
'local' : LocalGitProvider
}

Expand Down
203 changes: 203 additions & 0 deletions pr_agent/git_providers/codecommit_client.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,203 @@
import boto3
import botocore


class CodeCommitDifferencesResponse:
"""
CodeCommitDifferencesResponse is the response object returned from our get_differences() function.
It maps the JSON response to member variables of this class.
"""

def __init__(self, json: dict):
before_blob = json.get("beforeBlob", {})
after_blob = json.get("afterBlob", {})

self.before_blob_id = before_blob.get("blobId", "")
self.before_blob_path = before_blob.get("path", "")
self.after_blob_id = after_blob.get("blobId", "")
self.after_blob_path = after_blob.get("path", "")
self.change_type = json.get("changeType", "")


class CodeCommitPullRequestResponse:
"""
CodeCommitPullRequestResponse is the response object returned from our get_pr() function.
It maps the JSON response to member variables of this class.
"""

def __init__(self, json: dict):
self.title = json.get("title", "")
self.description = json.get("description", "")

self.targets = []
for target in json.get("pullRequestTargets", []):
self.targets.append(CodeCommitPullRequestResponse.CodeCommitPullRequestTarget(target))

class CodeCommitPullRequestTarget:
"""
CodeCommitPullRequestTarget is a subclass of CodeCommitPullRequestResponse that
holds details about an individual target commit.
"""

def __init__(self, json: dict):
self.source_commit = json.get("sourceCommit", "")
self.source_branch = json.get("sourceReference", "")
self.destination_commit = json.get("destinationCommit", "")
self.destination_branch = json.get("destinationReference", "")


class CodeCommitClient:
"""
CodeCommitClient is a wrapper around the AWS boto3 SDK for the CodeCommit client
"""

def __init__(self):
self.boto_client = None

def _connect_boto_client(self):
try:
self.boto_client = boto3.client("codecommit")
except Exception as e:
raise ValueError(f"Failed to connect to AWS CodeCommit: {e}")

def get_differences(self, repo_name: int, destination_commit: str, source_commit: str):
"""
Get the differences between two commits in CodeCommit.
Parameters:
- repo_name: Name of the repository
- destination_commit: Commit hash you want to merge into (the "before" hash) (usually on the main or master branch)
- source_commit: Commit hash of the code you are adding (the "after" branch)
Returns:
- List of CodeCommitDifferencesResponse objects
Boto3 Documentation:
aws codecommit get-differences
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_differences.html
"""
if self.boto_client is None:
self._connect_boto_client()

# The differences response from AWS is paginated, so we need to iterate through the pages to get all the differences.
differences = []
try:
paginator = self.boto_client.get_paginator("get_differences")
for page in paginator.paginate(
repositoryName=repo_name,
beforeCommitSpecifier=destination_commit,
afterCommitSpecifier=source_commit,
):
differences.extend(page.get("differences", []))
except botocore.exceptions.ClientError as e:
raise ValueError(f"Failed to retrieve differences from CodeCommit PR #{self.pr_num}") from e

output = []
for json in differences:
output.append(CodeCommitDifferencesResponse(json))
return output

def get_file(self, repo_name: str, file_path: str, sha_hash: str, optional: bool = False):
"""
Retrieve a file from CodeCommit.
Parameters:
- repo_name: Name of the repository
- file_path: Path to the file you are retrieving
- sha_hash: Commit hash of the file you are retrieving
Returns:
- File contents
Boto3 Documentation:
aws codecommit get_file
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_file.html
"""
if not file_path:
return ""

if self.boto_client is None:
self._connect_boto_client()

try:
response = self.boto_client.get_file(repositoryName=repo_name, commitSpecifier=sha_hash, filePath=file_path)
except botocore.exceptions.ClientError as e:
# if the file does not exist, but is flagged as optional, then return an empty string
if optional and e.response["Error"]["Code"] == 'FileDoesNotExistException':
return ""
raise ValueError(f"CodeCommit cannot retrieve file '{file_path}' from repository '{repo_name}'") from e
except Exception as e:
raise ValueError(f"CodeCommit cannot retrieve file '{file_path}' from repository '{repo_name}'") from e
if "fileContent" not in response:
raise ValueError(f"File content is empty for file: {file_path}")

return response.get("fileContent", "")

def get_pr(self, pr_number: int):
"""
Get a information about a CodeCommit PR.
Parameters:
- pr_number: The PR number you are requesting
Returns:
- CodeCommitPullRequestResponse object
Boto3 Documentation:
aws codecommit get_pull_request
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/get_pull_request.html
"""
if self.boto_client is None:
self._connect_boto_client()

try:
response = self.boto_client.get_pull_request(pullRequestId=str(pr_number))
except botocore.exceptions.ClientError as e:
if e.response["Error"]["Code"] == 'PullRequestDoesNotExistException':
raise ValueError(f"CodeCommit cannot retrieve PR: PR number does not exist: {pr_number}") from e
raise ValueError(f"CodeCommit cannot retrieve PR: {pr_number}: boto client error") from e
except Exception as e:
raise ValueError(f"CodeCommit cannot retrieve PR: {pr_number}") from e

if "pullRequest" not in response:
raise ValueError("CodeCommit PR number not found: {pr_number}")

return CodeCommitPullRequestResponse(response.get("pullRequest", {}))

def publish_comment(self, repo_name: str, pr_number: int, destination_commit: str, source_commit: str, comment: str):
"""
Publish a comment to a pull request
Parameters:
- repo_name: name of the repository
- pr_number: number of the pull request
- destination_commit: The commit hash you want to merge into (the "before" hash) (usually on the main or master branch)
- source_commit: The commit hash of the code you are adding (the "after" branch)
- pr_comment: comment
Returns:
- None
Boto3 Documentation:
aws codecommit post_comment_for_pull_request
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/codecommit/client/post_comment_for_pull_request.html
"""
if self.boto_client is None:
self._connect_boto_client()

try:
self.boto_client.post_comment_for_pull_request(
pullRequestId=str(pr_number),
repositoryName=repo_name,
beforeCommitId=destination_commit,
afterCommitId=source_commit,
content=comment,
)
except botocore.exceptions.ClientError as e:
if e.response["Error"]["Code"] == 'RepositoryDoesNotExistException':
raise ValueError(f"Repository does not exist: {repo_name}") from e
if e.response["Error"]["Code"] == 'PullRequestDoesNotExistException':
raise ValueError(f"PR number does not exist: {pr_number}") from e
raise ValueError(f"Boto3 client error calling post_comment_for_pull_request") from e
except Exception as e:
raise ValueError(f"Error calling post_comment_for_pull_request") from e
Loading

0 comments on commit d31b66b

Please sign in to comment.