Skip to content

Commit

Permalink
Move prompts from devpilot plugin to gateway. (openpilot-hub#46)
Browse files Browse the repository at this point in the history
* 1.0.0-RELEASE

* Feature/improve code

* limit response in md format

* fix test err

* 变更报名和项目名称

* 修复不可变list导致的add报错问题

* Release/1.1.0

* Code refactor

* 现在默认会优先去找系统中的用户名字

* fix source code jump

* Add some test cases

* add generate-tests unit test

* multi model

* Revert "Merge branch 'test-generate-unit-test' into 'develop'"

This reverts merge request !62

* update style

* fix getIcon err

* [PTAR-T-585887] Internal/develop

* Merge github

* Fix release link and some other minor format issues.

* Revert "Merge branch 'cherry-pick-1_2_0' into 'develop'"

This reverts merge request !71

* Feature/za sso

* Feature/github sso

* 2.1.0

* [PTAR-T-593451] Extract markdown contents for diff view.

* commit code complete(代码未整理)

* move code location

* update suggest code

* update suggest code

* update

* update setting(代码需要精简)

* remove unused code files(代码需要精简)

* remove unused code files(todo 代码需要精简)

* remove unused code files

* updatea CompletionMetadata

* remove unused code files

* remove unused code files(todo remove look element)

* remove unused code files

* remove unused code files

* remove unused code files

* remove unused code files

* remove unused code files

* update accept action

* update document listener

* remove unused code files

* rework code files

* rework code files

* rework code files

* rework code files

* rework code files

* rework code files

* rework code files

* rework code files

* rework code files

* rework code files

* code convert

* code to java

* add code completion config

* add code completion config

* add status bar

* add status bar

* rework code

* rework code

* rework code

* 快捷键开启与自动补全不关联

* trial improve

* auto format code(checkstyle)

* merge code

* merge code

* add completion status change

* add completion status change

* add completion status change

* add completion status change

* Feature/wx login

* Refactor AI gateway completion URL and remove a deprecated constant

Changed the `AI_GATEWAY_INSTRUCT_COMPLETION` constant in the `DefaultConst` class from `/ai/test/azure/gpt-35-turbo-instruct/completions` to `/devpilot/v1/completions`.

In the `AIGatewayServiceProvider` class, removed the usage of the deprecated constant `AI_GATEWAY_INSTRUCT_COMPLETION_ACCESS_KEY_TEMP` and its associated header.

* Update project version to "2.2.0". Create JBCefBrowser with "createBrowser()" method. Update "EMPTY_SYMBOL" value to a space character.

* Fix checkstyle errors

* Update change-notes in plugin.xml

* modify default source string and update page

* Feature/create new file multi language

* update page

* ignore json unknown

* 2.3.0

* json set

* [PTAR-PR-623929] Refactor GitlabUtil.java to improve the logic for getting the remote URL of a Git repository. The changes include adding a new parameter `VirtualFile file` to the `getRemoteUrl` method, and refactoring the logic to handle cases where the `.git/config` file is not found in the project's base path.

* DevPilot名称修改

* Release 2.3.0. Marketplace uploaded 0307.

* login unify

* delete unuse code and modify auth utils

* login type config fix

* import fix

* code completion trigger rule update

* theme bug fix and white flash bug fix

* update comment content check

* remove unused code

* update check rule info

* update check rule info

* update check rule info

* update keyup

* update keyup

* 登录改造

* fix locate bug

* Feature/chat shortcut

* [PTAR-PR-630459]Add Intellij IDEA 2024.1.x supports

* telemetry upload

* Feature/host fix

* Feature/embed hint

* Prefix suffix max limit

* Release0326

* web page fix

* update var name

* Add session clear

* Feature/version compatibility

* Default language for RAG

* Feature/improve completion

* completions conflict fix.

* update index.html to highlight code in chat view

* Add headers to the requests in CompletionFacade.java and AIGatewayServiceProvider.java

- Include User-Agent and Auth-Type headers in the request in CompletionFacade.java
- Change header "Embedded-Repos" to "Embedded-Repos-V2" and add "X-B3-Language" header in the request in AIGatewayServiceProvider.java

* Upgrade index.html for rag context

* Refactor CompletionUtils to use StringUtils for string manipulation. Improve readability and consistency in the code. Remove unused import in ManualTriggerDevPilotInlineCompletionAction.

* Refactor LlmProvider and DevPilotSuccessStreamingResponse for handling response content and streaming. Add support for RagResp containing files in streaming response.

* rag 文件定位跳转

* rag repo优化

* 修复rag路径问题

* fix label

* Feature/plugin update check

* webview 初始化时添加变量用于记录指标

* Task/exception fix

* support mulit-language and conversationWindowMemory

* Update index.html for context menu.

* add jtokkit dependence

* fix check style

* Fix some bug

* 2.4.2

* 修复checkstyle

* Prepare for github

* auto import

* auto import

* auto import

* auto import

* auto import

* auto import

* 修改readme

* 修改网关描述指向github仓库

* 修改默认服务端口为8085

* Prompts move to gateway.

* feat: add ollama support (openpilot-hub#35)

* feat: add ollama support

* fix:checkstyle problems

* bugfix:Unrecognized field "system_fingerprint" (class com.zhongan.devpilot.integrations.llms.entity.DevPilotSuccessStreamingResponse), not marked as ignorable

* list openai models using API instead of hardcoding in enum; handle list model request in provider instead of in form; use jackson to parse response instead of gson.

* remove LlamaServiceProvider.java

* Prompts move to gateway.

* Adjust request data.

* answerLanguage

* Rename prompts command.

* prompts 服务端化改造。

* code refine.

* Refine code.

---------

Co-authored-by: 赵鹏俊 <[email protected]>
Co-authored-by: xiangtianyu <[email protected]>
Co-authored-by: 陈耀 <[email protected]>
Co-authored-by: 肖亮 <[email protected]>
Co-authored-by: 赵鹏俊 <[email protected]>
Co-authored-by: yangxuefeng <[email protected]>
Co-authored-by: geeklog <[email protected]>
Co-authored-by: zhangzhisheng <[email protected]>
Co-authored-by: 毛镇 <[email protected]>
Co-authored-by: 刘海鹏 <[email protected]>
Co-authored-by: zhen.mao <[email protected]>
  • Loading branch information
12 people authored Jul 16, 2024
1 parent 08fa9a5 commit 3112009
Show file tree
Hide file tree
Showing 19 changed files with 194 additions and 267 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -24,15 +24,11 @@
import com.intellij.vcs.commit.AbstractCommitWorkflowHandler;
import com.zhongan.devpilot.DevPilotIcons;
import com.zhongan.devpilot.actions.notifications.DevPilotNotification;
import com.zhongan.devpilot.constant.DefaultConst;
import com.zhongan.devpilot.constant.PromptConst;
import com.zhongan.devpilot.integrations.llms.LlmProviderFactory;
import com.zhongan.devpilot.integrations.llms.entity.DevPilotChatCompletionRequest;
import com.zhongan.devpilot.integrations.llms.entity.DevPilotChatCompletionResponse;
import com.zhongan.devpilot.integrations.llms.entity.DevPilotMessage;
import com.zhongan.devpilot.settings.state.LanguageSettingsState;
import com.zhongan.devpilot.util.DevPilotMessageBundle;
import com.zhongan.devpilot.util.DocumentUtil;
import com.zhongan.devpilot.util.MessageUtil;

import java.io.StringWriter;
Expand All @@ -43,7 +39,7 @@
import java.util.Locale;
import java.util.Map;

import org.apache.commons.lang.StringUtils;
import org.apache.commons.lang3.StringUtils;
import org.jetbrains.annotations.NotNull;

import git4idea.repo.GitRepository;
Expand Down Expand Up @@ -83,33 +79,25 @@ public void actionPerformed(@NotNull AnActionEvent e) {
var editor = commitMessage != null ? commitMessage.getEditorField().getEditor() : null;
ApplicationManager.getApplication().invokeLater(() ->
ApplicationManager.getApplication().runWriteAction(() ->
WriteCommandAction.runWriteCommandAction(project, () -> {
if (editor != null) {
editor.getDocument().setText(" ");
}
})));
WriteCommandAction.runWriteCommandAction(project, () -> {
if (editor != null) {
editor.getDocument().setText(" ");
}
})));
generateCommitMessage(project, diff, editor);
} catch (Exception ex) {
DevPilotNotification.warn("Exception occurred while generating commit message");
}
}

private void generateCommitMessage(Project project, String diff, Editor editor) {
if (DocumentUtil.experienceEstimatedTokens(diff) + DocumentUtil.experienceEstimatedTokens(PromptConst.GENERATE_COMMIT) > DefaultConst.GPT_35_TOKEN_MAX_LENGTH) {
DevPilotNotification.warn(DevPilotMessageBundle.get("devpilot.changesview.tokens.estimation.overflow"));
}
new Task.Backgroundable(project, DevPilotMessageBundle.get("devpilot.commit.tip"), true) {
@Override
public void run(@NotNull ProgressIndicator progressIndicator) {
if (editor != null) {
((EditorEx) editor).setCaretVisible(false);

String prompt = constructPrompt(PromptConst.GENERATE_COMMIT);
String diffPrompt = PromptConst.DIFF_PREVIEW.replace("{diff}", diff);
DevPilotMessage userMessage = MessageUtil.createUserMessage(diffPrompt, "-1");
DevPilotChatCompletionRequest devPilotChatCompletionRequest = new DevPilotChatCompletionRequest();
devPilotChatCompletionRequest.getMessages().add(MessageUtil.createSystemMessage(prompt));
devPilotChatCompletionRequest.getMessages().add(userMessage);
devPilotChatCompletionRequest.getMessages().add(MessageUtil.createPromptMessage("-1", "GENERATE_COMMIT", Map.of("locale", getLocale(), "diff", diff)));
devPilotChatCompletionRequest.setStream(Boolean.FALSE);
var llmProvider = new LlmProviderFactory().getLlmProvider(project);
DevPilotChatCompletionResponse result = llmProvider.chatCompletionSync(devPilotChatCompletionRequest);
Expand Down Expand Up @@ -188,9 +176,10 @@ private String getGitDiff(AnActionEvent event, List<Change> includedChanges) {
return result.toString();
}

public String constructPrompt(String promptContent) {
public static String getLocale() {
Integer languageIndex = LanguageSettingsState.getInstance().getLanguageIndex();
Locale locale = languageIndex == 0 ? Locale.ENGLISH : Locale.SIMPLIFIED_CHINESE;
return promptContent.replace("{locale}", locale.getDisplayLanguage());
return locale.getDisplayLanguage();
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,9 @@

public class GenerateMethodCommentAction extends SelectedCodeGenerateBaseAction {

@Override
protected String getPrompt() {
return EditorActionEnum.GENERATE_METHOD_COMMENTS.getPrompt();
}

@Override
protected EditorActionEnum getEditorActionEnum() {
return EditorActionEnum.GENERATE_METHOD_COMMENTS;
return EditorActionEnum.COMMENT_METHOD;
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,17 @@
import com.zhongan.devpilot.gui.toolwindows.components.EditorInfo;
import com.zhongan.devpilot.settings.state.DevPilotLlmSettingsState;
import com.zhongan.devpilot.util.DevPilotMessageBundle;
import com.zhongan.devpilot.util.TokenUtils;
import com.zhongan.devpilot.webview.model.CodeReferenceModel;
import com.zhongan.devpilot.webview.model.MessageModel;

import java.util.Map;
import java.util.UUID;
import java.util.function.Consumer;

import org.jetbrains.annotations.NotNull;

import static com.zhongan.devpilot.actions.editor.popupmenu.PopupMenuEditorActionGroupUtil.validateResult;
import static com.zhongan.devpilot.enums.EditorActionEnum.COMMENT_METHOD;

public abstract class SelectedCodeGenerateBaseAction extends AnAction {

Expand All @@ -48,11 +49,6 @@ public void actionPerformed(@NotNull AnActionEvent e) {

Editor editor = FileEditorManager.getInstance(project).getSelectedTextEditor();
String selectedText = editor.getSelectionModel().getSelectedText();
String prompt = getPrompt().replace("{{selectedCode}}", selectedText);
if (TokenUtils.isInputExceedLimit(prompt)) {
DevPilotNotification.info(DevPilotMessageBundle.get("devpilot.notification.input.tooLong"));
return;
}

EditorInfo editorInfo = new EditorInfo(editor);
var service = project.getService(DevPilotChatToolWindowService.class);
Expand All @@ -66,11 +62,9 @@ public void actionPerformed(@NotNull AnActionEvent e) {
var codeMessage = MessageModel.buildCodeMessage(
UUID.randomUUID().toString(), System.currentTimeMillis(), showText, username, codeReference);

service.sendMessage(SessionTypeEnum.MULTI_TURN.getCode(), prompt, callback, codeMessage);
service.sendMessage(SessionTypeEnum.MULTI_TURN.getCode(), COMMENT_METHOD.name(), Map.of("selectedCode", selectedText), null, callback, codeMessage);
}

protected abstract String getPrompt();

protected abstract EditorActionEnum getEditorActionEnum();

protected abstract String getShowText();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,9 @@ public boolean collect(@NotNull PsiElement psiElement, @NotNull Editor editor, @
factory.seq(factory.textSpacePlaceholder(computeInitialWhitespace(editor, psiElement), false),
factory.icon(DevPilotIcons.SYSTEM_ICON_INLAY),
buildClickableTextChatShortcutEntry(" " + DevPilotMessageBundle.get("devpilot.inlay.shortcut.explain")
+ " | ", EditorActionEnum.EXPLAIN_THIS, psiElement),
+ " | ", EditorActionEnum.EXPLAIN_CODE, psiElement),
buildClickableTextChatShortcutEntry(DevPilotMessageBundle.get("devpilot.inlay.shortcut.fix")
+ " | ", EditorActionEnum.FIX_THIS, psiElement),
+ " | ", EditorActionEnum.FIX_CODE, psiElement),
buildClickableTextChatShortcutEntry(DevPilotMessageBundle.get("devpilot.inlay.shortcut.inlineComment")
+ " | ", EditorActionEnum.GENERATE_COMMENTS, psiElement),
buildClickableMethodCommentsShortcutEntry(DevPilotMessageBundle.get("devpilot.inlay.shortcut.methodComments") +
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,12 @@
import com.zhongan.devpilot.util.DevPilotMessageBundle;
import com.zhongan.devpilot.util.DocumentUtil;
import com.zhongan.devpilot.util.LanguageUtil;
import com.zhongan.devpilot.util.PromptTemplate;
import com.zhongan.devpilot.util.PerformanceCheckUtils;
import com.zhongan.devpilot.util.PsiFileUtil;
import com.zhongan.devpilot.util.TokenUtils;
import com.zhongan.devpilot.webview.model.CodeReferenceModel;
import com.zhongan.devpilot.webview.model.MessageModel;

import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Objects;
Expand All @@ -39,6 +39,7 @@
import javax.swing.Icon;

import static com.zhongan.devpilot.constant.PlaceholderConst.ADDITIONAL_MOCK_PROMPT;
import static com.zhongan.devpilot.constant.PlaceholderConst.ANSWER_LANGUAGE;
import static com.zhongan.devpilot.constant.PlaceholderConst.LANGUAGE;
import static com.zhongan.devpilot.constant.PlaceholderConst.MOCK_FRAMEWORK;
import static com.zhongan.devpilot.constant.PlaceholderConst.SELECTED_CODE;
Expand All @@ -47,12 +48,12 @@
public class PopupMenuEditorActionGroupUtil {

private static final Map<String, Icon> ICONS = new LinkedHashMap<>(Map.of(
EditorActionEnum.PERFORMANCE_CHECK.getLabel(), AllIcons.Plugins.Updated,
EditorActionEnum.GENERATE_COMMENTS.getLabel(), AllIcons.Actions.InlayRenameInCommentsActive,
EditorActionEnum.GENERATE_TESTS.getLabel(), AllIcons.Modules.GeneratedTestRoot,
EditorActionEnum.FIX_THIS.getLabel(), AllIcons.Actions.QuickfixBulb,
EditorActionEnum.REVIEW_CODE.getLabel(), AllIcons.Actions.PreviewDetailsVertically,
EditorActionEnum.EXPLAIN_THIS.getLabel(), AllIcons.Actions.Preview));
EditorActionEnum.CHECK_PERFORMANCE.getLabel(), AllIcons.Plugins.Updated,
EditorActionEnum.GENERATE_COMMENTS.getLabel(), AllIcons.Actions.InlayRenameInCommentsActive,
EditorActionEnum.GENERATE_TESTS.getLabel(), AllIcons.Modules.GeneratedTestRoot,
EditorActionEnum.FIX_CODE.getLabel(), AllIcons.Actions.QuickfixBulb,
EditorActionEnum.REVIEW_CODE.getLabel(), AllIcons.Actions.PreviewDetailsVertically,
EditorActionEnum.EXPLAIN_CODE.getLabel(), AllIcons.Actions.Preview));

public static void refreshActions(Project project) {
AnAction actionGroup = ActionManager.getInstance().getAction("com.zhongan.devpilot.actions.editor.popupmenu.BasicEditorAction");
Expand All @@ -63,17 +64,12 @@ public static void refreshActions(Project project) {
group.addSeparator();

var defaultActions = EditorActionConfigurationState.getInstance().getDefaultActions();
defaultActions.forEach((label, prompt) -> {
defaultActions.forEach((label) -> {
var action = new BasicEditorAction(DevPilotMessageBundle.get(label), DevPilotMessageBundle.get(label), ICONS.getOrDefault(label, AllIcons.FileTypes.Unknown)) {
@Override
protected void actionPerformed(Project project, Editor editor, String selectedText) {
ToolWindow toolWindow = ToolWindowManager.getInstance(project).getToolWindow("DevPilot");
toolWindow.show();
if (TokenUtils.isInputExceedLimit(selectedText, prompt)) {
DevPilotNotification.info(DevPilotMessageBundle.get("devpilot.notification.input.tooLong"));
return;
}

var editorActionEnum = EditorActionEnum.getEnumByLabel(label);
if (Objects.isNull(editorActionEnum)) {
return;
Expand All @@ -86,29 +82,37 @@ protected void actionPerformed(Project project, Editor editor, String selectedTe
return;
}

if (editorActionEnum == EditorActionEnum.GENERATE_COMMENTS) {
DocumentUtil.diffCommentAndFormatWindow(project, editor, result);
switch (editorActionEnum) {
case CHECK_PERFORMANCE:
// display result, and open diff window
PerformanceCheckUtils.showDiffWindow(selectedText, project, editor);
break;
case GENERATE_COMMENTS:
DocumentUtil.diffCommentAndFormatWindow(project, editor, result);
break;
default:
break;
}
};

Map<String, String> data = new HashMap<>();
data.put(SELECTED_CODE, selectedText);
EditorInfo editorInfo = new EditorInfo(editor);
PromptTemplate promptTemplate = PromptTemplate.of(prompt);
promptTemplate.setVariable(SELECTED_CODE, selectedText);
if (editorActionEnum == EditorActionEnum.GENERATE_TESTS) {
Optional.ofNullable(FileDocumentManager.getInstance().getFile(editor.getDocument()))
.map(vFile -> LanguageUtil.getLanguageByExtension(vFile.getExtension()))
.ifPresent(language -> {
promptTemplate.setVariable(LANGUAGE, language.getLanguageName());
promptTemplate.setVariable(TEST_FRAMEWORK, language.getDefaultTestFramework());
promptTemplate.setVariable(MOCK_FRAMEWORK, language.getDefaultMockFramework());
if (language.isJvmPlatform() && PsiFileUtil.isCaretInWebClass(project, editor)) {
promptTemplate.setVariable(ADDITIONAL_MOCK_PROMPT, PromptConst.MOCK_WEB_MVC);
data.put(ADDITIONAL_MOCK_PROMPT, PromptConst.MOCK_WEB_MVC);
}
data.put(LANGUAGE, language.getLanguageName());
data.put(TEST_FRAMEWORK, language.getDefaultTestFramework());
data.put(MOCK_FRAMEWORK, language.getDefaultMockFramework());
});
}
if (LanguageSettingsState.getInstance().getLanguageIndex() == 1
&& editorActionEnum != EditorActionEnum.GENERATE_COMMENTS) {
promptTemplate.appendLast(PromptConst.ANSWER_IN_CHINESE);
// todo 拿到用户真正希望回答的语言
data.put(ANSWER_LANGUAGE, "zh_CN");
}

var service = project.getService(DevPilotChatToolWindowService.class);
Expand All @@ -117,12 +121,12 @@ protected void actionPerformed(Project project, Editor editor, String selectedTe

var showText = DevPilotMessageBundle.get(label);
var codeReference = new CodeReferenceModel(editorInfo.getFilePresentableUrl(),
editorInfo.getFileName(), editorInfo.getSelectedStartLine(), editorInfo.getSelectedEndLine(), editorActionEnum);
editorInfo.getFileName(), editorInfo.getSelectedStartLine(), editorInfo.getSelectedEndLine(), editorActionEnum);

var codeMessage = MessageModel.buildCodeMessage(
UUID.randomUUID().toString(), System.currentTimeMillis(), showText, username, codeReference);
UUID.randomUUID().toString(), System.currentTimeMillis(), showText, username, codeReference);

service.sendMessage(SessionTypeEnum.MULTI_TURN.getCode(), promptTemplate.getPrompt(), callback, codeMessage);
service.sendMessage(SessionTypeEnum.MULTI_TURN.getCode(), editorActionEnum.name(), data, null, callback, codeMessage);
}
};
group.add(action);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,8 @@
import com.zhongan.devpilot.completions.requests.AutocompleteRequest;
import com.zhongan.devpilot.completions.requests.AutocompleteResponse;
import com.zhongan.devpilot.completions.requests.ResultEntry;
import com.zhongan.devpilot.enums.EditorActionEnum;
import com.zhongan.devpilot.integrations.llms.LlmProviderFactory;
import com.zhongan.devpilot.integrations.llms.entity.DevPilotInstructCompletionRequest;
import com.zhongan.devpilot.integrations.llms.entity.DevPilotMessage;
import com.zhongan.devpilot.statusBar.DevPilotStatusBarBaseWidget;
import com.zhongan.devpilot.statusBar.status.DevPilotStatusEnum;
import com.zhongan.devpilot.util.LoginUtils;
Expand All @@ -30,7 +28,6 @@

import static com.zhongan.devpilot.completions.general.StaticConfig.MAX_COMPLETIONS;
import static com.zhongan.devpilot.completions.general.StaticConfig.MAX_INSTRUCT_COMPLETION_TOKENS;
import static com.zhongan.devpilot.completions.general.StaticConfig.MIN_CHAT_COMPLETION_MESSAGE_LENGTH;
import static com.zhongan.devpilot.completions.general.StaticConfig.PREFIX_MAX_OFFSET;
import static com.zhongan.devpilot.completions.general.StaticConfig.SUFFIX_MAX_OFFSET;

Expand Down Expand Up @@ -90,16 +87,6 @@ private AutocompleteResponse retrieveCompletions(
completionAdjustment.adjustRequest(req);
}

//Simulating request response. Calling OpenAI or returning a value.
DevPilotMessage devPilotMessage = new DevPilotMessage();
devPilotMessage.setRole("user");
String content = EditorActionEnum.CODE_COMPLETIONS.getPrompt()
.replace("{{offsetCode}}", document.getText(new TextRange(req.before.lastIndexOf("\n"), offset)))
.replace("{{selectedCode}}", getFileExtension(editor) + " " + req.before)
.replace("{{maxCompletionLength}}", MIN_CHAT_COMPLETION_MESSAGE_LENGTH);
devPilotMessage.setContent(content);

devPilotMessage.setContent(req.before);
DevPilotInstructCompletionRequest request = new DevPilotInstructCompletionRequest();
request.setPrompt(req.before);
request.setSuffix(req.after);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ public class PlaceholderConst {

public final static String LANGUAGE = "language";

public final static String ANSWER_LANGUAGE = "answerLanguage";

public final static String TEST_FRAMEWORK = "testFramework";

public final static String MOCK_FRAMEWORK = "mockFramework";
Expand Down
Loading

0 comments on commit 3112009

Please sign in to comment.