[Obs AI Assistant] Re-index KB if the index doesn't support semantic text
#210204
Assignees
Labels
bug
Fixes for quality problems that affect the customer experience
Team:Obs AI Assistant
Observability AI Assistant
Comments
sorenlouv
added
bug
|
Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant) |
sorenlouv
changed the title
semantic textsemantic text
sorenlouv
added a commit
that referenced
this issue
Feb 17, 2025
…t` bug (#210386) Closes #210204 This will automatically re-index the knowledge base if upon adding a KB entry there is this error: > The [sparse_vector] field type is not supported on indices created on versions 8.0 to 8.10 That error means that semantic_text is not supported in the given index, and it should therefore be re-indexed. **How to test this PR:** **8.10** - `git checkout -B 8.10 origin/8.10` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` **8.19** - `git checkout -B 8.19 origin/8.x` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` - Install Knowledge base - Try adding an item to KB (it should fail ❌️) **9.1.0** - `gh pr checkout 210386` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` - Try adding an item to KB (it should succeed ✅️) **TODO:** - Add an upgrade test that covers this flow --------- Co-authored-by: Viduni Wickramarachchi <[email protected]>
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 17, 2025
…t` bug (elastic#210386) Closes elastic#210204 This will automatically re-index the knowledge base if upon adding a KB entry there is this error: > The [sparse_vector] field type is not supported on indices created on versions 8.0 to 8.10 That error means that semantic_text is not supported in the given index, and it should therefore be re-indexed. **How to test this PR:** **8.10** - `git checkout -B 8.10 origin/8.10` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` **8.19** - `git checkout -B 8.19 origin/8.x` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` - Install Knowledge base - Try adding an item to KB (it should fail ❌️) **9.1.0** - `gh pr checkout 210386` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` - Try adding an item to KB (it should succeed ✅️) **TODO:** - Add an upgrade test that covers this flow --------- Co-authored-by: Viduni Wickramarachchi <[email protected]> (cherry picked from commit df67a09)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 17, 2025
…t` bug (elastic#210386) Closes elastic#210204 This will automatically re-index the knowledge base if upon adding a KB entry there is this error: > The [sparse_vector] field type is not supported on indices created on versions 8.0 to 8.10 That error means that semantic_text is not supported in the given index, and it should therefore be re-indexed. **How to test this PR:** **8.10** - `git checkout -B 8.10 origin/8.10` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` **8.19** - `git checkout -B 8.19 origin/8.x` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` - Install Knowledge base - Try adding an item to KB (it should fail ❌️) **9.1.0** - `gh pr checkout 210386` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` - Try adding an item to KB (it should succeed ✅️) **TODO:** - Add an upgrade test that covers this flow --------- Co-authored-by: Viduni Wickramarachchi <[email protected]> (cherry picked from commit df67a09)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 17, 2025
…t` bug (elastic#210386) Closes elastic#210204 This will automatically re-index the knowledge base if upon adding a KB entry there is this error: > The [sparse_vector] field type is not supported on indices created on versions 8.0 to 8.10 That error means that semantic_text is not supported in the given index, and it should therefore be re-indexed. **How to test this PR:** **8.10** - `git checkout -B 8.10 origin/8.10` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` **8.19** - `git checkout -B 8.19 origin/8.x` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` - Install Knowledge base - Try adding an item to KB (it should fail ❌️) **9.1.0** - `gh pr checkout 210386` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` - Try adding an item to KB (it should succeed ✅️) **TODO:** - Add an upgrade test that covers this flow --------- Co-authored-by: Viduni Wickramarachchi <[email protected]> (cherry picked from commit df67a09)
sorenlouv
added a commit
to sorenlouv/kibana
that referenced
this issue
Feb 17, 2025
…t` bug (elastic#210386) Closes elastic#210204 This will automatically re-index the knowledge base if upon adding a KB entry there is this error: > The [sparse_vector] field type is not supported on indices created on versions 8.0 to 8.10 That error means that semantic_text is not supported in the given index, and it should therefore be re-indexed. **How to test this PR:** **8.10** - `git checkout -B 8.10 origin/8.10` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` **8.19** - `git checkout -B 8.19 origin/8.x` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` - Install Knowledge base - Try adding an item to KB (it should fail ❌️) **9.1.0** - `gh pr checkout 210386` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` - Try adding an item to KB (it should succeed ✅️) **TODO:** - Add an upgrade test that covers this flow --------- Co-authored-by: Viduni Wickramarachchi <[email protected]> (cherry picked from commit df67a09)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 17, 2025
…t` bug (elastic#210386) Closes elastic#210204 This will automatically re-index the knowledge base if upon adding a KB entry there is this error: > The [sparse_vector] field type is not supported on indices created on versions 8.0 to 8.10 That error means that semantic_text is not supported in the given index, and it should therefore be re-indexed. **How to test this PR:** **8.10** - `git checkout -B 8.10 origin/8.10` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` **8.19** - `git checkout -B 8.19 origin/8.x` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` - Install Knowledge base - Try adding an item to KB (it should fail ❌️) **9.1.0** - `gh pr checkout 210386` - Start Kibana: - `nvm use && yarn kbn bootstrap && yarn start` - Start ES - `nvm use && yarn es snapshot --license trial --E path.data="/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing"` - Try adding an item to KB (it should succeed ✅️) **TODO:** - Add an upgrade test that covers this flow --------- Co-authored-by: Viduni Wickramarachchi <[email protected]> (cherry picked from commit df67a09)
kibanamachine
added a commit
that referenced
this issue
Feb 17, 2025
…ic_text` bug (#210386) (#211509) # Backport This will backport the following commits from `main` to `9.0`: - [[Obs AI Assistant] Add KB re-indexing when encountering `semantic_text` bug (#210386)](#210386) <!--- Backport version: 9.6.5 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Søren Louv-Jansen","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-02-17T17:20:50Z","message":"[Obs AI Assistant] Add KB re-indexing when encountering `semantic_text` bug (#210386)\n\nCloses https://github.com/elastic/kibana/issues/210204\n\nThis will automatically re-index the knowledge base if upon adding a KB\nentry there is this error:\n\n> The [sparse_vector] field type is not supported on indices created on\nversions 8.0 to 8.10\n\nThat error means that semantic_text is not supported in the given index,\nand it should therefore be re-indexed.\n\n**How to test this PR:**\n\n**8.10**\n- `git checkout -B 8.10 origin/8.10`\n- Start Kibana:\n - `nvm use && yarn kbn bootstrap && yarn start`\n- Start ES\n- `nvm use && yarn es snapshot --license trial --E\npath.data=\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\"`\n\n**8.19**\n- `git checkout -B 8.19 origin/8.x`\n- Start Kibana:\n - `nvm use && yarn kbn bootstrap && yarn start`\n- Start ES\n- `nvm use && yarn es snapshot --license trial --E\npath.data=\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\"`\n- Install Knowledge base\n- Try adding an item to KB (it should fail ❌️)\n\n**9.1.0**\n- `gh pr checkout 210386`\n- Start Kibana:\n - `nvm use && yarn kbn bootstrap && yarn start`\n- Start ES\n- `nvm use && yarn es snapshot --license trial --E\npath.data=\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\"`\n- Try adding an item to KB (it should succeed ✅️)\n\n**TODO:**\n\n- Add an upgrade test that covers this flow\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <[email protected]>","sha":"df67a09afab22521dfa9ff3ec3a4f624a039c462","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","v9.0.0","backport:prev-minor","Team:Obs AI Assistant","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Obs AI Assistant] Add KB re-indexing when encountering `semantic_text` bug","number":210386,"url":"https://github.com/elastic/kibana/pull/210386","mergeCommit":{"message":"[Obs AI Assistant] Add KB re-indexing when encountering `semantic_text` bug (#210386)\n\nCloses https://github.com/elastic/kibana/issues/210204\n\nThis will automatically re-index the knowledge base if upon adding a KB\nentry there is this error:\n\n> The [sparse_vector] field type is not supported on indices created on\nversions 8.0 to 8.10\n\nThat error means that semantic_text is not supported in the given index,\nand it should therefore be re-indexed.\n\n**How to test this PR:**\n\n**8.10**\n- `git checkout -B 8.10 origin/8.10`\n- Start Kibana:\n - `nvm use && yarn kbn bootstrap && yarn start`\n- Start ES\n- `nvm use && yarn es snapshot --license trial --E\npath.data=\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\"`\n\n**8.19**\n- `git checkout -B 8.19 origin/8.x`\n- Start Kibana:\n - `nvm use && yarn kbn bootstrap && yarn start`\n- Start ES\n- `nvm use && yarn es snapshot --license trial --E\npath.data=\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\"`\n- Install Knowledge base\n- Try adding an item to KB (it should fail ❌️)\n\n**9.1.0**\n- `gh pr checkout 210386`\n- Start Kibana:\n - `nvm use && yarn kbn bootstrap && yarn start`\n- Start ES\n- `nvm use && yarn es snapshot --license trial --E\npath.data=\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\"`\n- Try adding an item to KB (it should succeed ✅️)\n\n**TODO:**\n\n- Add an upgrade test that covers this flow\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <[email protected]>","sha":"df67a09afab22521dfa9ff3ec3a4f624a039c462"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/211504","number":211504,"state":"OPEN"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/211497","number":211497,"state":"MERGED","mergeCommit":{"sha":"bac6d90d65f350bb58e999ebf63a977bc4e6714f","message":"[8.18] [Obs AI Assistant] Add KB re-indexing when encountering `semantic_text` bug (#210386) (#211497)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.18`:\n- [[Obs AI Assistant] Add KB re-indexing when encountering\n`semantic_text` bug\n(#210386)](https://github.com/elastic/kibana/pull/210386)\n\n<!--- Backport version: 9.6.5 -->\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n<!--BACKPORT [{\"author\":{\"name\":\"Søren\nLouv-Jansen\",\"email\":\"[email protected]\"},\"sourceCommit\":{\"committedDate\":\"2025-02-17T17:20:50Z\",\"message\":\"[Obs\nAI Assistant] Add KB re-indexing when encountering `semantic_text` bug\n(#210386)\\n\\nCloses\nhttps://github.com//issues/210204\\n\\nThis will\nautomatically re-index the knowledge base if upon adding a KB\\nentry\nthere is this error:\\n\\n> The [sparse_vector] field type is not\nsupported on indices created on\\nversions 8.0 to 8.10\\n\\nThat error\nmeans that semantic_text is not supported in the given index,\\nand it\nshould therefore be re-indexed.\\n\\n**How to test this\nPR:**\\n\\n**8.10**\\n- `git checkout -B 8.10 origin/8.10`\\n- Start\nKibana:\\n - `nvm use && yarn kbn bootstrap && yarn start`\\n- Start ES\\n-\n`nvm use && yarn es snapshot --license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n\\n**8.19**\\n-\n`git checkout -B 8.19 origin/8.x`\\n- Start Kibana:\\n - `nvm use && yarn\nkbn bootstrap && yarn start`\\n- Start ES\\n- `nvm use && yarn es snapshot\n--license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n-\nInstall Knowledge base\\n- Try adding an item to KB (it should fail\n❌️)\\n\\n**9.1.0**\\n- `gh pr checkout 210386`\\n- Start Kibana:\\n - `nvm\nuse && yarn kbn bootstrap && yarn start`\\n- Start ES\\n- `nvm use && yarn\nes snapshot --license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n-\nTry adding an item to KB (it should succeed ✅️)\\n\\n**TODO:**\\n\\n- Add an\nupgrade test that covers this flow\\n\\n---------\\n\\nCo-authored-by:\nViduni Wickramarachchi\n<[email protected]>\",\"sha\":\"df67a09afab22521dfa9ff3ec3a4f624a039c462\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:fix\",\"backport:prev-major\",\"Team:Obs\nAI Assistant\",\"v9.1.0\"],\"title\":\"[Obs AI Assistant] Add KB re-indexing\nwhen encountering `semantic_text`\nbug\",\"number\":210386,\"url\":\"https://github.com/elastic/kibana/pull/210386\",\"mergeCommit\":{\"message\":\"[Obs\nAI Assistant] Add KB re-indexing when encountering `semantic_text` bug\n(#210386)\\n\\nCloses\nhttps://github.com//issues/210204\\n\\nThis will\nautomatically re-index the knowledge base if upon adding a KB\\nentry\nthere is this error:\\n\\n> The [sparse_vector] field type is not\nsupported on indices created on\\nversions 8.0 to 8.10\\n\\nThat error\nmeans that semantic_text is not supported in the given index,\\nand it\nshould therefore be re-indexed.\\n\\n**How to test this\nPR:**\\n\\n**8.10**\\n- `git checkout -B 8.10 origin/8.10`\\n- Start\nKibana:\\n - `nvm use && yarn kbn bootstrap && yarn start`\\n- Start ES\\n-\n`nvm use && yarn es snapshot --license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n\\n**8.19**\\n-\n`git checkout -B 8.19 origin/8.x`\\n- Start Kibana:\\n - `nvm use && yarn\nkbn bootstrap && yarn start`\\n- Start ES\\n- `nvm use && yarn es snapshot\n--license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n-\nInstall Knowledge base\\n- Try adding an item to KB (it should fail\n❌️)\\n\\n**9.1.0**\\n- `gh pr checkout 210386`\\n- Start Kibana:\\n - `nvm\nuse && yarn kbn bootstrap && yarn start`\\n- Start ES\\n- `nvm use && yarn\nes snapshot --license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n-\nTry adding an item to KB (it should succeed ✅️)\\n\\n**TODO:**\\n\\n- Add an\nupgrade test that covers this flow\\n\\n---------\\n\\nCo-authored-by:\nViduni Wickramarachchi\n<[email protected]>\",\"sha\":\"df67a09afab22521dfa9ff3ec3a4f624a039c462\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/210386\",\"number\":210386,\"mergeCommit\":{\"message\":\"[Obs\nAI Assistant] Add KB re-indexing when encountering `semantic_text` bug\n(#210386)\\n\\nCloses\nhttps://github.com//issues/210204\\n\\nThis will\nautomatically re-index the knowledge base if upon adding a KB\\nentry\nthere is this error:\\n\\n> The [sparse_vector] field type is not\nsupported on indices created on\\nversions 8.0 to 8.10\\n\\nThat error\nmeans that semantic_text is not supported in the given index,\\nand it\nshould therefore be re-indexed.\\n\\n**How to test this\nPR:**\\n\\n**8.10**\\n- `git checkout -B 8.10 origin/8.10`\\n- Start\nKibana:\\n - `nvm use && yarn kbn bootstrap && yarn start`\\n- Start ES\\n-\n`nvm use && yarn es snapshot --license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n\\n**8.19**\\n-\n`git checkout -B 8.19 origin/8.x`\\n- Start Kibana:\\n - `nvm use && yarn\nkbn bootstrap && yarn start`\\n- Start ES\\n- `nvm use && yarn es snapshot\n--license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n-\nInstall Knowledge base\\n- Try adding an item to KB (it should fail\n❌️)\\n\\n**9.1.0**\\n- `gh pr checkout 210386`\\n- Start Kibana:\\n - `nvm\nuse && yarn kbn bootstrap && yarn start`\\n- Start ES\\n- `nvm use && yarn\nes snapshot --license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n-\nTry adding an item to KB (it should succeed ✅️)\\n\\n**TODO:**\\n\\n- Add an\nupgrade test that covers this flow\\n\\n---------\\n\\nCo-authored-by:\nViduni Wickramarachchi\n<[email protected]>\",\"sha\":\"df67a09afab22521dfa9ff3ec3a4f624a039c462\"}}]}]\nBACKPORT-->\n\nCo-authored-by: Søren Louv-Jansen <[email protected]>"}},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/210386","number":210386,"mergeCommit":{"message":"[Obs AI Assistant] Add KB re-indexing when encountering `semantic_text` bug (#210386)\n\nCloses https://github.com/elastic/kibana/issues/210204\n\nThis will automatically re-index the knowledge base if upon adding a KB\nentry there is this error:\n\n> The [sparse_vector] field type is not supported on indices created on\nversions 8.0 to 8.10\n\nThat error means that semantic_text is not supported in the given index,\nand it should therefore be re-indexed.\n\n**How to test this PR:**\n\n**8.10**\n- `git checkout -B 8.10 origin/8.10`\n- Start Kibana:\n - `nvm use && yarn kbn bootstrap && yarn start`\n- Start ES\n- `nvm use && yarn es snapshot --license trial --E\npath.data=\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\"`\n\n**8.19**\n- `git checkout -B 8.19 origin/8.x`\n- Start Kibana:\n - `nvm use && yarn kbn bootstrap && yarn start`\n- Start ES\n- `nvm use && yarn es snapshot --license trial --E\npath.data=\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\"`\n- Install Knowledge base\n- Try adding an item to KB (it should fail ❌️)\n\n**9.1.0**\n- `gh pr checkout 210386`\n- Start Kibana:\n - `nvm use && yarn kbn bootstrap && yarn start`\n- Start ES\n- `nvm use && yarn es snapshot --license trial --E\npath.data=\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\"`\n- Try adding an item to KB (it should succeed ✅️)\n\n**TODO:**\n\n- Add an upgrade test that covers this flow\n\n---------\n\nCo-authored-by: Viduni Wickramarachchi <[email protected]>","sha":"df67a09afab22521dfa9ff3ec3a4f624a039c462"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/211498","number":211498,"state":"MERGED","mergeCommit":{"sha":"ddd2ccd0a4246d5e4f2bc73f6e9716ee666025e2","message":"[8.x] [Obs AI Assistant] Add KB re-indexing when encountering `semantic_text` bug (#210386) (#211498)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.x`:\n- [[Obs AI Assistant] Add KB re-indexing when encountering\n`semantic_text` bug\n(#210386)](https://github.com/elastic/kibana/pull/210386)\n\n<!--- Backport version: 9.6.5 -->\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n<!--BACKPORT [{\"author\":{\"name\":\"Søren\nLouv-Jansen\",\"email\":\"[email protected]\"},\"sourceCommit\":{\"committedDate\":\"2025-02-17T17:20:50Z\",\"message\":\"[Obs\nAI Assistant] Add KB re-indexing when encountering `semantic_text` bug\n(#210386)\\n\\nCloses\nhttps://github.com//issues/210204\\n\\nThis will\nautomatically re-index the knowledge base if upon adding a KB\\nentry\nthere is this error:\\n\\n> The [sparse_vector] field type is not\nsupported on indices created on\\nversions 8.0 to 8.10\\n\\nThat error\nmeans that semantic_text is not supported in the given index,\\nand it\nshould therefore be re-indexed.\\n\\n**How to test this\nPR:**\\n\\n**8.10**\\n- `git checkout -B 8.10 origin/8.10`\\n- Start\nKibana:\\n - `nvm use && yarn kbn bootstrap && yarn start`\\n- Start ES\\n-\n`nvm use && yarn es snapshot --license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n\\n**8.19**\\n-\n`git checkout -B 8.19 origin/8.x`\\n- Start Kibana:\\n - `nvm use && yarn\nkbn bootstrap && yarn start`\\n- Start ES\\n- `nvm use && yarn es snapshot\n--license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n-\nInstall Knowledge base\\n- Try adding an item to KB (it should fail\n❌️)\\n\\n**9.1.0**\\n- `gh pr checkout 210386`\\n- Start Kibana:\\n - `nvm\nuse && yarn kbn bootstrap && yarn start`\\n- Start ES\\n- `nvm use && yarn\nes snapshot --license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n-\nTry adding an item to KB (it should succeed ✅️)\\n\\n**TODO:**\\n\\n- Add an\nupgrade test that covers this flow\\n\\n---------\\n\\nCo-authored-by:\nViduni Wickramarachchi\n<[email protected]>\",\"sha\":\"df67a09afab22521dfa9ff3ec3a4f624a039c462\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:fix\",\"backport:prev-major\",\"Team:Obs\nAI Assistant\",\"v9.1.0\"],\"title\":\"[Obs AI Assistant] Add KB re-indexing\nwhen encountering `semantic_text`\nbug\",\"number\":210386,\"url\":\"https://github.com/elastic/kibana/pull/210386\",\"mergeCommit\":{\"message\":\"[Obs\nAI Assistant] Add KB re-indexing when encountering `semantic_text` bug\n(#210386)\\n\\nCloses\nhttps://github.com//issues/210204\\n\\nThis will\nautomatically re-index the knowledge base if upon adding a KB\\nentry\nthere is this error:\\n\\n> The [sparse_vector] field type is not\nsupported on indices created on\\nversions 8.0 to 8.10\\n\\nThat error\nmeans that semantic_text is not supported in the given index,\\nand it\nshould therefore be re-indexed.\\n\\n**How to test this\nPR:**\\n\\n**8.10**\\n- `git checkout -B 8.10 origin/8.10`\\n- Start\nKibana:\\n - `nvm use && yarn kbn bootstrap && yarn start`\\n- Start ES\\n-\n`nvm use && yarn es snapshot --license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n\\n**8.19**\\n-\n`git checkout -B 8.19 origin/8.x`\\n- Start Kibana:\\n - `nvm use && yarn\nkbn bootstrap && yarn start`\\n- Start ES\\n- `nvm use && yarn es snapshot\n--license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n-\nInstall Knowledge base\\n- Try adding an item to KB (it should fail\n❌️)\\n\\n**9.1.0**\\n- `gh pr checkout 210386`\\n- Start Kibana:\\n - `nvm\nuse && yarn kbn bootstrap && yarn start`\\n- Start ES\\n- `nvm use && yarn\nes snapshot --license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n-\nTry adding an item to KB (it should succeed ✅️)\\n\\n**TODO:**\\n\\n- Add an\nupgrade test that covers this flow\\n\\n---------\\n\\nCo-authored-by:\nViduni Wickramarachchi\n<[email protected]>\",\"sha\":\"df67a09afab22521dfa9ff3ec3a4f624a039c462\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/210386\",\"number\":210386,\"mergeCommit\":{\"message\":\"[Obs\nAI Assistant] Add KB re-indexing when encountering `semantic_text` bug\n(#210386)\\n\\nCloses\nhttps://github.com//issues/210204\\n\\nThis will\nautomatically re-index the knowledge base if upon adding a KB\\nentry\nthere is this error:\\n\\n> The [sparse_vector] field type is not\nsupported on indices created on\\nversions 8.0 to 8.10\\n\\nThat error\nmeans that semantic_text is not supported in the given index,\\nand it\nshould therefore be re-indexed.\\n\\n**How to test this\nPR:**\\n\\n**8.10**\\n- `git checkout -B 8.10 origin/8.10`\\n- Start\nKibana:\\n - `nvm use && yarn kbn bootstrap && yarn start`\\n- Start ES\\n-\n`nvm use && yarn es snapshot --license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n\\n**8.19**\\n-\n`git checkout -B 8.19 origin/8.x`\\n- Start Kibana:\\n - `nvm use && yarn\nkbn bootstrap && yarn start`\\n- Start ES\\n- `nvm use && yarn es snapshot\n--license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n-\nInstall Knowledge base\\n- Try adding an item to KB (it should fail\n❌️)\\n\\n**9.1.0**\\n- `gh pr checkout 210386`\\n- Start Kibana:\\n - `nvm\nuse && yarn kbn bootstrap && yarn start`\\n- Start ES\\n- `nvm use && yarn\nes snapshot --license trial\n--E\\npath.data=\\\"/Users/sorenlouv/elastic/kbn_es_data/upgrade_testing\\\"`\\n-\nTry adding an item to KB (it should succeed ✅️)\\n\\n**TODO:**\\n\\n- Add an\nupgrade test that covers this flow\\n\\n---------\\n\\nCo-authored-by:\nViduni Wickramarachchi\n<[email protected]>\",\"sha\":\"df67a09afab22521dfa9ff3ec3a4f624a039c462\"}}]}]\nBACKPORT-->\n\nCo-authored-by: Søren Louv-Jansen <[email protected]>"}}]}] BACKPORT--> Co-authored-by: Søren Louv-Jansen <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Timeline
sparse_vectorfield for semantic processing.semantic_textfield.semantic_textfieldProblem
Any user on 8.10 will have a KB index that does not support
semantic_text. This means that if a user who has used 8.10 upgrades to 8.17 or later, they will no longer be able to ingest documents into the knowledge base.Reproduce
GET .kibana-observability-ai-assistant-kb)Solution
Upon adding documents to the KB we should catch the error. We should re-index the KB index if the error thrown is:
Note: We should make sure to block writes to the index as soon as we start re-indexing in order to avoid multiple users starting the migration.
The text was updated successfully, but these errors were encountered: