Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No information how using docker image for spyt cluster #5

Open
ykuc opened this issue Jun 5, 2024 · 3 comments
Open

No information how using docker image for spyt cluster #5

ykuc opened this issue Jun 5, 2024 · 3 comments
Assignees

Comments

@ykuc
Copy link

ykuc commented Jun 5, 2024

We want to change Python version in spyt cluster, but on the exec nodes has only python3.8

We were advised to use a docker image.

spark-launch-yt \
--params '{operation_spec={tasks={history={docker_image="MY_DOCKER_IMAGE"};master={docker_image="MY_DOCKER_IMAGE"};workers={docker_image="MY_DOCKER_IMAGE"}}}}'

image can be like this. But it is too much for us... May be need only openjdk-11-jdk , unzip , python and spyt libs...

Could you publish base spyt docker image and add information in docs?

@ykuc
Copy link
Author

ykuc commented Oct 7, 2024

@Alexvsalexvsalex
Hello,
I can add docs for it.
Could you help me with the correct place for:

  • docker image example
  • doc

Look like there.
add docket example for on page or better on GitHub and link at doc?

@alextokarew
Copy link
Collaborator

@ykuc Hello. Documentation is in our main ytsaurus repository. You can find the required page here: https://github.com/ytsaurus/ytsaurus/blob/main/yt/docs/en/user-guide/data-processing/spyt/cluster/configuration.md

@ykuc
Copy link
Author

ykuc commented Oct 8, 2024

@alextokarew
It was the same place))
My pr

robot-piglet pushed a commit to ytsaurus/ytsaurus that referenced this issue Oct 10, 2024
[issue](ytsaurus/ytsaurus-spyt#5)

---

Pull Request resolved: #879
commit_hash:f40f7c91c4ad65246325a144d2df0f1c565c8cde
robot-piglet pushed a commit to ytsaurus/ytsaurus that referenced this issue Oct 10, 2024
[issue](ytsaurus/ytsaurus-spyt#5)

---

Pull Request resolved: #879
commit_hash:f40f7c91c4ad65246325a144d2df0f1c565c8cde
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants