Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consumer Source with relative offset definitions #538

Open
2m opened this issue Jul 11, 2018 · 3 comments
Open

Consumer Source with relative offset definitions #538

2m opened this issue Jul 11, 2018 · 3 comments

Comments

@2m
Copy link
Contributor

2m commented Jul 11, 2018

There has been a need to query certain Kafka topics with relative offsets, for example last offset, or last offset minus 100, or last offset minus 10 minutes.

Currently it is possible to do this by using metadata API to query for offsets and manually deriving an offset for which a source should be created. This could be made simpler by abstracting such logic in a user-friendly consumer source factory.

@2m 2m added the enhancement label Jul 11, 2018
@2m
Copy link
Contributor Author

2m commented Jul 11, 2018

/cc @agemooij @RayRoestenburg

@jewertow
Copy link
Contributor

I would be very happy if I could take care of it. It will be my first contribution to alpakka-kafka, so maybe could you give me some more suggestion or could you explain me what do you exactly expect from user-friendly consumer source factory? Otherwise I could suggest implementation.

@ennru ennru changed the title Consumer Source with relative offset defitions Consumer Source with relative offset definitions Aug 5, 2019
@ennru
Copy link
Member

ennru commented Aug 5, 2019

I believe this would be best implemented as something that retrieves the current offset and creates subscriptions from those offsets.
This would benefit from implementing #765 first, I guess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants