Skip to content

[Prototype] Simplify PipelineRL#427

Closed
jlamypoirier wants to merge 14 commits intodenis/new_datasetsfrom
jlp_pipeline_rl
Closed

[Prototype] Simplify PipelineRL#427
jlamypoirier wants to merge 14 commits intodenis/new_datasetsfrom
jlp_pipeline_rl

Conversation

@jlamypoirier
Copy link
Collaborator

✨ Description

  • Drop the alternative implementations of streaming datasets, keep only the consumer-group one.
  • Remove most configuration options for redis, as configurable message names are likely to bring confusion and could be hard to synchronize with PipelineRL. Use hard-coded values instead.
  • Drop the new dataset structures, since the existing ones are good enough for streaming datasets. (though we need to overlook some details like sampling configs and sample indexes not making sense)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants