Skip to content

AMD/ROCm implementation and testing (llama.cpp/vLLM) #659

@ericcurtin

Description

@ericcurtin

On completion, a person will have tested and implemented ROCm on llama.cpp and vLLM successfully with ROCm acceleration. Our ROCm vLLM container should install using something similar to this:

https://www.phoronix.com/news/AMD-ROCm-vLLM-Wheel

Requires a well supported AMD/ROCm card to do this work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions