Skip to content

llama : add retrieval example #5692

@ggerganov

Description

@ggerganov

Since we now support embedding models in llama.cpp we should add a simple example to demonstrate retrieval functionality. Here is how it should work:

  • load a set of text files (provided from the command line)
  • split the text into chunks of user-configurable size, each chunk ending on a configurable stop string
  • embed all chunks using an embedding model (BERT / SBERT)
  • receive input from the command line, embed it and display the top N most relevant chunks based on cosine similarity between the input and chunk emebeddings

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions