Add llama 2 model

[tikikun]

Meta just released llama 2 model, allowing commercial usage

https://ai.meta.com/resources/models-and-libraries/llama/

I have checked the model implementation and it seems different from llama_v1, maybe need a re-implementation

[Green-Sky]

link to paper: https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/

[Azeirah]

Interesting to note that the model evaluation section in their paper lists a 34b model even though the site doesn't talk about it. I wonder if it'll be available.

Does anyone have access to the models yet? I signed up but haven't received an e-mail. It's not super clear to me if it's meant to be instant or not.

[Green-Sky]

Interestingly, the paper talks about a 34B model, which is missing from the model card.
edit: @Azeirah was faster lol

[slaren]

The paper implies that they are planning to release the 34B model later.

[Green-Sky]

@Azeirah no, i did not hear back yet either.

Once your request is approved, you will receive a signed URL over email. Then run the download.sh script, passing the URL provided when prompted to start the download.

Keep in mind that the links expire after 24 hours and a certain amount of downloads. If you start seeing errors such as 403: Forbidden, you can always re-request a link.

also, they are available on hf if your email is the same https://huggingface.co/meta-llama

[Azeirah]

I was really hopeful for an alternative to gpt-4 for coding assistance, but the evaluation states their 70B model is about equivalent in performance to gpt-3.5.

Not bad, but the jump in quality from 3.5 to 4 has been what it made it really useful in day-to-day coding tasks. ;(

At the very least, it does look like the 7B and 13B variants will be amazing local chatbots for low perf devices.

[dmadisetti]

I just got access, but the download is flaky, check sums are not matching and the auth is hit or miss.
Notable is the chat specific models:

https://github.com/facebookresearch/llama/blob/main/download.sh#L24C1-L43C7

Will update if I am actually able to download these weights

[goranmoomin]

The updated model code for Llama 2 is at the same facebookresearch/llama repo, diff here: meta-llama/llama@6d4c0c2

Seems codewise, the only difference is the addition of GQA on large models, i.e. the repeat_kv part that repeats the same k/v attention heads on larger models to require less memory for the k/v cache.

According to the paper, smaller models (i.e. the 7b/13b ones) don't have GQA, so in theory it seems it should be able to run unmodified.

[dmadisetti]

Email below with tracking links stripped. Same as llama-1 for the most part. Now if it would actually download.....

---

You’re all set to start building with Llama 2.

The models listed below are now available to you as a commercial license holder. By downloading a model, you are agreeing to the terms and conditions of the license, acceptable use policy and Meta’s privacy policy.

Model weights available:

Llama-2-7b
Llama-2-7b-chat
Llama-2-13b
Llama-2-13b-chat
Llama-2-70b
Llama-2-70b-chat

With each model download, you’ll receive a copy of the Llama 2 Community License and Acceptable Use Policy, and can find all other information on the model and code on GitHub.

How to download the models:

Visit GitHub and clone [the Llama repository](https://github.com/facebookresearch/llama) from there in order to download the model code
Run the download.sh script and and follow the prompts for downloading the models.
When asked for your unique custom URL, please insert the following:
<redacted for legal reasons>
Select which model weights to download

The unique custom URL provided will remain valid for model downloads for 24 hours, and requests can be submitted multiple times.
Now you’re ready to start building with Llama 2.

Helpful tips:
Please read the instructions in the GitHub repo and use the provided code examples to understand how to best interact with the models. In particular, for the fine-tuned chat models you must use appropriate formatting and correct system/instruction tokens to get the best results from the model.

You can find additional information about how to responsibly deploy Llama models in our Responsible Use Guide.

If you need to report issues:
If you or any Llama 2 user becomes aware of any violation of our license or acceptable use policies - or any bug or issues with Llama 2 that could lead to any such violations - please report it through one of the following means:

Reporting issues with the model: Llama GitHub
Giving feedback about potentially problematic output generated by the model: [Llama output feedback](https://developers.facebook.com/llama_output_feedback)
Reporting bugs and security concerns: [Bug Bounty Program](https://facebook.com/whitehat/info)
Reporting violations of the Acceptable Use Policy: [LlamaUseReport@meta.com](mailto:LlamaUseReport@meta.com)

Subscribe to get the latest updates on Llama and Meta AI.

Meta’s GenAI Team

[swyxio]

anyone else also randomly getting

Resolving download.llamameta.net (download.llamameta.net)... 13.33.88.72, 13.33.88.62, 13.33.88.45, ...
Connecting to download.llamameta.net (download.llamameta.net)|13.33.88.72|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2023-07-19 01:24:43 ERROR 403: Forbidden.

for the small files? but /llama-2-7b-chat/consolidated.00.pth is downloading fine it seems. will share checksums when i have them

[BetaDoggo]

I tried the 7B and it seems to be working fine, with cuda acceleration as well.

[Azeirah]

anyone else also randomly getting

Resolving download.llamameta.net (download.llamameta.net)... 13.33.88.72, 13.33.88.62, 13.33.88.45, ...
Connecting to download.llamameta.net (download.llamameta.net)|13.33.88.72|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2023-07-19 01:24:43 ERROR 403: Forbidden.

for the small files? but /llama-2-7b-chat/consolidated.00.pth is downloading fine it seems. will share checksums when i have them

I genuinely just think their servers are a bit overloaded given what I see posted here. It's a big release