Solutions for
Please select
Try the Web App
Back to glossary

Term

Code Llama

Code Llama is Metas coding-specialized Llama 2 variant from August 2023 — open-weights, in several sizes and with base, Python and Instruct flavors.

Code Llama — explained in more detail

Code Llama is a coding-specialized model family from Meta, built on Llama 2 and released on August 24, 2023. The models were further trained on large amounts of code and code-related data and are designed to generate, complete and explain code as well as translate natural language into code. Like Llama 2, the weights are openly available and may be used commercially.

Key facts

  • Release: August 24, 2023, access via open weights (Hugging Face, Meta).
  • Sizes: 7B, 13B and 34B parameters at launch; the 70B model followed in January 2024.
  • Variants: Code Llama (base), Code Llama - Python (specialized for Python) and Code Llama - Instruct (fine-tuned for instruction following, with about 5B additional tokens).
  • Training: the initial models on roughly 500B tokens of code, the 70B model on roughly 1 trillion tokens.
  • Context: trained on 16,000-token sequences, with improvements on inputs up to 100,000 tokens.

Example / Practical use

The 7B and 13B models additionally support infilling (inserting code at a cursor position), making them suitable for IDE autocompletion. The Instruct variants are meant for dialogue and assistance use, the Python variants for Python-heavy projects. Thanks to the open weights, the models can be self-hosted and fine-tuned on your own codebases — relevant for data protection and domain-specific assistants.

Delimitation

Code Llama is the code specialization of Llama 2, distinguished from the generalist base model by code-focused further training and longer-context capabilities. Against proprietary code models such as GitHub Copilots backend or the coding modes of GPT and Claude it offers open weights, but trails current generations in performance. Successors in the open space are later Llama generations and specialized coding models such as Qwen-Coder.

More on this topic

See everything in one place:Llama
GlossaryLlama 2Llama 2 is Metas open-weights language model from July 2023 — the first Llama with a commercially usable license, in 7B, 13B and 70B sizes with a 4K context.GlossaryLlama 3Meta's open model generation, released on 18 April 2024. It launched in the 8B and 70B sizes (each in base and Instruct variants) under the Llama 3 community license; freely downloadable, for research and commercial use.GlossaryLlama 3.1Llama 3.1 (July 2024) is Metas open-weight family in 8B, 70B and 405B with a 128K context and multilinguality — the 405B was the first open model on par with GPT-4o and Claude 3.5 Sonnet.