Skip to main content

Increasing Context Length for Coder Models

Feature Requests

Rather than having multiple models with 32K context length, I think it would be more effective to have at least one coder model with an extended context length.

For example, 'llama 3.2 3b' seems to have a good context length, but it's not particularly useful for code and math.

On the other hand, 'llama 3.1 405b' has an 'ok' context length, but it becomes slow as the context grows.

In contrast, 'qwen2.5-coder 32b' can handle up to 128K context length, but Venice is currently using 32K.

My suggestion is:

a) Increase the context length in the current 'qwen2.5-coder 32b' model.

b) And/or consider adding a smaller version with a higher context length, such as 'qwen2.5-coder 7b'.

Status: Completed5 comments

Log in to comment and vote

Comments5

  • Yellow Apple

    •

    Feb 11, 2025

    Thank you JP, it really helped a lot :)

  • Indigo Straw

    •

    Feb 4, 2025

    Both Qwen Coder 2.5 and Deepseek 671B have been updated to support their full context via the API. We’ll continue to increase the remaining models over the coming days.

    Please let us know if that solves your needs here.

  • Rose Beaver

    •

    Jan 31, 2025

    please do support 128k+ context length, really need it for coding!

  • Aquamarine Oak

    •

    Jan 29, 2025

    This is important, for coding we need at least double the current context length and output length. Preferably, 128k+ context length is better!

  • Yellow Nucleon

    •

    Jan 28, 2025

    I agree. we also need R1 model with 128k context length