Skip to main content

Mishandling of whitespaces and formatting in uploaded documents

The whitespaces and formatting are not handled correctly when they are present in files and documents. I attach one file containing a tree structured defined in a loose and casual text based format. Every model has difficulty parsing and understanding it because the formatting gets removed.

fmt-test.txt

63 bytes• Text


Now if I base64 encode the above data, and then tell the models to first decode and then try to parse, they are able to parse it correctly.

fmt-64.txt

85 bytes• Text


I tested the plain text format with commercial services like ChatGPT and Grok. They are able to parse the tree correctly, just like the models in VeniceAI when given with base64 encoded version.

This mishandling of spaces occur with prompts and system prompts too most likely.

The prompt I use with plaintext version “fmt-test.txt” above is :

The file fmt-test.txt contains a tree where each node is a number. The tree is described using a loose and casual text based format. Try to parse the tree and show to me what was parsed. If you fail to parse then state the reason for failure.

Status: Completed2 comments

Log in to comment and vote

Comments2

  • JP

    Team•

    Apr 8, 2025

    Thank you for this very detailed report. It is very much appreciated.

    This is now fixed and is being released tonight.

    • Red Glacier

      •

      Apr 8, 2025

      I tested it and it works now with uploaded files, thank you ! It also works correctly with prompts.

      But system prompts are still partially bugged. The indentation is removed. You can test it by asking any model to output some part of system prompt within some xml like tags. The output will have all the newlines preserved, but indentation will be collapsed to at max one space, regardless of how many spaces were there in original. Make sure to tell the model to output wrapped in code block. That way the formatting is visible.