When a model is overloaded Venice will display an error stating the model is overloaded try again later or something like that.ย
trying a few seconds later usually the prompt gets thru but sometimes will see the error many times requiring the user to try and submit the prompt again or switching model to one that is not overloaded.ย
Could the behavior instead be that if a model is overloaded show a warning and enter a prompt queue that is sequential so the prompt eventual gets processed if the user waits instead of playing a game of roulette where many users try spamming their prompt to the overloaded model to get it processed?
Have the option to exit this queue and switch models as well
Please authenticate to join the conversation.
New Submission
Feature Requests
New Model
9 days ago
Get notified by email when there are changes.
New Submission
Feature Requests
New Model
9 days ago
Get notified by email when there are changes.