Is there any way to get the ideal max_tokens value depending of the number of token input ?
#234
by
Antoine0
- opened
Hello everyone,
I'm wondering if there is an already existing table or a way to get the ideal number of token depending on the max_tokens parameter_value for relation extraction with the following shape (subject, predicate, object). Of course, I don't want to put 1k token in input for max_tokens = 100. I'm currently using meta-llama/Llama-3.3-70B-Instruct model for this purpose with a 60 sec of timeout.
Of course I can test myself but it's a bit long and painful to compare everything manually.
Thank you all in advance!