Replies: 1 comment
-
|
It can be challenging to do this in general. I think what is easier and preferable is to export a model with the sequence length being dynamic (that is, a symbolic dimension called SeqLen or whatever). Once you have such a model, it is much easier to specialize it to a given length, by replacing SeqLen by 256 through out the model. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I have an onnx model of llama2, and its input and output are static shape, with 1024 sequence length. Now I'd like to make a 256 sequence length model, and I know I can convert from pytorch again. I'm just curious whether it's possible to directly modify the dims inside onnx graph?
I tried changing the dims of model.graph.input and model.graph.output, but for all the nodes in between, I'm not sure how to do this...
Beta Was this translation helpful? Give feedback.
All reactions