System information

• AutoDist version:
• Are you willing to contribute it (Yes/No):

Describe the new feature and the current behavior/state
The current graph transformation (in-graph specifically) will be unacceptably slow when working on large graphs such as BERT-large. We want to benchmark and know where the slowness comes from and improve it.

Will this change the current API? How?

Describe alternatives you've considered

Additional context