Excessive memory  consumption when training Mask-RCNN-Res50-FPN

Hi, I am using the scripts in gluon-cv to train the Mask-RCNN model based on Resnet-50 with FPN turned on. I noticed that the memory consumption goes up to ~11G. However, the reported memory consumption is only ~4G in [mmdetection](https://github.com/open-mmlab/mmdetection/blob/master/MODEL_ZOO.md). Is that normal or do I misconfigure the environment so that excessive memory is used? 