If you are ok with the Shapes inside not being able to be moved, then you can just set the isDraggable to false on the shapes inside the Container and that took care of it for me.
↧