We don't have specific recommendations about the size of what should and shouldn't be cached but would recommend you review what fields have been mapped to the Always Cache category of the Report Forms Answers Cache group.
It would be recommended to remove any fields which pre-populate with, for example, personal details which pre-populate into a form from system data etc. or fields which are not used for reporting purposes.
You can move these form questions to unmapped, or to the Never Cache group. By default, any fields not mapped to the Never Cache which are contained within an active version of the form will be cached.