This is a known issue fixed from 22.1.0.0.
Check that the worker has the delete_casenote system function enabled.
Click Edit for the CaseNote to identify and remove any apostrophes found in the first 50 characters then click Save.
If this doesn't resolve the issue, please raise a new case online and reference the title of this article.