It seems like I am the only one that was not able to turn of the vibration / the vibration sound. Or maybe I am the only one that did not like that vibration
Anyway, I downloaded OpenBoard from F-droid and installed that keyboard instead. That solved my problem.