Support for multi master clusters exists for clusters >=1.17.x.
Check for an existing public cluster template with this feature - suffix
For custom cluster templates, make sure you pass this extra flag:
openstack coe cluster template create myclustertemplate --master-lb-enabled ...
Work is ongoing to set this at cluster creation, which will simplify the setup significantly.