Support for multi master clusters exists for clusters >=1.17.x.
Check for an existing public cluster template with this feature - suffix
For custom cluster templates, make sure you pass this extra flag:
openstack coe cluster template create myclustertemplate --master-lb-enabled ...
The number of masters should ideally be odd (3, 5 or 7), since it is backed by an etcd cluster.
Work is ongoing to set this at cluster creation, which will simplify the setup significantly.