Since I had to make the waves bigger than what the RPG Maker engine allows (which is only 1/4 of a tile), I had to put the "inside curves" on a seperate autotile. In RPG Maker, simply assign both autotiles to the map/tileset, and map the ocean area with the "Ocean01" autotile. For those tiles that need an "inner curve", select autotile "Ocean02", double click on it to select the correct variant, and place it on the map. In case you have no more free autotile slots for your map, I also included the "Ocean02" autotile as charset format to be placed as events.