The male seals tend to do absolutely nothing all day bar lie on the rocks, cool down and have a good scrap. We saw a few on the rocks in the morning just last week, and a couple in the water. Book it, you won' regret it as the guys from Kaikoura Kayaks will find interesting things besides seals for you if they are unavailable that day.
They choose the best side of the headland dependent on conditions and you're 110% safe.