Count on half a year in which less happens. That is the real cost, and it appears on no invoice.
The four items
1. The notice period. Three to six months in which you pay for an agency that knows it is leaving. Even with the best will, their best team is already somewhere else.
2. The selection. Two to four months, plus the internal hours of everyone who briefs, listens and decides. With three agencies that quickly runs to several man-days per person. See what a pitch costs you internally.
3. The new team's learning curve. The first months go on questions you have answered before, a first proposal that is still off, and a team that does not yet sense your sensitivities. You had already paid that cost at your previous agency.
4. The handover. Source files, access, running licences, historical data. Done well it costs days; done badly it costs weeks, and sometimes work that has to be remade. See how to make sure the handover goes well.
The item nobody counts
The campaigns not made during that half year. Marketing running at half speed because the attention sits in a procedure is lost return that gets booked nowhere but exists all the same.
When it is worth that cost
When the problem is structural: a team that changed three times, proposals you systematically have to rebrief, or an agency that has outgrown your category. Otherwise you keep paying that cost every year in the form of mediocre work.
When it is not worth it: one disappointing campaign, a price discussion, or a new marketing lead bringing their own agency. See when to switch agencies.
The calculation to make
Put the four items in hours and euro next to the expected gain from switching. If you cannot name that gain, that is your answer: you are switching on instinct, and instinct is an expensive adviser here.
The item you can compress most is the second: a selection run tightly, with a longlist that holds up and three agencies instead of six. That is what IKAg does under Agency selection & pitches.
See also: how to end things with an agency properly.