If you're talking about 2 inches to shrink, you can move the straps and buckles (on the body of the kilt, not on the over-apron) and you're fine. If it's just an inch and you don't think you'll shrink any more, I'd be tempted to re-do the right-hand edge of the over-apron. If it's two-plus inches I would *not* do it that way, I'd move the buckles/straps.

If it's more than that, you're talking about a major rebuild if this is a nice kilt. If it's a casual, non-wool kilt, especially if it's pleated to stripe, then there are a few things I would do if I dearly loved the kilt, but they're so much work that it's almost worth it to just get another kilt.