Preheat your oven to 350°F (175°C) and grease a 9x9 inch baking dish or line it with parchment paper.
In a medium bowl, whisk together flour, baking powder, baking soda, and salt. Set aside.
In a large bowl, cream butter and sugar using a mixer until light and fluffy.
Add egg and vanilla extract. Mix until combined. Slowly pour in the buttermilk and mix on low speed.
Gradually add dry ingredients into the wet. Mix just until combined — do not overmix.
In a small bowl, mix together cinnamon and sugar for the swirl.
Pour half the batter into the dish. Sprinkle half the cinnamon sugar mixture. Add the rest of the batter, top with remaining swirl mix, and gently swirl with a knife.
For the topping, mix brown sugar, flour, melted butter, and nutmeg until crumbly. Sprinkle over the top.
Bake for 35–40 minutes, or until a toothpick comes out clean.
Cool on a rack for 15 minutes before slicing and serving.