Using a hand mixer, whip the softened butter in a medium mixing bowl until smooth. Add the ¾ cup granulated sugar and ¾ cup light brown sugar, beating well after each addition until all ingredients are combined and smooth.
Add the eggs, one at a time, mixing well after each addition. Beat in vanilla extract.
In a separate small mixing bowl, whisk together the flour, salt, and baking soda.
Alternately add the flour mixture and the milk to the egg mixture in small increments, beating well after each addition and scraping down the sides of the bowl as needed.
Pour ½ of the batter into the prepared loaf pan and smooth to all sides, then pour about ½ of the peach swirl mixture (juice included) on top of the smoothed batter.
Add remainin half of batter and then top with the remaining half of the peach swirl mixture.
Take a butter knife and swirl the peaches through the batter, then sprinkle 1 tbsp of brown sugar over the top of the swirled batter.
Place loaf pan on the baking sheet and bake in oven at 325F for about 85 minutes, or until toothpick stuck into the center comes out completely clean and the top of the cake is a deep golden brown.
Let cake cool completely before removing it from the loaf pan. Dust with powdered sugar before serving.