Preheat the oven to 350°. Line a 9x5-inch loaf pan with parchment paper.
In a large bowl, beat the brown and granulated sugar with the eggs until fluffy, about 2 minutes. Add the pumpkin, oil, milk, and vanilla.
In a separate bowl, combine the flour, baking soda, powder, cinnamon, salt, and pumpkin spice. Stir gently to combine, then mix with the wet ingredients.
In a separate bowl, beat the cream, sugar, egg, and vanilla until creamy.
Pour half of the batter into the bread loaf, add the cream cheese mixture, then top with the remaining batter.
Distribute the chopped walnuts over the batter.
Bake for 50-60 minutes, or until a toothpick inserted into the center comes clean.
Remove the loaf from the pan and transfer it to a wire rack to cool.
Whisk together the sugar and milk for the glaze. Drizzle over the top of the loaf.