Preheat the oven to 350°F. Grease a 8x8" baking pan and line with parchment paper. Set aside.
In a large mixing bowl, beat together the softened butter, sugar, and brown sugar until smooth and creamy. Beat in the eggs one at a time until combined. Whisk in the vanilla and buttermilk. Sift in the flour, baking powder, baking soda, salt, and spices. Beat until there are no traces of flour.
Pour the batter into the prepared dish and bake for 35-40 minutes or until a toothpick comes out clean from the center.
Allow the cake to cool in the pan for 30 minutes, then remove from the pan and allow to cool on a cooling rack.
Frosting: Beat together butter, cream cheese, and powdered sugar until smooth. Set aside.
Milk Pour: In a small pot, bring to a boil the milk, tea leaves, cinnamon, ginger, and cardamom powder. Remove from heat and strain out the pieces, then mix in the condensed milk.
Once the cake is completely cool, poke holes into the top with a fork. Pour the milk mixture across the top and then lather on the frosting. Cut yourself a slice and enjoy!