In a large mixing bowl, beat butter with sugar using an electric mixer for 2 minutes or until creamy. Then, add egg yolk, milk, and almond extract. Mix until combined. Gradually add flour (a few tablespoons at a time). Mix until combined and chunky. Cover this mixing bowl with plastic wrap and place it in the refrigerator. Chill for at least one hour.
In another mixing bowl, mix together almond paste, sugar, egg yolk, milk, and almond extract.
Preheat oven to 350°F. Line a baking sheet with parchment paper.
Remove the almond pillow cookies dough from the refrigerator. Scoop up a ball using a cookie scoop or ice cream scoop. Roll the dough into the ball. Then, flatten the ball and place a tablespoon or teaspoon (depending on the size of your cookie scoop) of almond paste filling in the center of the ball. Form the ball again until the filling is completely covered with the dough. Place the almond pillow cookie on the prepared baking sheet. Then flatten it lightly. Repeat with the remaining dough.
Bake in the preheated oven for 15 minutes. Then, let them cool completely on the cooling rack. Dust with powdered sugar. Enjoy!