In a medium size saucepan, combine 4 Tablespoons (1/4 cup) of the creamy peanut butter and peanut butter chips. Heat over low heat until completely melted. Remove from heat and set aside to cool a bit.

In a large bowl using a hand mixer or the bowl of a stand mixer fitted with the whisk attachment, beat the heavy cream, sweetened condensed milk, vanilla extract, and remaining 2 Tablespoons of the peanut butter on high speed until smooth and thick (about 3-4 minutes).

Spoon 1/3 of the ice cream into the bottom of a freezer-safe container or bowl (I prefer a loaf pan). Layer half of the melted peanut butter chips evenly over the ice cream, using a spatula to spread it around. Spoon another 1/3 of the ice cream on top of the peanut butter layer, and repeat layering with remaining melted peanut butter chips and 1/3 of the ice cream. Submerge a butter knife into the container and run it up and down the length of the container to "swirl" the peanut butter. Cover tightly and freeze overnight, or at least 8 hours. Ice cream can be stored in the freezer for up to 2 weeks.