Dry the Bread - If using soft, fresh bread, set the oven rack to the middle position and preheat to 300ºF (149ºC). Place the slices on a wire rack set on a baking sheet. Warm the bread until dry to the touch but not browned, about 10 to 14 minutes, flipping halfway through. Alternatively, for a small batch, use a toaster on the lowest heat setting.

Make the Custard - In a large shallow bowl, whisk together the milk, eggs, egg yolks, brown sugar, vanilla, cinnamon, nutmeg, and salt.

Dip the Bread - Soak the bread for 15 to 20 seconds on each side. Use a spatula to transfer to a separate baking sheet.

Fry the Bread - Heat a 12-inch nonstick pan over medium-low heat. Melt ½ tablespoon of butter until it bubbles. Add two pieces of bread at a time. Cook until the first side is golden brown, about 3 to 5 minutes. Flip and cook the other side until golden, about 3 to 4 minutes.

To Serve - Right before serving, sprinkle powdered sugar on top. Serve with maple syrup or desired toppings.