In a saucepan off the heat, combine the sugar, cornstarch and flour. Whisk in the egg, egg yolk and vanilla. Stir in the milk. Bring to a boil over medium heat, stirring constantly with a whisk, about 5 minutes, taking care to scrape the bottom and sides of the pan.