Infant baptism is a rite in many churches, but some parents might wait a little longer to baptize their children because of family circumstances. Toddlers are baptized in the same ceremony as infants, and it is customary for them to wear a white gown or robe. Many churches honor this tradition but do not require parents to dress their children in any specific attire. In most cases, parents are encouraged to dress their children in whatever will make them comfortable. In the winter, that usually means warm clothes with layers.
Baptismal gowns are traditional, and many families share a gown that is passed down from generation to generation, used in the baptisms of all the children in the family. Gowns are considered unisex, and they look more like long nightgowns than dresses. Your toddler boy likely will be too young to know that a gown is considered "girls' clothes" by some people. Slip on a pair of long underwear or pants under the gown to keep your boy warm. Add a pair of socks and some shoes or booties. An overcoat should be enough to keep your boy warm when you are outside, and you can remove it when you come in for the baptism.
If you or your boy prefer something more gender-specific, you can dress him in a baptismal suit. You can choose a white suit in keeping with the tradition to represent purity, or if your church allows it, you can dress your toddler in any suit of your choosing. Layers are a good choice to keep your boy warm but to provide the option to cool down if the heat inside is up too high. Suits are built on layers, including a shirt, vest and jacket. You can even add a little bow tie to raise the cuteness factor. If you choose a white suit, try to wait until the last possible minute to put it on your boy. Toddlers get into everything, and it might not be so white by the time you get to the baptism.
In some churches, particularly Catholic churches, you might be required to dress your child in a baptismal robe. This robe looks similar to a choir robe, and it can be worn over other clothes. Though your boy can wear it without any other clothes on it, this can be cold in the winter. It's best to put the robe on over a suit or some other warm clothing, such as a long-sleeved shirt and some pants.
Clothing of Your Choice
Many churches allow you to dress your child in whatever makes him comfortable. That means that you can dress him in whatever you like. A sweater and some slacks would be both comfortable and warm. A long-sleeved shirt or button-up shirt would also be appropriate choices. Any clothes that are nice enough to wear to church would be suitable for baptismal attire.