Nusery Time... the novel!

Okay, so I know I need to just chill out... but I'm so concerned for my little Oliver. Once again he slept through nursery. Last Sunday our ward was realigned and we switched schedules so now we start at noon. I put Oliver down for a nap for a little over an hour before we left for church hoping that he would stay awake. Berto can't attend church with us for a few months until he gets a different schedule so I'm on my own at church with Oliver and our primary class. (I'm most likely going to be released soon... ) So I took Oliver into the nursery before I went to primary and was having trouble sneaking away. He's usually fine if he doesn't see me leave the room. (No matter where we are at home, or church or anywhere I have to sneak away) So I had him playing with toys and kept heading to the door and just as I would get to the door he'd come running behind me to pull me back to whatever he was playing with. Finally I was almost out and he came running but I just kept going. Another mom was in there with her daughter (and she happens to be one of the moms of one of the kids in my class so she tried distracting him) apparently it worked for a little while and he was kicking the balls around and playing with dinosaurs. Then he did his usual and climbed up on a chair and just observed. Apparently he started dozing off while sitting on the chair so the nursery leader laid him down on the floor. At one point my class needed a potty break and the bathroom is right near where the nursery is so while they were going to the bathroom I peaked in on Oliver and that's when I found out he had been asleep. Well he slept for most of the time and woke up hysterical a few minutes before I came to pick him up. He never sleeps like this at home. I really worried that this is some kind of defense mechanism he has and that he's under so much stress being in nursery that his body just shuts itself down and goes to sleep. I know the nursery leaders are glad he's not one of the ones that cries and cries the whole time they're in the nursery... but I'm so worried for my baby boy. I worry about his social skills... but also about what's going on in the inside... am I traumatizing him?? I wonder if I had play dates with some of the other little kids if he'd recognize them and have an easier time? We still haven't gotten to know too many people and him and I pretty much spend our days together with no one else. I know I probably shouldn't read so much into all of this but it's really upsetting to me. I would love any helpful tips our outlooks from you other mommies... I'm such a basket case right now...