Pet owner guilt is the worst kind of guilt. It's the sort of guilt that normal people don't get. They look at you in a way that tells you they think you are weird. This has happened to me more than once.
If it makes you feel any better, I once had an apartment with an enclosed sundeck, and once trapped my cat out there without knowing it because the door had an auto close hinge. The cat used this to his advantage and would always sneek in as the door was self closing, but I usually caught him or realized it right away. There was one time where I left him out there, with no water for at least a day in summer, because he'd gotten out there without me knowing it, then I'd left of work. He just sort of casually came out when I realized and let him in - like nothing happend. The auto- close turned into his enemy later when he was again trying to sneek out on to the deck and it slammed shut on his tail, trapping his tail between the door and frame so badly that I had to open the door for it to get free. He hid for three days straight, and I'd only hear him come out at night for food, water, and litter box. If I got up, he'd hide again. He's a rescue cat, and had been horribly abused before, and it had taken me over six months to get him to trust me, so I felt soo bad. After three days, he came out, but wouldn't let me touch him for a while. Then he sort of forgot, and went back to being his edgy but loving self. He has a perminant kink in his tail now. I'm pretty sure he broke his tail, but was asured by a vet that nothing could be done, and as long as he was not otherwise injured that he'd be fine. I still feel bad about it to this day though.