Its my guess he's just feeling a bit insecure. If you've only had him 4 days and then taken him away, then he's had a lot of upheaval in a very short time, getting used to you, your other dog and two new homes etc. How can he know you're going to come back? I'd try and establish some routine for him now so he can build up some trust and I'm sure he'll settle down before long.