I'm vaguely worried that I don't already love this baby the way I love Ellie.

I worried about that, too. It turned out fine.

I think the answer for me is that it's hard to focus on an imaginary baby the second time around when you have a real live child to focus on. Once the imaginary baby turns into a real baby it can compete for its share of the love. (Don't be surprised if it takes a little while -- at least until smile time and maybe a little longer. Ours had colic so it took longer still. But he's the apple of my eye, truly.)