If you want to see where Google is Headed with its combination of Google Earth and Google Sketchup have a look at The joint efforts of two (small) swiss companies: viewtec and cybercity as well as a german one RSS
Especially viewtec provides a full range of solutions for the 3d visualization of the world ranging from the acquisition of basic data to the efficient generation of 3d-City and Megacity models and the efficient streaming of these models over the web (unfortunately only ActiveX right now).

Cybercity and RSS (Remote Sensing Solutions, not what you have thought.) are generating 3D-Models from stereogrammetry images, hence (semi-automating) the manual process used right now by Google Earth.

Have a look at the webstreaming solution TerraViewWeb yourself.