According to the pilot, following preflight inspection he attempted to start the engine with the electrical starter but was unable to get the engine to turn over. He turned the magnetos to the off position and then got out of the airplane to "turn the prop through," but was not attempting to hand prop the engine. When the pilot rotated the propeller by hand, the engine started and the airplane immediately began moving forward. He attempted to enter the airplane but was unable to stop the airplane before it impacted an airport building. A passenger was sitting in the front right seat of the airplane and was able to evacuate the airplane after it collided with a building. The pilot stated that he should have chocked the wheels, but he was not planning on starting the engine. According to the airport manager who responded within minutes after the accident, he observed the ignition key selected to the left magneto, and then turned the key off during the post-accident emergency response. The airport manager also reported that the throttle was in the full forward position and the mixture was in the full rich position. During a post-accident examination, a Federal Aviation Administration Aviation Safety Inspector found no anomalies with the ignition wiring system or magneto P-leads. The airplane sustained substantial damage to the left wing and fuselage.