The pilot reported that he performed a preflight inspection and then began to taxi the airplane from the seaplane base to the departure area. He reported encountering a heavy rain shower during taxi, with wind out of the south between 4-6 knots. The air traffic control tower cleared the pilot for a westbound departure; however, he elected to continue with a crosswind takeoff. The pilot applied left-wing-down aileron control inputs as he applied engine power. As the airplane accelerated, it began a drift to the left. The pilot reported that he was unable to stop the drift with the application of full right rudder, and that during this time he may have inadvertently relaxed the aileron control input. The pilot elected to abort the takeoff, and as he reduced engine power the right wing struck the water; the airplane incurred substantial damage to the wing spar and aileron during the impact sequence. The pilot reported no preimpact mechanical malfunction or failures.