r = 0 or ∞
for a problem in which you seek an extreme value, you would ordinarily differentiate the variable of interest (area) in terms of some defining variable (h or r). setting that to zero will give you the location(s) of a function extreme.
here, that can mean you substitute your expression for h into the surface area equation, then differentiate area with respect to r.
solving for da/dr = 0 will give a value of r in the neighborhood of 2.632. however, you will find this is a minimum, not a maximum.
if you check the sign of the second derivative, you find it is positive everywhere. that is, the area function is concave upward, so has maxima at extreme values of radius (or height). it may not be so obvious that area increases as the radius gets smaller, but it should be quite clear that the area increases as the height goes to zero. the "cone" looks more and more like a flat disk, so to get any volume, it must have larger radius.