I was using Ice before the season started and Fire in the season, and also wondered why everyone was taking physical so just for interest sake I tried it and it boomed all the other elements out of the water, the pull from Cave In and combination of Battle Rage Bloodshed and Boulder Toss(Physical) creates a good synergy.
Short answer is no. It doesn't have to be physical.
But if you want the most damage then it should be.
Most people are using:
Ancient Spear - Boulder Toss
Leap - Death from Above
Ground Stomp - Wrenching Smash
Threatening shout - Falter
War Cry - Veteran's Warning/Impunity
Earthquake - Cave in
The reason this is good is because everything is physical. And the base skill Avalanche is physical. So even though you don't have it on your bar it still gets a bonus. This way everything benefits from +physical dmg.
If you switch to cold then you have to sacrifice some things. If you run the same setup and just switch the rune on Earthquake and stack cold damage then your Bouldertoss and Avalanche don't benefit. If you also want to run with the cold Avalanche rune then you have to sacrifice ALOT of damage. Cause then you have to remove one of the skills that procs your weapon or belt, removing alot of "free" damage.
There is sort of better control with cold but not by alot. And not nearly enough to make it worth sacrifcing a big chunk of your damage. You get two pull effects and two more then decent stuns with practically no CD considering Fury dumping.
Hope this explains why most people are going physical. But if you find cold effective, fun or anything else positive then by all means play it like that. Another possibility is fire. Earthquake unruned does fire damage so you could swap Earthquake for Avalanche on your bar and get fire damage from both. But with this scenario you loose utility aswell. And Boulder toss damage takes a loss. But experiment away.
I use cold i find that best since it freezes and slows down mobs with iceblink gem, gives me a bit time before cooldown on leap kicks in so i can hop again, toss warcry and boulder and repeat xxxx times
And also cold is very good when u play 4 group fire is meh and so is physical.
I did a gr 65 lastnight with non ancient earth weapon and over 4 min left so i can easy top 70 with ancient weapon i think...
Not seen anyone else doing it but I'm playing Earth with Seismic Slam (Fire), just hit Para 600 and have already been doing 70 Rifts with a lot of lacking gear, no bonus str added via cube to any gear. I've not done my research and not even looked at any sites or played D3 at all since early S4 but I'm guessing the popularity of Raekor or Earth with Boulder toss shows it should be better, but this Seismic Slam build will go a long way above 70 for sure.