And I am talking from a purely "ready to be ridden" point of view, just as the OP asked. The skeleton is the deciding factor on when a horse should be ridden, and a 2 year old, no matter how filled in they are in terms of flesh, is simply not matured enough in terms of skeletal development to ride.
The reason that Arabs, and other such breeds that are finer, seem to take so long to mature compared to QHs or drafts etc is a visual one. Stock breeds are far heavier from the start than an Arab, so it is to be expected that they seem thicker and more "built" at an earlier age. They still spend years more filling out, but yes, a two year old QH might indeed be chunkier than a full grown Arab that has done his filling out. That doesn't mean the 2 year old is ready to ride, just that it has the flesh to make it seem that way.
I still believe that 3 1/2 is the earliest they should be started, but I have yet to see a horse hurt when started- not ridden, necessarily- correctly and gently at 2. Emphasis on "started" and "correctly"... regardless, though, it's just too early for me.