The definition of critical density is that if the density of the universe is greater than this, the curvature will be positve, if it is smaller than this, it will be negative.
Before we start the derivation, we shall introduce the cosmological principle:
In Newtonian way, we need to slightly alter the definition of critical density, we will change it to: if the universe expand according to Hubble Law, then the critical density is the density of universe which it will halt its expansion when the age of universe becomes infinitly old. In the section of Fate of Universe, you will see the validity iof this assumption.
The escape velocity of a body with a mass of and a radius of is:
By Hubble Law, the recession velocity of a location with a distance to the observer is:
If these two velocity is equal to each other, then it means that the velocity caused by the expansion of the Universe cancels out the gravitational pull of matter from elsewhere. Meaning that the any stronger gravitational pull (larger density) will result in a collapse, any weaker gravitational pull (smaller density) will result in a forever expansion.
Knowing that the density of a sphere is:
Equate the two velocity and replace mass with density, we arrive on the definition of critical density:
Although this method is starightforward and easy, it sadly contradicts the cosmological principle, since now we are now assuming that the Universe is expanding like a sphere and a sphere is not homogeneous and isotropic, since the center of the sphere is a special point (it experience no gravity).
The accurate derivation should start with the Friedmann Equation derived from General Relativity.
If we divide both side by Hubble parameter squared, we get (since matter and radiation is basically the samething, we will combine these two terms for now for the sake of simplicity):
We should first recognize that the curvature of Universe cannot changed sign, thus meaning that on the right hand side, also cannot change sign, thus: