The problem asks us to determine the vertical distance h below the highest point at which a particle will leave a smooth vertical circle, given that it is slightly displaced from rest. We are given three options for h: (a) h = R, (b) h = R/3, and (c) h = R/2.

To solve this problem, we need to consider the forces acting on the particle at different points of the circle and analyze the energy changes.

1. Forces Acting on the Particle:
- At the highest point: The only force acting on the particle is the weight, which is directed downwards.
- At any other point: In addition to the weight, there is also a normal force acting on the particle, which is directed towards the center of the circle. This force provides the necessary centripetal force for circular motion.

2. Energy Analysis:
- At the highest point: The particle has maximum potential energy and zero kinetic energy.
- At any other point: The particle has both potential energy and kinetic energy. The total mechanical energy (E) of the particle is given by the sum of its potential energy (PE) and kinetic energy (KE): E = PE + KE.

3. Displacement from Rest:
- When the particle is slightly displaced from rest, it will have a non-zero initial velocity. This means that it will have kinetic energy at the highest point.

4. Conservation of Energy:
- According to the principle of conservation of energy, the total mechanical energy of the particle remains constant throughout its motion.
- Therefore, the sum of the particle's potential energy and kinetic energy at any point should be equal to the sum of its potential energy and kinetic energy at the highest point.

5. Analyzing the Options:
- (a) h = R: If the particle leaves the circle at a vertical distance equal to the radius (R), it means that it has lost all its potential energy and gained maximum kinetic energy. This would require a large initial displacement from rest, which is not mentioned in the problem.
- (b) h = R/3: This option suggests that the particle loses a significant amount of potential energy and gains some kinetic energy. This seems plausible as a slight displacement from rest would result in a smaller loss of potential energy.
- (c) h = R/2: This option suggests that the particle loses half of its potential energy and gains half of its kinetic energy. This seems plausible as a slight displacement from rest would result in a moderate loss of potential energy.

Based on the analysis, option (b) h = R/3 and option (c) h = R/2 seem more reasonable compared to option (a) h = R. However, without additional information about the initial displacement, it is difficult to determine the exact value of h.