Optical profilometers, such as scanning white light interferometers and confocal microscopes, provide high resolution measurements and are widely utilized in many fields for measuring surface topography. The techniques are capable of high-speed surface measurements with nanometer-scale repeatability, and are used in industries such as data storage, automotive, MEMS, electronics, micro-optics, and bio-medical, to name a few. The instrument works best on flat, stepped structures, and slope-dependent systematic errors can be present in the measurement of steep sloped regions. These errors can be the same order of magnitude as features on the surface to be measured. Researchers have carried out many studies of these errors from first principle analyses; however the errors depend on proprietary details of the optical design and cannot be exactly calculated from first principles. The problem is further complicated by a lack of calibration artifacts to measure the errors directly. We propose a self-calibration technique, the random ball test, for calibrating slope-dependent errors of such instruments. A simulation study validates the approach and shows that the random ball test is effective in practical limits. We demonstrate the calibration on a 50x confocal microscope and a 50x white light interferometer with a specific chosen algorithm, find a surface slope-dependent bias that increases monotonically with the magnitude of the surface slope. The uncertainty of the calibration is smaller than the observed measurement bias and is dominated by residual random noise. Effects such as distortion, drift and ball radius uncertainty were investigated to understand their contribution to the calibration uncertainty.