There is a profound dependence of cell behaviour on the stiffness of its microenvironment. To gain a better understanding of the regulation of cellular differentiation by mechanical cues, we investigated the influence of matrix stiffness (E = 1.46 kPa and E = 26.12 kPa) on differentiated osteogenic cell lineage of bone marrow stem cells (BM‐MSCs) and bone‐derived cells (BDCs) using flexible collagen‐coated polyacrylamide substrates. Differentiation potential was determined by measuring alkaline phosphatase activity, expression of osteoblast‐specific markers including alkaline phosphatase, osteocalcin, Runx2 and collagen type I, as well as assessment of mineralisation (Alizarin Red S staining). We found that osteogenic differentiation can be regulated by the rigidity of the substrate, which may depend on the commitment in multi‐ or uni‐potent targeting cells. Osteogenic differentiation of BM‐MSCs was enhanced on a stiff substrate compared to a soft one, whereas BDCs osteogenic differentiation did not vary depending on the substrate stiffness. The data help in understanding the role of the external mechanical determinants in stem cell differentiation, and can also be useful in translational approach in functional tissue engineering.