How do I source critical COTS parts?

Sourcing the critical components takes more skill and strategy.  You would need to consider the availability and pricing of the component in the architecture/design phase. You would also need to consider whether the supplier will be easy to work with before you create a design with that specific component.  Why? Because once you design, say, a specific microprocessor in (based on brand and model, footprint, pinouts and other specifications), you are single-sourced to one specific manufacturer. You had better thoroughly understand the available development tools for that chip, the distributors, any minimum order quantity (MOQ) requirements, as well as the typical lead times before you bake in the chip. 

By the way, there is no general guideline for selecting the right supplier – all depends on the specific circumstances of the project.  A project that is going high-volume from the get-go would have cost as the top priority. You might choose a component with a lower price, even though the MOQ’s would be higher than what you want and the supplier has a reputation of being rigid and slow.  A project that requires significant flexibility in timing of production lots might point you towards a supplier with a component at a higher price, but with lower MOQ’s and shorter lead times.