There are two basic uses for a Vendor Rating - Qualification and Differentiation.
Qualification - your vendor rating system should include a basement qualifying score under which vendors are excluded from procurement activities. Easy way to do it - scores based on 10 (averaged scores) or 100 (cumulative), with a cut off at 5/50.
Differentiation - Vendor Ratings are great tie breakers in Best Value procurements. Past performance is a valid evaluative criteria - vendor ratings should be taken into account when determining awardees resulting from competition.