This dataset contains the responses to two surveys that were used to assess the validity and reliability of a new 'risk score tool' that was designed to estimate the potential harm of a medication error by estimating the probability of a range of potential consequences. The risk score tool described five levels of potential harm. Judges estimated the likelihood of harm matching each level, from which a risk score (0-10) was calculated. Thirty judges (doctors, nurses and pharmacists) used this risk score and the existing Dean and Barber scale to estimate the potential harm of 50 medication errors, 15 with a known outcome (Survey 1). Two weeks later, the judges re-scored ten of the errors (Survey 2).