Observed agreement 4 is 0.79 for expert annotators and 0.76 for nonexperts.
using observed agreement to measure reliability does not take into account agreement that is due to chance and hence is not a good measure of reliability.