This was done in a purely bottom-up empirical manner, as opposed to using 'standard' evaluation criteria known from argumentation literature (Johnson and Blair, 2006;Schiappa and Nordin, 2013).
we split all reasons into several reason units by simple preprocessing (splitting using Stanford CoreNLP , segmentation into Elementary Discourse Units by RST tools (Surdeanu et al., 2015)) and identified the referenced arguments (A1 or A2) by pattern matching and dependency parsing.