scripts/srl/srl.py

#! /usr/bin/python
# -*- coding: utf-8 -*-

"""	
	This module defines classes for semantic role labeling and predicate
	argument structure.
	
	Version 1.6												(20-Jun-2014)
	- getPropositions() and getPredicates() are added to SRL.
	- getForm() is added to DepArg and ConstArg.
	- getStr() and __str__ are added to to Arguments.
	- All [pv]flgCollapsAdjunct are fixed to [pv]flgCollapseAdjunct
	- Proposition.getStr() is updated.
	
	Version 1.5												(09-May-2014)
	- SRLComp.frameMatchPreds() is updated to optionally ignore frame match,
	  i.e. two predicates will match only if they are aligned regardless 
	  of their frame match.
	- BiSRLComp is removed (not developed).
	
	Version 1.4												(11-Apr-2014)
	- Predicate type is added to SRL as an attribute to allow to load either
	  verb or noun predicates or all predicate types. _removeNounPreds() and
	  _removeVerbPreds() are added to SRL for this purpose. 
	- getPredicate() is added to Proposition.
	- isVerbPred() is added to DepPred and ConstPred.
	- Constraint violation handling argument of SRL.loadFromCoNLL2009()
	  is removed and now handled via annotation option argument.
	- The default values for predicate alignment resolutions are changed 
	  from 1 to 0 to perform no action.
	
	Version 1.3												(07-Apr-2014)
	- SRLComp is debugged and tested.
	- DepSRLProject.waProject() is changed to load the target dependency 
	  tree with the projected SRL.
	- Bug is fixed in Predicate.compareFrameWith() which used lemma for sense
	  comparison level.
	- getArguments() is added to Proposition and SRL.
	
	Version 1.2												(20-Mar-2014)
	- In DepSRLProject, non and one-to-many predicates and arguments are
	  collected and reported. The statistics are computed by counting them
	  instead of keeping the counts. Many-to-one is still handled by counts.
	  _initReport() is added for this purpose.
	- position, getNode() and getForm() are added to DepPred.
	- getForm() is added to ConstPred.
	
	Version 1.1												(05-Mar-2014)
	- SRLProject and DepSRLProject are added.
	- addProposition() is added to SRL.
	- The initialization value of propositions in SRL.__init__() is changed
	  from none to [].
	- Constructor of Proposition is edited to have a default value for
	  plArguments in the method's argument.
	- addArguments() is added to Proposition.
	- "ignore" is added to OnConstraintViolation in SRL.checkConstraints().
	  
	
	Version 1.0												(19-Feb-2014)
	- getNode() is added to DepArg and ConstArg.
	- Constraints #5 and #6 are added to ConstArg._checkElevationConstraints().
	- ConstArg now optionally loads itself into the corresponding constituency
	  node during creation.
	- Conversion to constituency involves loading the annotation to the 
	  corresponding constituency tree by default. This however can be overridden
	  by setting the corresponding method argument. This early loading helps
	  applying global constraints to elevating argument node in the tree
	  when converted form dependency. Therefore it should be let done unless
	  it is required or other conversion methods that elevation is used.
	- getPOSTag() is added to DepPred and DepArg.
	- position is added to DepArg.
	- getLabel() is added to Predicate.
	
	Version 0.9												(17-Feb-2014)
	- Proposition.compareStructWith() is renamed to isStructEqualTo() and
	  predicate frame comparison is removed (i.e. proposition structure 
	  consists of only its argument roles).
	- SRLComp and BiSRLComp are added.
	- Predicate.compareFrameWith()
	- Argument.getLabel() is added.
	- getSynTag() is added to ConstPred, ConstArg.
	- pAnnotationOptions is added to SRL.loadFromCoNLL2009().
	- Dependency node is added to DepPred and DepArg. The constructors as
	  calling methods such as loadFromCoNLL2009() are edited accordingly.
	
	Version 0.8												(11-Feb-2014)
	- span is moved to Arguments and DepArg and ConstArg were changed
	  accordingly.
	- Argument.size() is added.
	- ConstArg.elevate() is edited to update span after elevation.
	- Bug in _checkElevationConstraints() is fixed to consider collapsed
	  terminal cases as pre-teriminal nodes.
	- A sanity check is added SRL.depToConst() to take care of dependency
	  SRL and constituency tree mismatch (e.g. non-parsed sentence for which
	  there however is a dependency SRL).
	
	Version 0.7												(07-Feb-2014)
	- Proposition.argCount() is changes from property to method and renamed
	  to getArgCount() to take adjunct collapse option as argument.
	- Bug is fixed in Argument.getAdjunctType() and Argument.isAdjunct().
	
	Version 0.6												(06-Feb-2014)
	- propCount(), getAvgArgCount() are added to SRL.
	- argCount(), compareStructWith() are added to Proposition
	- compareFrameWith() is added to Predicate  
	- isAdjunct(), compareRoleWith() are added to Argument.
	- __str__() and getStr() are added to SRL and Proposition
	
	Version 0.5												(29-Jan-2014)
	- Constraints #3, #4 are added in ConstArg._checkElevationConstraints().
	- getNode() is added to ConstPred.
	- Proposition is added as an attribute to Argument.
	- Argument.gerPredicate()
	
	Version 0.4												(27-Jan-2014)
	- DepArg and ConstArg are derived from Argument and dependency- and
	  constituency-based arguments as separated.
	- DepPred and ConstPred are derived from Predicate and dependency- and
	  constituency-based predicates as separated.
	- language is added to SRL and Argument.
	- getAdjunctType() is added to Argument.
	- Language-specific constraints are added to ConstArg.
	
	Version 0.3												(20-Jan-2014)
	- SRL.type is added specify the type of SRL formalism: dependency or 
	  consistuency 
	- isConstituency(), isConstituency(), convertToConst(), depToConst()
	  are added to SRL.
	
	Version 0.2												(27-Dec-2013)
	- SRL.loadFromCoNLL2009() is edited to allow for constraints check.
	- SRL.checkConstraints() is added.
	
	Version 0.1												(23-Dec-2013)
	- SRL, Proposition, Predicate, Argument are defined.
	
"""


import sys
import dloader, parse
from utils import util


class SRL:
	'''
	Class for semantic role labeling of a sentence.
	
	It is designed to label a sentence or loads the labels from various 
	formats e.g. CoNLL 2009.
	'''
	
	def __init__(self, pLanguage, pType = ''):
		'''
		Creates an SRL object
		'''
		
		# type of SRL formalism: (d)ependency or (c)onstituency
		self.type = pType
		
		self.language = pLanguage
		
		# list of propositions in the sentence
		self.propositions = []
		
	
	
	def isConstituency(self):
		'''
		Returns true if the SRL formalism is constituency
		'''
		
		return self.type.lower().startswith('c')
		
	
	
	def isDependency(self):
		'''
		Returns true if the SRL formalism is constituency
		'''
		
		return self.type.lower().startswith('d')
		
	
	
	def addProposition(self, pProposition):
		'''
		Adds a proposition 
		'''
		
		self.propositions.append(pProposition)
		
	
	
	def loadFromCoNLL2009(self, pCoNLL2009Sent, pdAnnotationOptions = {}):
		'''
		Loads semantic role labeling from a CoNLL 2009 sentence.
		
		pdAnnotationOptions include the following options:
		- on-srl-constraint-violation can be:
		  - exception: raise exception in case of any linguistic constraint 
		               violation
		  - fix      : fix linguistic constraint violations
		  - ignore   : ignore the problem but report
		- gold-or-predicted: can be gold or pred specifying which annotation
		  should be loaded
		- predicate-type: can be (v)erb or (n)oun or (a)ll
		'''
		
		# SRL CoNLL 2009 format is in dependency formalism
		self.type = 'dependency'
		
		vDepTree = pCoNLL2009Sent.getDepTree(pdAnnotationOptions = pdAnnotationOptions)
		
		# creating proposition placeholders
		self.propositions = [Proposition(None, None) for i in range(pCoNLL2009Sent.predicateCount)]
		
		vCurrPredNo = 0
		for i, vToken in enumerate(pCoNLL2009Sent.conllTokens, start = 1):
			# 1. creating arguments and adding to corresponding propositions
			for vArgLabel, vPredNo in vToken.args:
				self.propositions[vPredNo - 1].arguments.append(DepArg(vArgLabel, i, vDepTree.getNode(i), self.propositions[vPredNo - 1], self.language))
			
			# 2. creating predicates and adding to corresponding proposition 
			if vToken.fillPred:
				self.propositions[vCurrPredNo].predicate = DepPred(vToken.pred, i, vDepTree.getNode(i))
				vCurrPredNo += 1
		
		# applying predicate type
		if "predicate-type" in pdAnnotationOptions:
			if pdAnnotationOptions["predicate-type"].lower().startswith('v'):
				self._removeNounPreds()
			elif pdAnnotationOptions["predicate-type"].lower().startswith('n'):
				self._removeVerbPreds()
			elif not pdAnnotationOptions["predicate-type"].lower().startswith('a'):
				raise Exception("Unknown predicate type in annotation options: %s" % pdAnnotationOptions["predicate-type"])
		
		# checking for linguistic constraints
		if "on-srl-constraint-violation" in pdAnnotationOptions:
			vOnConstraintViolation = pdAnnotationOptions["on-srl-constraint-violation"]
		else:
			vOnConstraintViolation = "exception"
		
		return self.checkConstraints(vOnConstraintViolation)
		
	
	
	def _removeNounPreds(self):
		'''
		Removes propositions of noun predicates, i.e. keeps only propositions
		of verb predicates
		'''
		
		self.propositions = [p for p in self.propositions if p.getPredicate().isVerbPred()]
		
	
	
	def _removeVerbPreds(self):
		'''
		Removes propositions of verb predicates, i.e. keeps only propositions
		of noun predicates
		'''
		
		self.propositions = [p for p in self.propositions if not p.getPredicate().isVerbPred()]
		
	
	
	def convertToConst(self, pConstTree, pConversionMethod, pflgLoadIntoTree = True):
		'''
		Creates a new SRL object in constituency formalism by converting 
		from the current formalism given the constituency tree
		
		pflgLoadIntoTree determines whether the annotation must be loaded
		into the corresponding constituency tree after conversion or not.
		'''
		
		if self.isDependency():
			return self.depToConst(pConstTree, pConversionMethod, pflgLoadIntoTree)
		else:
			raise Exception("Conversion from %s is not implemented yet", self.type)
		
	
	
	def depToConst(self, pConstTree, pConversionMethod, pflgLoadIntoTree):
		'''
		Creates a new SRL object in constituency formalism by converting 
		from current dependency formalism given the constituency tree
		
		pflgLoadIntoTree determines whether the annotation must be loaded
		into the corresponding constituency tree after conversion or not.
		'''
		
		vConstSRL = SRL(pLanguage = self.language)
		
		vConstSRL.type = "constituency"
		vConstSRL.propositions = []
		
		# sanity check; useful for e.g. non-parsed trees having dep SRL
		vSentLength = pConstTree.getSentLength()
		if self.propCount > vSentLength or self.propCount == vSentLength == 1:
			sys.stderr.write('Number of propositions (%s) is bigger than sentence length (%s) according to constituency tree! Skipped.\n' % (self.propCount, vSentLength))
			return vConstSRL
		
		# NOTE: we don't use deepcopy()
		for vProp in self.propositions:
			vConstSRL.propositions.append(vProp.depToConst(pConstTree, pConversionMethod, pflgLoadIntoTree))
		
		return vConstSRL
		
	
	
	def checkConstraints(self, pOnConstraintViolation):
		'''
		Checks for a set of linguistic constraints
		'''
		
		vlErrStr = []              # violation errors caught
		vlArgToRemove = []         # violent arguments to remove 
		
		# 1. predicate-argument overlap
		
		for vProp in self.propositions:
			for vArg in vProp.arguments:
				if vArg.span[0] <= vProp.predicate.position <= vArg.span[1]:
					vlErrStr.append("Predicate-argument overlap: %s and (%s, %s)" % (vProp.predicate.position, vArg.span[0], vArg.span[1]))
					if pOnConstraintViolation.lower() == "exception":
						raise Exception(vlErrStr[-1])
					elif pOnConstraintViolation.lower() == "ignore":
						vlErrStr[-1] = "(Ignocopyred) " + vlErrStr[-1]
						continue
					elif pOnConstraintViolation.lower() == "fix":
						## We remove the argument that overlaps with its own
						## predicate.
						vlArgToRemove.append(vArg)
						vlErrStr[-1] = "(Fixed) " + vlErrStr[-1]
		
			# removing violent arguments
			for vArg in vlArgToRemove:
				vProp.arguments.remove(vArg)
			vlArgToRemove = []
		
		# TODO: add other constraints
		
		return vlErrStr
		 
	
	
	@property
	def propCount(self):
		'''
		Returns the number of propositions
		'''
		
		return len(self.propositions)
		
	
	
	@property
	def globalArgCount(self):
		'''
		Returns the number of arguments across all predicates
		'''
		
		vArgCount = 0
		for vProp in self.propositions:
			vArgCount += vProp.getArgCount()
		
		return vArgCount
		
	
	
	def getAvgArgCount(self):
		'''
		Returns the average number of arguments per proposition
		'''
		
		if self.propCount == 0:
			return 0
		else:
			return sum([p.getArgCount() for p in self.propositions]) * 1.0 / self.propCount
		
	
	
	def __str__(self):
		'''
		String representation of the class instance
		'''
		
		return self.getStr()
		
	
	
	def getStr(self):
		'''
		Returns string representation of the class instance
		'''
		
		return "\n".join([p.getStr() for p in self.propositions])
		
	
	
	def getArguments(self):
		'''
		Returns all arguments of the SRL
		'''
		
		vlArgs = []
		for vProp in self.propositions:
			vlArgs += vProp.getArguments()
		
		return vlArgs
		
	
	
	def getPropositions(self):
		'''
		Returns all the propositions
		'''
		
		return self.propositions
		
	
	
	def getPredicates(self):
		'''
		Returns predicates of all propositions
		'''
		 
		return [p.getPredicate() for p in self.getPropositions()]
		 
	
	

class Proposition:
	'''
	Class for proposition
	
	A proposition consists of a predicate and its arguments.
	'''
	
	def __init__(self, pPredicate, plArguments = None):
		'''
		Creates a proposition object
		'''
		
		self.predicate = pPredicate
		
		if plArguments == None:
			self.arguments = []
		else:
			self.arguments = plArguments
		
	
	
	def getPredicate(self):
		'''
		Returns the predicate
		'''
		
		return self.predicate
		
	
	
	def getArguments(self):
		'''
		Returns arguments
		'''
		
		return self.arguments
		
	
	
	def addArguments(self, plArguments):
		'''
		Adds given arguments to proposition
		'''
		
		self.arguments += plArguments
		
	
	
	def depToConst(self, pConstTree, pConversionMethod, pflgLoadIntoTree):
		'''
		Creates a new proposition object by converting the predicate and
		arguments of this proposition to constituency formalism given 
		the constituency tree.
		
		In dependency formalism, an argument role is assigned to a dependency
		node which corresponds to a token position. In constituency formalism,
		it is assigned to a phrase which corresponds to contituency node.
		Since a dependency-based argument cannot deterministically be mapped
		to a constituency node, simplifying assumptions are made:
		- preterminal: assign argument role to the preterminal at the 
		               argument position
		- elevate: assign argument role to the highest possible parent 
		           node covering the argument position. The possibility
		           is determined by checking against a set of constraints.
		
		pflgLoadIntoTree determines whether the annotation must be loaded
		into the corresponding constituency tree after conversion or not.
		'''
		
		# creating and converting predicate
		vNewPred = self.predicate.depToConst(pConstTree, pflgLoadIntoTree)
		
		# creating proposition
		vNewProp = Proposition(pPredicate = vNewPred, plArguments = [])
		
		# creating, converting and adding arguments
		for vArg in self.arguments:
			vNewProp.arguments.append(vArg.depToConst(pConstTree, vNewProp, pflgLoadIntoTree))
		
		## applying conversion method
		## NOTE: In order to elevation to be fully done, the SRL must be
		## loaded into the tree while converting (pflgLoadIntoTree). 
		## Otherwise, some elevation constraints will not work (e.g. #5)
		if pConversionMethod.lower() == "elevate":
			for vArg in vNewProp.arguments:
				vArg.elevate()
		
		return vNewProp
		
	
	
	def getArgCount(self, pflgCollapseAdjunct = False):
		'''
		Returns the number of arguments
		
		If pflgCollapseAdjunct is set to true, the adjunct roles are collapsed
		into a single role.
		'''
		
		if pflgCollapseAdjunct and len([a for a in self.arguments if a.isAdjunct()]) > 0:
			return len([a for a in self.arguments if not a.isAdjunct()]) + 1
		else:
			return len(self.arguments)
		
	
	
	def __str__(self):
		'''
		String representation of the class instance
		'''
		
		return self.getStr()
		
	
	
	def getStr(self):
		'''
		Returns string representation of the class instance
		'''
		
		return "%s (%s): %s" % (self.predicate.getLabel(pLevel = "frameset"), self.predicate.getForm(), ' '.join(["%s (%s)" % (a.label, a.getForm()) for a in self.arguments]))
		
	
	
	def isStructEqualTo(self, pAnotherProp, pflgCollapseAdjunct):
		'''
		VERIFY AND MOVE TO SRLCOMP
		
		Returns true is the structure of this proposition with that of another
		proposition is the same
		
		Structure of a proposition consists of its arguments role labels.
		(Previously consisted of its predicate's label as well.)
		
		pflgCollapseAdjunct specifies the comparison method at argument 
		level. See Argument class methods for details.
		'''
		
		if self.getArgCount() != pAnotherProp.getArgCount():
			return False
		else:
			vlArgs2 = pAnotherProp.arguments[ : ]
			
			for vArg1 in self.arguments:
				vArg2Itr = 0
				vMatchFound = False
				for vArg2 in vlArgs2:
					if vArg1.compareRoleWith(vArg2, pflgCollapseAdjunct) == True:
						del vlArgs2[vArg2Itr]
						vMatchFound = True
						break
					else:
						vArg2Itr += 1
						continue
				
				if vMatchFound == False:
					return False
		
		return True
		
	
	


class Predicate:
	'''
	Class for predicate
	'''
	
	def __init__(self, pPredLabel):
		'''
		Creates a predicate object
		'''
		
		vSensePos = pPredLabel.rfind('.')
		self.frameset = pPredLabel
		self.lemma = pPredLabel[ : vSensePos]
		self.sense = pPredLabel[vSensePos + 1 : ]
		
	
	
	def compareFrameWith(self, pAnotherPred, pLevel = "frameset"):
		'''
		MOVE TO SRLComp
		
		Returns true if this predicate has the same frameset or lemma as
		a given predicate
		
		pLevel specifies whether the comparison should be done at frameset
		lemma, or sense level. Sense level may not make sense but it is 
		for compatibility with CoNLL2009 scorer.
		'''
		
		if pLevel == "frameset":
			if self.frameset == pAnotherPred.frameset:
				return True
			else:
				return False
		elif pLevel == "lemma":
			if self.lemma == pAnotherPred.lemma:
				return True
			else:
				return False
		elif pLevel == "sense":
			if self.sense == pAnotherPred.sense:
				return True
			else:
				return False
		
	
	
	def getLabel(self, pLevel = "frameset"):
		'''
		Returns predicate label at given level
		'''
		
		vLevel = pLevel.lower()
		
		if vLevel == "frameset":
			return self.frameset
		elif vLevel == "lemma":
			return self.lemma
		elif vLevel == "sense":
			return self.sense
		
	
	


class DepPred(Predicate):
	'''
	Class for dependency-based predicate
	'''
	
	def __init__(self, pPredLabel, pTokenPos, pDepNode):
		'''
		Creates a dependency-based predicate object
		
		In dependency-based arguments, the predicate is identified by the 
		position of the token to which it is assigned.
		'''
		
		Predicate.__init__(self, pPredLabel)
		
		self.position = pTokenPos
		self.node = pDepNode
		
	
	
	def depToConst(self, pConstTree, pflgLoadIntoTree):
		'''
		Creates a new constituency predicate object by converting from
		current dependency formalism given the constituency tree
		
		It assigns the predicate to the preterminal node that spans predicate
		token.
		
		pflgLoadIntoTree determines whether the annotation must be loaded
		into the corresponding constituency tree node after conversion or
		not.
		'''
		
		return ConstPred(self.frameset, pConstTree.getPreTerminalNodeAt(self.position), pflgLoadIntoTree)
		
	
	
	def getDepRel(self):
		'''
		Returns dependency relation of argument node
		'''
		
		return self.node.getDepRel()
		
	
	
	@property
	def position(self):
		'''
		Returns the token position of the predicate in the sentence
		
		It assumes that the predicate is assigned to a preterminal that 
		spans only a token.
		'''
		
		return self.node.getTokenSpan()[0]
		
	
	
	def getNode(self):
		'''
		Returns the dependency node of the predicate
		'''
		
		return self.node
		
	
	
	def getPOSTag(self):
		'''
		Returns the POS tag of the predicate
		'''
		
		return self.node.getPOSTag()
		
	
	
	def getForm(self):
		'''
		Returns the surface form of the predicate
		'''
		
		return self.node.getForm()
		
	
	
	def isVerbPred(self):
		'''
		Returns true of the predicate is verb
		
		A predicate is recognized as a verb predicate if its POS tag starts
		with 'v'. 
		'''
		
		return self.getPOSTag().lower().startswith('v')
		
	
	


class ConstPred(Predicate):
	'''
	Class for constituency-based predicate
	'''
	
	def __init__(self, pPredLabel, pConstNode, pflgLoadIntoTree):
		'''
		Creates a constituency-based predicate object
		
		In constituency-based arguments, the predicate is identified by the 
		constituency node to which it is assigned.
		'''
		
		Predicate.__init__(self, pPredLabel)
		
		self.setNode(pConstNode, pflgLoadIntoTree)
		
	
	
	def setNode(self, pNode, pflgLoadIntoTree):
		'''
		Sets the constituency node of argument
		
		pflgLoadIntoTree determines whether the annotation must be loaded
		into the corresponding constituency tree node or not.
		
		NOTE: If the node is not empty, it will be overridden. In that case
		the predicate-role annotation in the current node will remain
		intact and cause inconsistency. Therefore, for such cases, use
		changeNode() instead.
		'''
		
		self.node = pNode
		
		if pflgLoadIntoTree:
			self.node.addPredRole((None, self.frameset))
		
	
	
	@property
	def position(self):
		'''
		Returns the token position of the predicate in the sentence
		
		It assumes that the predicate is assigned to a preterminal that 
		spans only a token.
		'''
		
		return self.node.getTokenSpan()[0]
		
	
	
	def getNode(self):
		'''
		Returns the constituency node of the predicate
		'''
		
		return self.node
		
	
	
	def getSynTag(self):
		'''
		Returns syntactic tag of predicate node
		'''
		
		return self.node.getSynTag()
		
	
	
	def getForm(self):
		'''
		Returns the surface form of the predicate
		'''
		
		return self.node.surface
		
	
	
	def isVerbPred(self):
		'''
		Returns true of the predicate is verb
		
		A predicate is recognized as a verb predicate if its POS tag starts
		with 'v'. 
		'''
		
		return self.getSynTag().lower().startswith('v')
		
	
	


class Argument:
	'''
	Class for argument
	'''
	
	def __init__(self, pArgLabel, pTokenSpan, pProposition, pLanguage):
		'''
		Creates an argument object
		'''
		
		self.label = pArgLabel
		self.span = pTokenSpan
		self.language = pLanguage
		self.proposition = pProposition
		
	
	
	def getAdjunctType(self):
		'''
		Returns the type of adjunct
		
		It assumes that adjuncts are labeled in *AM-Type format e.g. AM-TMP 
		or R-AM-TMP.
		'''
		
		if self.isAdjunct():
			vPos = self.label.find('AM-')
			if vPos == -1:
				raise Exception("%s is not adjunct!" % self.label)
			else:
				return self.label[vPos + 3 : ]
		else:
			raise Exception("%s is not adjunct!" % self.label)
		
	
	
	def isAdjunct(self):
		'''
		Returns true if the argument is an adjunct
		
		It assumes that adjuncts are labeled in *AM-Type format.
		'''
		
		return self.label.find('AM-') != -1
		
	
	
	def getPredicate(self):
		'''
		Returns the predicate of the argument
		'''
		
		return self.proposition.predicate
		
	
	
	def compareRoleWith(self, pAnotherArg, pflgCollapseAdjunct = False):
		'''
		MOVE TO SRLComp
		
		Returns true if this argument has the same role label as a given
		argument
		
		If pflgCollapseAdjunct is set to true, the adjunct argument labels 
		will be collapsed to a single label.
		'''
		
		if pflgCollapseAdjunct:
			if (self.label == pAnotherArg.label) or (self.isAdjunct() and pAnotherArg.isAdjunct()):
				return True
			else:
				return False
		else:
			if self.label == pAnotherArg.label:
				return True
			else:
				return False
		
	
	
	@property
	def size(self):
		'''
		Returns the size of token span of the argument in the sentence
		'''
		
		return self.span[1] - self.span[0] + 1
		
	
	
	def getLabel(self, pflgCollapseAdjunct = False):
		'''
		Returns argument label
		'''
		
		if pflgCollapseAdjunct and self.isAdjunct():
			# R-AM and C-AM taken into account
			vPos = self.label.rfind('-')
			return self.label[ : vPos]
		else:
			return self.label
		
	
	
	def __str__(self):
		'''
		String representation of the class instance
		'''
		
		return self.getStr()
		
	
	
	def getStr(self):
		'''
		Returns string representation of the class instance
		'''
		
		return "%s (%s)" % (self.getLabel(), self.getForm())
		
	
	


class DepArg(Argument):
	'''
	Class for dependency-based argument
	'''
	
	def __init__(self, pArgLabel, pTokenPos, pDepNode, pProposition, pLanguage):
		'''
		Creates a dependnecy-based argument
		
		In dependency-based arguments, the argument is identified by the 
		position of the token to which it is assigned.
		'''
		
		Argument.__init__(self, pArgLabel, (pTokenPos, pTokenPos), pProposition, pLanguage)
		
		self.node = pDepNode
		
	
	
	def depToConst(self, pConstTree, pProposition, pflgLoadIntoTree):
		'''
		Creates a new constituency argument object by converting from
		current dependency formalism given the constituency tree.
		
		pflgLoadIntoTree determines whether the annotation must be loaded
		into the corresponding constituency tree node after conversion or
		not.
		'''
		
		return ConstArg(self.label, pConstTree.getPreTerminalNodeAt(self.span[0]), pProposition, self.language, pflgLoadIntoTree)
		
	
	
	def getNode(self):
		'''
		Returns the dependency node of argument
		'''
		
		return self.node
		
	
	
	def getDepRel(self):
		'''
		Returns dependency relation of argument node
		'''
		
		return self.node.getDepRel()
		
	
	
	def getPOSTag(self):
		'''
		Returns the POS tag of the argument
		'''
		
		return self.node.getPOSTag()
		
	
	
	@property
	def position(self):
		'''
		Returns the position of argument roll filler in sentence
		'''
		
		return self.span[0]
		
	
	
	def getForm(self):
		'''
		Returns the surface form of the predicate
		'''
		
		return self.getNode().getForm()
		
	
	


class ConstArg(Argument):
	'''
	Class for constituency-based argument
	'''
	
	def __init__(self, pArgLabel, pConstNode, pProposition, pLanguage, pflgLoadIntoTree):
		'''
		Creates a constituency-based argument
		
		In constituency-based arguments, the argument is identified by 
		the constituent node to which it is assigned.
		'''
		
		Argument.__init__(self, pArgLabel, pConstNode.getTokenSpan(), pProposition, pLanguage)
		
		self.setNode(pConstNode, pflgLoadIntoTree)
		
	
	
	def setNode(self, pNode, pflgLoadIntoTree):
		'''
		Sets the constituency node of argument
		
		pflgLoadIntoTree determines whether the annotation must be loaded
		into the corresponding constituency tree node or not.
		
		NOTE: If the node is not empty, it will be overridden. In that case
		the predicate-role annotation in the current node will remain
		intact and will cause inconsistency. Therefore, for such cases, use
		changeNode() instead.
		'''
		
		self.node = pNode
		
		if pflgLoadIntoTree:
			self.node.addPredRole((self.getPredicate().getNode(), self.getLabel()))
		
	
	
	def getNode(self):
		'''
		Returns the constituency node of argument
		'''
		
		return self.node
		
	
	
	def changeNode(self, pNewNode):
		'''
		Changes the constituency node of argument to a new node
		
		It checks the current constituency node and if it contains the 
		predicate-role annotation of this argument, it will be moved into
		new node. Otherwise, it wont change the predicate-role annotation
		of the nodes.
		'''
		
		vPredRole = (self.getPredicate().getNode(), self.getLabel())
		if self.getNode().hasPredRole(vPredRole):
			self.getNode().removePredRole(vPredRole)
			self.setNode(pNewNode, pflgLoadIntoTree = True)
		else:
			self.setNode(pNewNode, pflgLoadIntoTree = False)
		
	
	
	def getSynTag(self):
		'''
		Returns syntactic tag of argument node
		'''
		
		return self.node.getSynTag()
		
	
	
	def elevate(self):
		'''
		Elevates the argument node to the highest possible parent node. 
		
		The possibility is determined by checking against a set of constraints.
		'''
		
		if self.node.isRoot():
			return 
		else:
			vParent = self.node.getParent()
		
		## The parent node spanning (exactly) the same span is a valid level
		## to elvate the node to.
		if vParent.getTokenSpan() == self.span:
			self.changeNode(vParent)
			self.elevate()
		elif self._checkElevationConstraints():
			self.changeNode(vParent)
			self.span = self.node.getTokenSpan()
			self.elevate()
		
	
	
	def _checkElevationConstraints(self):
		'''
		Returns true if elevating the argument node to its parent does 
		not violate a set of constraints
		'''
		
		# NOTE: numbering constraints are chronological
		
		# common
		
		# 3. Root node cannot take a role.
		if self.getNode().getParent().isRoot():
			return False
		
		# 2. adjunct type which are normally assigned to single tokens
		if self.isAdjunct() and self.getAdjunctType() in ["NEG", "MOD", "DIS"]:
			return False
		
		# 6. Argument of a node with these tags cannot be elevated
		if self.getSynTag() in ["POS"]:
			return False
		
		# 4. argument node cannot dominate its predicate node
		if self.getNode().getParent().ifDominates(self.getPredicate().getNode()):
			return False
		
		## 5. argument node cannot dominate another argument node of its 
		## predicate
		if len([vSibling for vSibling in self.getNode().getSibling() if vSibling.isArgumentOf(self.getPredicate().getNode())]) > 0:
			return False
		
		# 1. parnet of a pre-terminal can usually take the role
		if self.getNode().isPreTerminal() or self.getNode().isCollapsedTerminal():
			return True
		
		## TODO: do something for argument overlap for a proposition
		## TODO: add common constraints
		
		# language-specific
		
		if self.language.lower().startswith("en"):
			return self._checkElevationConstraintsEN()
		elif self.language.lower().startswith("fr"):
			return self._checkElevationConstraintsFR()
		
	
	
	def _checkElevationConstraintsEN(self):
		'''
		Checks English-specific argument elevation constraints
		'''
		
		# TODO: add constriants
		
		return False
		
	
	
	def _checkElevationConstraintsFR(self):
		'''
		Checks French-specific argument elevation constraints
		'''
		
		# TODO: add constriants
		
		return False
		
	
	
	def getForm(self):
		'''
		Returns the surface form of the predicate
		'''
		
		return self.getNode().surface
		
	
	


## Comparisons #########################################################


class SRLComp:
	'''
	Class for comparing two SRLs of a sentence
	'''
	
	def __init__(self, pPredLabelMatchLevel = "frameset", pflgCollapseAdjunct = False):
		'''
		Constructor
		
		Predicate label matching level is one of the following:
		- frameset       : the entire frameset labels match (e.g. go.01 vs. go.01) 
		- lemma          : only the lemmas match (e.g. go.01 vs. go.02)
		- sense          : only predicate sense (e.g. go.01 vs. eat.01); 
		                   This may not make sense but it is for compatibility
		                   with CoNLL2009 scorer.
		- none or ignore : no frame match check, i.e. two predicates will
		                   match only if they are aligned regardless of 
		                   their frame match.
		
		Adjunct labels can be optionally collapsed into one label to ignore
		adjunct type.            
		
		'''
		
		self.predLabelMatchLevel = pPredLabelMatchLevel.lower()
		self.collapseAdjunct = pflgCollapseAdjunct
		
		# list of tuples of propositions whose predicates are aligned
		self.alignedProps = []
		# list of tuples of arguments whose fillers are aligned
		self.alignedArgs = []
		
	
	
	@property
	def alignedPreds(self):
		'''
		Returns list of aligned predicates (regardless of label match)
		'''
		
		return [(p1.predicate, p2.predicate) for p1, p2 in self.alignedProps]
		
	
	
	@property
	def frameMatchPreds(self):
		'''
		Returns list of aligned predicates whose frame label match at the
		preset level
		
		See constructor documentation for predicate label matching level.
		'''
		
		if self.predLabelMatchLevel.lower() in ["none", "ignore"]:
			## ignoring frame match, i.e. two predicates will match only
			## if they are aligned regardless of their frame match.
			return self.alignedPreds
		else:
			return [(p1.predicate, p2.predicate) for p1, p2 in self.alignedProps if p1.predicate.compareFrameWith(p2.predicate, self.predLabelMatchLevel)]
		
	
	
	@property
	def alignedPredCount(self):
		'''
		Returns the number of matching predicates
		'''
		
		return len(self.alignedProps)
		
	
	
	@property
	def frameMatchPredCount(self):
		'''
		Returns the number of aligned predicates whose frame label match
		'''
		
		return len(self.frameMatchPreds)
		
	
	
	@property
	def roleMatchArgs(self):
		'''
		Returns list of aligned arguments whose role label match considering
		the setting of adjuncts match (original or collapsed).
		'''
		
		return [(a1, a2) for a1, a2 in self.alignedArgs if a1.compareRoleWith(a2, pflgCollapseAdjunct = self.collapseAdjunct)]
		
	
	
	@property
	def alignedArgCount(self):
		'''
		Returns the number of matching arguments
		'''
		
		return len(self.alignedArgs)
		
	
	
	@property
	def roleMatchArgCount(self):
		'''
		Returns the number of aligned arguments whose role label match 
		considering the setting of adjuncts match (original or collapsed).
		'''
		
		return len(self.roleMatchArgs)
		
	
	
	def compare(self, pSRL1, pSRL2):
		'''
		NOT TESTED
		Compare two SRLs
		
		Results are set into the corresponding attributes.
		'''
		
		self.alignedProps = self._extractAlignedProps(pSRL1, pSRL2)
		for vProp1, vProp2 in self.alignedProps:
			self.alignedArgs += self._extractAlignedArgs(vProp1, vProp2)
		
	
	
	def _extractAlignedProps(self, pSRL1, pSRL2):
		'''
		Extracts from two given SRLs pairs of propositions whose predicates
		are aligned
		'''
		
		## coping list of propositions of second SRL into a new list to 
		## be able to delete form it later
		vlProps2 = pSRL2.propositions[ : ]
		
		vlAligned = []
		for vProp1 in pSRL1.propositions:
			vProp2Itr = 0
			for vProp2 in vlProps2:
				if self._arePredsAligend(vProp1.predicate, vProp2.predicate):
					del vlProps2[vProp2Itr]
					vlAligned.append((vProp1, vProp2))
					break
				else:
					vProp2Itr += 1
		
		return vlAligned
		
	
	
	def _arePredsAligend(self, pPred1, pPred2):
		'''
		NOT TESTED
		Returns true if the given predicates are aligned, i.e. their positions
		in the sentence are the same
		'''
		
		return pPred1.position == pPred2.position
		
	
	
	def _extractAlignedArgs(self, pProp1, pProp2):
		'''
		NOT TESTED
		Extracts from given two propositions, pairs of aligned arguments
		'''
		
		## coping list of arguments of second proposition into a new list 
		## to be able to delete form it later
		vlArgs2 = pProp2.arguments[ : ]
		
		vlMatchPairs = []
		for vArg1 in pProp1.arguments:
			vArg2Itr = 0
			for vArg2 in vlArgs2:
				if self._areArgsAligned(vArg1, vArg2):
					del vlArgs2[vArg2Itr]
					vlMatchPairs.append((vArg1, vArg2))
					break
				else:
					vArg2Itr += 1
		
		return vlMatchPairs
		
	
	
	def _areArgsAligned(self, pArg1, pArg2):
		'''
		NOT TESTED
		Returns true if the given arguments are aligned, i.e. their boundaries
		match.
		'''
		
		return pArg1.span == pArg2.span
		
	
	


## Projection ##########################################################


class SRLProject:
	'''
	Base Class for projecting SRL from one the source side of the translation
	to the target side
	'''
	
	def __init__(self, pSrcSurface, pSrcSRL, pSrcParse, pSrcLang):
		'''
		Constructor
		'''
		
		self.srcSurface = pSrcSurface
		self.srcSRL = pSrcSRL
		self.srcParse = pSrcParse
		self.pSrcLang = pSrcLang
		
		# initializing projection statistics attributes
		self._initStat()
		
	
	
	def _initStat(self):
		'''
		Initializes projection statistics
		
		Only many-to-ones are counted in this way. The others are handled
		by _initReport().
		'''
		
		#self._cntNonAlignedPred = 0         # number of source predicates with no alignment (not necessarily word-alignment) in the target
		#self._cntNonProjectedPred = 0       # number of source predicated not finally projected
		#self._cntOneToManyAlignedPred = 0   # number of source predicates aligned (not necessarily word-aligned) with non-consecutive target role fillers
		#self._cntOneToManyProjectedPred = 0 # number of source predicates projected to non-consecutive target role fillers (not happening in practice)
		self._cntManyToOneAlignedPred = 0   # number of times different predicates aligned (not necessarily word-aligned) to same filler
		self._cntManyToOneProjectedPred = 0 # number of times different predicates projected to same filler
		
		#self._cntNonAlignedArg = 0          # number of source arguments with no alignment (not necessarily word-alignment) in the target
		#self._cntNonProjectedArg = 0        # number of source arguments not finally projected including those due to non-projected predicates
		#self._cntOneToManyAlignedArg = 0    # number of source arguments aligned (not necessarily word-aligned) with non-consecutive target role fillers
		#self._cntOneToManyProjectedArg = 0  # number of source arguments projected to non-consecutive target role fillers
		self._cntManyToOneAlignedArg = 0    # number of times different arguments of same proposition aligned (not necessarily word-aligned) to same filler
		self._cntManyToOneProjectedArg = 0  # number of times different arguments of same proposition projected to same filler
		
	
	
	def _initReport(self):
		'''
		Initializes projection report
		
		Many-to-ones are not supported yet and handled by _initStat() only 
		in terms of counts.
		'''
		
		self.nonAlignedPred = []         # source predicates with no alignment (not necessarily word-alignment) in the target
		self.nonProjectedPred = []       # source predicated not finally projected
		self.oneToManyAlignedPred = []   # source predicates aligned (not necessarily word-aligned) with non-consecutive target role fillers
		self.oneToManyProjectedPred = [] # source predicates projected to non-consecutive target role fillers (not happening in practice)
		#self.manyToOneAlignedPred = []   # sets of different predicates aligned (not necessarily word-aligned) to same filler
		#self.manyToOneProjectedPred = [] # sets of different predicates projected to same filler
		
		self.nonAlignedArg = []          # source arguments with no alignment (not necessarily word-alignment) in the target
		self.nonProjectedArg = []        # source arguments not finally projected including those due to non-projected predicates
		self.oneToManyAlignedArg = []    # source arguments aligned (not necessarily word-aligned) with non-consecutive target role fillers
		self.oneToManyProjectedArg = []  # source arguments projected to non-consecutive target role fillers
		#self.manyToOneAlignedArg = []    # sets of different arguments of same proposition aligned (not necessarily word-aligned) to same filler
		#self.manyToOneProjectedArg = []  # sets of different arguments of same proposition projected to same filler
		
	
	
	def getStat(self):
		'''
		Returns the projection statistics in a dictionary
		
		4 types of statistics are returned:
		- Counts
		- Aggregation of counts based on various factors
		- Percentages
		- Aggregation of percentages based on various factors
		'''
		
		vdCount = {"Non-aligned predicate count":            len(self.nonAlignedPred), \
		           "Non-projected predicate count":          len(self.nonProjectedPred), \
		           "One-to-many aligned predicate count":    len(self.oneToManyAlignedPred), \
		           "One-to-many projected predicate count":  len(self.oneToManyProjectedPred), \
		           "Many-to-one aligned predicate count":    self._cntManyToOneAlignedPred, \
		           "Many-to-one projected predicate count":  self._cntManyToOneProjectedPred, \
		           "Non-aligned argument count":             len(self.nonAlignedArg), \
		           "Non-projected argument count":           len(self.nonProjectedArg), \
		           "One-to-many aligned argument count":     len(self.oneToManyAlignedArg), \
		           "One-to-many projected argument count":   len(self.oneToManyProjectedArg), \
		           "Many-to-one aligned argument count":     self._cntManyToOneAlignedArg, \
		           "Many-to-one projected argument count":   self._cntManyToOneProjectedArg}
		
		vdAggCount = {"Total non-aligned count":               len(self.nonAlignedPred) + len(self.nonAlignedArg), \
		              "Total non-projected count":             len(self.nonProjectedPred) + len(self.nonProjectedArg), \
		              "Total one-to-many alignment count":     len(self.oneToManyAlignedPred) + len(self.oneToManyAlignedArg), \
		              "Total one-to-many projection count":    len(self.oneToManyProjectedPred) + len(self.oneToManyProjectedArg), \
		              "Total many-to-one alignment count":     self._cntManyToOneAlignedPred + self._cntManyToOneAlignedArg, \
		              "Total many-to-one projection count":    self._cntManyToOneProjectedPred + self._cntManyToOneProjectedArg, \
		              "Total predicate alignment case count":  len(self.nonAlignedPred) + len(self.oneToManyAlignedPred) + self._cntManyToOneAlignedPred, \
		              "Total predicate projection case count": len(self.nonProjectedPred) + len(self.oneToManyProjectedPred) + self._cntManyToOneProjectedPred, \
		              "Total argument alignment case count":   len(self.nonAlignedArg) + len(self.oneToManyAlignedArg) + self._cntManyToOneAlignedArg, \
		              "Total argument projection case count":  len(self.nonProjectedArg) + len(self.oneToManyProjectedArg) + self._cntManyToOneProjectedArg, \
		              "Total alignment case count":            len(self.nonAlignedPred) + len(self.oneToManyAlignedPred) + self._cntManyToOneAlignedPred + len(self.nonAlignedArg) + len(self.oneToManyAlignedArg) + self._cntManyToOneAlignedArg, \
		              "Total projection case count":           len(self.nonProjectedPred) + len(self.oneToManyProjectedPred) + self._cntManyToOneProjectedPred + len(self.nonProjectedArg) + len(self.oneToManyProjectedArg) + self._cntManyToOneProjectedArg, \
		              "Total predicate case count":            len(self.nonAlignedPred) + len(self.nonProjectedPred) + len(self.oneToManyAlignedPred) + len(self.oneToManyProjectedPred) + self._cntManyToOneAlignedPred + self._cntManyToOneProjectedPred, \
		              "Total argument case count":             len(self.nonAlignedArg) + len(self.nonProjectedArg) + len(self.oneToManyAlignedArg) + len(self.oneToManyProjectedArg) + self._cntManyToOneAlignedArg + self._cntManyToOneProjectedArg, \
		              "Total problem count":                   len(self.nonAlignedPred) + len(self.oneToManyAlignedPred) + self._cntManyToOneAlignedPred + len(self.nonAlignedArg) + len(self.oneToManyAlignedArg) + self._cntManyToOneAlignedArg + \
		                                                       len(self.nonProjectedPred) + len(self.oneToManyProjectedPred) + self._cntManyToOneProjectedPred + len(self.nonProjectedArg) + len(self.oneToManyProjectedArg) + self._cntManyToOneProjectedArg}
		
		vdPercentage = {"Non-aligned predicate percentage":           util.percent(len(self.nonAlignedPred), self.srcSRL.propCount), \
		                "Non-projected predicate percentage":         util.percent(len(self.nonProjectedPred), self.srcSRL.propCount), \
		                "One-to-many aligned predicate percentage":   util.percent(len(self.oneToManyAlignedPred), self.srcSRL.propCount), \
		                "One-to-many projected predicate percentage": util.percent(len(self.oneToManyProjectedPred), self.srcSRL.propCount), \
		                "Many-to-one aligned predicate percentage":   util.percent(self._cntManyToOneAlignedPred, self.srcSRL.propCount), \
		                "Many-to-one projected predicate percentage": util.percent(self._cntManyToOneProjectedPred, self.srcSRL.propCount), \
		                "Non-aligned argument percentage":            util.percent(len(self.nonAlignedArg), self.srcSRL.globalArgCount), \
		                "Non-projected argument percentage":          util.percent(len(self.nonProjectedArg), self.srcSRL.globalArgCount), \
		                "One-to-many aligned argument percentage":    util.percent(len(self.oneToManyAlignedArg), self.srcSRL.globalArgCount), \
		                "One-to-many projected argument percentage":  util.percent(len(self.oneToManyProjectedArg), self.srcSRL.globalArgCount), \
		                "Many-to-one aligned argument percentage":    util.percent(self._cntManyToOneAlignedArg, self.srcSRL.globalArgCount), \
		                "Many-to-one projected argument percentage":  util.percent(self._cntManyToOneProjectedArg, self.srcSRL.globalArgCount)}
		
		vdAggPercentage = {"Total non-aligned percentage":               util.percent(len(self.nonAlignedPred) + len(self.nonAlignedArg), self.srcSRL.propCount + self.srcSRL.globalArgCount), \
		                   "Total non-projected percentage":             util.percent(len(self.nonProjectedPred) + len(self.nonProjectedArg), self.srcSRL.propCount + self.srcSRL.globalArgCount), \
		                   "Total one-to-many alignment percentage":     util.percent(len(self.oneToManyAlignedPred) + len(self.oneToManyAlignedArg), self.srcSRL.propCount + self.srcSRL.globalArgCount), \
		                   "Total one-to-many projection percentage":    util.percent(len(self.oneToManyProjectedPred) + len(self.oneToManyProjectedArg), self.srcSRL.propCount + self.srcSRL.globalArgCount), \
		                   "Total many-to-one alignment percentage":     util.percent(self._cntManyToOneAlignedPred + self._cntManyToOneAlignedArg, self.srcSRL.propCount + self.srcSRL.globalArgCount), \
		                   "Total many-to-one projection percentage":    util.percent(self._cntManyToOneProjectedPred + self._cntManyToOneProjectedArg, self.srcSRL.propCount + self.srcSRL.globalArgCount), \
		                   "Total predicate alignment case percentage":  util.percent(len(self.nonAlignedPred) + len(self.oneToManyAlignedPred) + self._cntManyToOneAlignedPred, self.srcSRL.propCount), \
		                   "Total predicate projection case percentage": util.percent(len(self.nonProjectedPred) + len(self.oneToManyProjectedPred) + self._cntManyToOneProjectedPred, self.srcSRL.propCount), \
		                   "Total argument alignment case percentage":   util.percent(len(self.nonAlignedArg) + len(self.oneToManyAlignedArg) + self._cntManyToOneAlignedArg, self.srcSRL.globalArgCount), \
		                   "Total argument projection case percentage":  util.percent(len(self.nonProjectedArg) + len(self.oneToManyProjectedArg) + self._cntManyToOneProjectedArg, self.srcSRL.globalArgCount), \
		                   "Total alignment case percentage":            util.percent(len(self.nonAlignedPred) + len(self.oneToManyAlignedPred) + self._cntManyToOneAlignedPred + len(self.nonAlignedArg) + len(self.oneToManyAlignedArg) + self._cntManyToOneAlignedArg, self.srcSRL.propCount + self.srcSRL.globalArgCount), \
		                   "Total projection case percentage":           util.percent(len(self.nonProjectedPred) + len(self.oneToManyProjectedPred) + self._cntManyToOneProjectedPred + len(self.nonProjectedArg) + len(self.oneToManyProjectedArg) + self._cntManyToOneProjectedArg, self.srcSRL.propCount + self.srcSRL.globalArgCount), \
		                   "Total predicate case percentage":            util.percent(len(self.nonAlignedPred) + len(self.nonProjectedPred) + len(self.oneToManyAlignedPred) + len(self.oneToManyProjectedPred) + self._cntManyToOneAlignedPred + self._cntManyToOneProjectedPred, self.srcSRL.propCount), \
		                   "Total argument case percentage":             util.percent(len(self.nonAlignedArg) + len(self.nonProjectedArg) + len(self.oneToManyAlignedArg) + len(self.oneToManyProjectedArg) + self._cntManyToOneAlignedArg + self._cntManyToOneProjectedArg, self.srcSRL.globalArgCount), \
		                   "Total problem percentage":                   util.percent(len(self.nonAlignedPred) + len(self.oneToManyAlignedPred) + self._cntManyToOneAlignedPred + len(self.nonAlignedArg) + len(self.oneToManyAlignedArg) + self._cntManyToOneAlignedArg + \
		                                                                              len(self.nonProjectedPred) + len(self.oneToManyProjectedPred) + self._cntManyToOneProjectedPred + len(self.nonProjectedArg) + len(self.oneToManyProjectedArg) + self._cntManyToOneProjectedArg, self.srcSRL.propCount + self.srcSRL.globalArgCount)}
		
		return vdCount, vdAggCount, vdPercentage, vdAggPercentage
		
	
	
	def printStat(self, pflgCountStat = True, pflgAggCountStat = False, pflgPercentStat = True, pflgAggPercentStat = True):
		'''
		Print the projection statistics in user-friendly format
		
		4 types of statistics are printed each upon request:
		- Counts
		- Aggregation of counts based on various factors
		- Percentages
		- Aggregation of percentages based on various factors
		'''
		
		vdCntStat, vdAggCntStat, vdPctStat, vdAggPctStat = self.getStat()
		
		vPredStatTxt = "Non-aligned predicate count ............. %s \n" % vdCntStat["Non-aligned predicate count"] + \
		               "Non-aligned predicate pct. .............. %s \n" % vdPctStat["Non-aligned predicate percentage"] + \
		               "Non-projected predicate count ........... %s \n" % vdCntStat["Non-projected predicate count"] + \
		               "Non-projected predicate pct. ............ %s \n" % vdPctStat["Non-projected predicate percentage"] + \
		               "One-to-many aligned predicate count ..... %s \n" % vdCntStat["One-to-many aligned predicate count"] + \
		               "One-to-many aligned predicate pct. ...... %s \n" % vdPctStat["One-to-many aligned predicate percentage"] + \
		               "One-to-many projected predicate count ... %s \n" % vdCntStat["One-to-many projected predicate count"] + \
		               "One-to-many projected predicate pct. .... %s \n" % vdPctStat["One-to-many projected predicate percentage"] + \
		               "Many-to-one aligned predicate count ..... %s \n" % vdCntStat["Many-to-one aligned predicate count"] + \
		               "Many-to-one aligned predicate pct. ...... %s \n" % vdPctStat["Many-to-one aligned predicate percentage"] + \
		               "Many-to-one projected predicate count ... %s \n" % vdCntStat["Many-to-one projected predicate count"] + \
		               "Many-to-one projected predicate pct. .... %s \n" % vdPctStat["Many-to-one projected predicate percentage"]
		
		vArgStatTxt = "Non-aligned argument count .............. %s \n" % vdCntStat["Non-aligned argument count"] + \
		              "Non-aligned argument pct. ............... %s \n" % vdPctStat["Non-aligned argument percentage"] + \
		              "Non-projected argument count ............ %s \n" % vdCntStat["Non-projected argument count"] + \
		              "Non-projected argument pct. ............. %s \n" % vdPctStat["Non-projected argument percentage"] + \
		              "One-to-many aligned argument count ...... %s \n" % vdCntStat["One-to-many aligned argument count"] + \
		              "One-to-many aligned argument pct. ....... %s \n" % vdPctStat["One-to-many aligned argument percentage"] + \
		              "One-to-many projected argument count .... %s \n" % vdCntStat["One-to-many projected argument count"] + \
		              "One-to-many projected argument pct. ..... %s \n" % vdPctStat["One-to-many projected argument percentage"] + \
		              "Many-to-one aligned argument count ...... %s \n" % vdCntStat["Many-to-one aligned argument count"] + \
		              "Many-to-one aligned argument pct. ....... %s \n" % vdPctStat["Many-to-one aligned argument percentage"] + \
		              "Many-to-one projected argument count .... %s \n" % vdCntStat["Many-to-one projected argument count"] + \
		              "Many-to-one projected argument pct. ..... %s \n" % vdPctStat["Many-to-one projected argument percentage"] 
		
		vAggStatTxt = "Total non-aligned count .............. %s \n" % vdAggCntStat["Total non-aligned count"] + \
		              "Total non-aligned pct. ............... %s \n" % vdAggPctStat["Total non-aligned percentage"] + \
		              "Total non-projected count ............ %s \n" % vdAggCntStat["Total non-projected count"] + \
		              "Total non-projected pct. ............. %s \n" % vdAggPctStat["Total non-projected percentage"] + \
		              "Total one-to-many alignment count .... %s \n" % vdAggCntStat["Total one-to-many alignment count"] + \
		              "Total one-to-many alignment pct. ..... %s \n" % vdAggPctStat["Total one-to-many alignment percentage"] + \
		              "Total one-to-many projection count ... %s \n" % vdAggCntStat["Total one-to-many projection count"] + \
		              "Total one-to-many projection pct. .... %s \n" % vdAggPctStat["Total one-to-many projection percentage"] + \
		              "Total many-to-one alignment count .... %s \n" % vdAggCntStat["Total many-to-one alignment count"] + \
		              "Total many-to-one alignment pct. ..... %s \n" % vdAggPctStat["Total many-to-one alignment percentage"] + \
		              "Total many-to-one projection count ... %s \n" % vdAggCntStat["Total many-to-one projection count"] + \
		              "Total many-to-one projection pct. .... %s \n" % vdAggPctStat["Total many-to-one projection percentage"] + \
		              "Total alignment case count ........... %s \n" % vdAggCntStat["Total alignment case count"] + \
		              "Total alignment case pct. ............ %s \n" % vdAggPctStat["Total alignment case percentage"] + \
		              "Total projection case count .......... %s \n" % vdAggCntStat["Total projection case count"] + \
		              "Total projection case pct. ........... %s \n" % vdAggPctStat["Total projection case percentage"] + \
		              "Total predicate case count ........... %s \n" % vdAggCntStat["Total predicate case count"] + \
		              "Total predicate case pct. ............ %s \n" % vdAggPctStat["Total predicate case percentage"] + \
		              "Total argument case count ............ %s \n" % vdAggCntStat["Total argument case count"] + \
		              "Total argument case pct. ............. %s \n" % vdAggPctStat["Total argument case percentage"] + \
		              "Total problem count .................. %s \n" % vdAggCntStat["Total problem count"] + \
		              "Total problem pct. ................... %s \n" % vdAggPctStat["Total problem percentage"]
		
		sys.stdout.write(vPredStatTxt + '\n' + vArgStatTxt + '\n' + vAggStatTxt + '\n')
		
	
	


class DepSRLProject(SRLProject):
	'''
	Base Class for projecting SRL from one the source side of the translation
	to the target side
	'''
	
	def __init__(self, pSrcSurface, pSrcSRL, pSrcDepTree, pSrcLang):
		'''
		Constructor
		'''
		
		SRLProject.__init__(self, pSrcSurface, pSrcSRL, pSrcParse = pSrcDepTree, pSrcLang = pSrcLang)
		
	
	
	def waProject(self, pTrgSurface, pTrgDepTree, pWordAlignment, pTrgLang, pPredOneToManyMethod = 0, pPredOverlapMethod = 0, pArgOneToManyMethod = 0, pArgOverlapMethod = 0, pflgClassicDRule = False):
		'''
		Projects source side SRL to given target dependency tree using 
		given word alignment, loads the projected SRL to the tree and 
		returns the projected SRL
		
		Word alignment is of type mt.WordAlignment
		
		pPredOneToManyMethod specifies the method to resolve one-to-many
		predicate projection (see _resPredOneToMany()). Default is method
		1.
		
		pPredOverlapMethod specifies the method to resolve many-to-one
		predicate projection (target predicate overlap; see _resPredOverlap()).
		Default is method 1.
		
		pArgOneToManyMethod specifies the method to resolve one-to-many
		argument projection (see _resArgOneToMany()). Default is project
		to many (do not resolve).
		
		pArgOverlapMethod specifies the method to resolve many-to-one argument
		projection (target argument overlap; see _resArgOverlap()). Default 
		is project to many (do not resolve).
		
		if pflgClassicDRule set to true, the D-Rule used in Classic project
		is applied in argument transfer. Default is false.
		'''
		
		self._initReport()
		#### reconsider
		self._initStat()
		
		vProjSRL = SRL(pLanguage = pTrgLang, pType = 'd')
		
		# projection
		
		vlPredsAndProjTrgPositions = []
		for vSrcProp in self.srcSRL.propositions:
			# 1. projecting predicate
			
			vProjPredPos = self._waProjectPred(vSrcProp.predicate, pWordAlignment, pTrgDepTree, pPredOneToManyMethod)
			vlPredsAndProjTrgPositions.append((vSrcProp.predicate, vProjPredPos))
			
			
			# 2. projecting arguments
			
			## NOTE: we do not create a target proposition if no word
			## is found to be aligned with source predicate. However, we
			## proceed with argument projection to extract statistics.
			## ToDo: find alternative ways to project non-aligned predicates
			vlArgsAndProjTrgPositions = []
			for vSrcArg in vSrcProp.arguments:
				## When a source argument role filler is aligned with more
				## than one token in the target, there will be more than 
				## one token assigned the same role in the target except
				## some cases (see _projectArg).
				vlProjArgPositions = self._waProjectArg(vSrcArg, pWordAlignment, vSrcProp, pTrgDepTree, pTrgLang, pArgOneToManyMethod, pflgClassicDRule)
				vlArgsAndProjTrgPositions.append((vSrcArg, vlProjArgPositions))
			
			
			# 3. creating and adding projected propositions (only if predicate is projected)
			# ToDo: Move to a general method for all projection methods (eg. self.project())
			
			if vProjPredPos != None:
				# creating predicate
				vProjPred = DepPred(vSrcProp.predicate.getLabel(), vProjPredPos, pTrgDepTree.getNode(vProjPredPos))
				
				# creating proposition
				vProjProp = Proposition(vProjPred)
				
				# creating and adding arguments
				vlProjArgs = []
				for vSrcArg, vlProjArgPositions in vlArgsAndProjTrgPositions:
					vProjProp.addArguments([DepArg(pArgLabel = vSrcArg.getLabel(), 
					                               pTokenPos = vProjArgPos, 
					                               pDepNode = pTrgDepTree.getNode(vProjArgPos), 
					                               pProposition = vProjProp, 
					                               pLanguage = pTrgLang) 
					                        for vProjArgPos in vlProjArgPositions])
				
				# adding proposition
				vProjSRL.addProposition(vProjProp)
			
			
			# statistics of argument projection
			
			# finding many-to-one argument projections for the proposition
			vlProjArgPositions = [vPos for vArgPositions in vlArgsAndProjTrgPositions for vPos in vArgPositions[1]]
			for vPosCnt in util.groupBy(vlProjArgPositions).itervalues():
				if vPosCnt > 1:
					self._cntManyToOneProjectedArg += 1
			
			# finding many-to-one argument alignments for the proposition (independent of projection)
			vlSrcArgPositions = [a.position for a in vSrcProp.arguments]
			self._cntManyToOneAlignedArg += len(pWordAlignment.subsetBySrcPos(vlSrcArgPositions).subsetManyToOne().getTrgPositions())
		
		
		# statistics of predicate projection
		
		# finding many-to-one predicate projections 
		vlTrgPredPositions = [vPredPos[1] for vPredPos in vlPredsAndProjTrgPositions if vPredPos[1] != None]
		for vPosCnt in util.groupBy(vlTrgPredPositions).itervalues():
			if vPosCnt > 1:
				self._cntManyToOneProjectedPred += 1
		
		# finding many-to-one predicate alignments (independent of projection)
		vlSrcPredPoistions = [p.predicate.position for p in self.srcSRL.propositions]
		self._cntManyToOneAlignedPred += len(pWordAlignment.subsetBySrcPos(vlSrcPredPoistions).subsetManyToOne().getTrgPositions())
		
		
		pTrgDepTree.loadSRL(vProjSRL)
		
		return vProjSRL
		
	
	
	def _waProjectPred(self, pSrcPred, pWordAlignment, pTrgDepTree, pPredOneToManyMethod):
		'''
		Projects given source predicate to target using given word alignment
		and returns projected target position
		'''
		
		# A word may be aligned to more than one words in the other side
		vlTrgPredPos = pWordAlignment.getTrgAlignedTo(pSrcPred.position)
		
		if len(vlTrgPredPos) == 0:
			self.nonAlignedPred.append(pSrcPred)
			self.nonProjectedPred.append(pSrcPred)
			# statistics: remove
			#self._cntNonAlignedPred += 1
			#self._cntNonProjectedPred += 1
			
			return None
		elif len(vlTrgPredPos) == 1:
			vTrgPredPos = vlTrgPredPos[0]
		elif len(vlTrgPredPos) > 1:                 # 1-to-n alignment case
			self.oneToManyAlignedPred.append(pSrcPred)
			# statistics: remove
			#self._cntOneToManyAlignedPred += 1
			
			vTrgPredPos = self._resPredOneToMany(vlTrgPredPos, pPredOneToManyMethod, pTrgDepTree)
		
		return vTrgPredPos
		
	
	
	def _waProjectArg(self, pSrcArg, pWordAlignment, pProposition, pTrgDepTree, pTrgLang, pArgOneToManyMethod, pflgClassicDRule):
		'''
		Projects given source argument to target using given word alignment
		and returns projected target position
		'''
		
		# A word may be aligned to more than one words in the other side
		vlTrgArgPos = pWordAlignment.getTrgAlignedTo(pSrcArg.position)
		
		if len(vlTrgArgPos) == 0:
			self.nonAlignedArg.append(pSrcArg)
			# statistics: remove
			#self._cntNonAlignedArg += 1
			
			# applying Classic D-Rule
			if pflgClassicDRule:
				vlTrgArgPos = self._applyClassicDRule(pSrcArg, pWordAlignment)
			else:
				vlTrgArgPos = []
		
		if len(vlTrgArgPos) == 0:
			self.nonProjectedArg.append(pSrcArg)
			# statistics: remove commented
			#self._cntNonProjectedArg += 1
			
			return []
		elif len(vlTrgArgPos) > 1:                    # 1-to-n alignment
			vlTrgArgPos = self._resArgOneToMany(vlTrgArgPos, pArgOneToManyMethod, pTrgDepTree)
			
			self.oneToManyAlignedArg.append(pSrcArg)
			# statistics: remove commented
			#self._cntOneToManyAlignedArg += 1
			if len(vlTrgArgPos) > 1:
				self.oneToManyProjectedArg.append(pSrcArg)
				#self._cntOneToManyProjectedArg += 1
			
		
		return vlTrgArgPos
		
	
	
	def _resPredOneToMany(self, plProjPositions, pMethod, pTrgDepTree):
		'''
		Resolves one-to-many predicate projection case using given method
		
		The default is to return the first position in the list.
		'''
		
		
		def _resPredOneToMany1(plProjPositions, pTrgDepTree):
			'''
			Resolves one-to-many predicate projection case using method 1
			
			This method chooses the first verb word among Many target words.
			If a verb is not found, it chooses the first word.
			
			ToDo: improve tie-breaking
			'''
			
			vlVerbPos = [pos for pos in plProjPositions if pTrgDepTree.getNode(pos).isVerb()]
			if len(vlVerbPos) == 0:
				return plProjPositions[0]
			else:
				return vlVerbPos[0]
			
		
		if pMethod == 0:
			return plProjPositions[0]
		elif pMethod == 1:
			return _resPredOneToMany1(plProjPositions, pTrgDepTree)
		
	
	
	def _resPredOverlap(self, pMethod):
		'''
		Resolves many-to-one predicate projection case (target predicate
		overlap) using given method
		'''
		
		def _resPredOverlap1(self):
			'''
			Resolves many-to-one predicate projection case (target predicate
			overlap) using method 1
			
			This method checks whether there is other alignment for f
			'''
			
			pass
		
	
	
	def _resArgOneToMany(self, plProjPositions, pMethod, pTrgDepTree):
		'''
		Resolves one-to-many argument projection case using given method
		'''
		
		
		def _resArgOneToMany1(plProjPositions, pTrgDepTree):
			'''
			Resolves one-to-many argument projection case using method 1
			
			This method does the following:
			1- remove all the words in Many side which are dependents of
			  any other word in Many.
			2- create an argument for each remaining word in Many
			
			The purpose is to assign the role to only head if both dependent 
			and nodes are among words in Many.
			
			ToDo: improve the tie-breaking
			'''
			
			def __isHeadIn(pPos, plPos, pDepTree):
				'''
				Returns true if the position of a head of a node at pPos is
				in plPos using pDepTree
				'''
				
				# handling multiple head
				for vHead in pDepTree.getNode(pos).getHeadPos():
					if vHead in plPos:
						return True
				
				return False
				
			
			
			vlHeadPos = [pos for pos in plProjPositions if not __isHeadIn(pos, plProjPositions, pTrgDepTree)]
			if len(vlHeadPos) == 0:
				# a rare situation!
				raise Exception("All-circular head/dependent in 1-ton alignment of argument role filler!")
			else:
				return vlHeadPos
		
		
		if pMethod == 0:
			return plProjPositions
		elif pMethod == 1:
			return _resArgOneToMany1(plProjPositions, pTrgDepTree)
		
	
	
	def _resArgOverlap(self, pMethod):
		'''
		Resolves many-to-one argument projection case (target predicate
		overlap) using given method
		'''
		
		pass
		
	
	
	def _applyClassicDRule(self, pSrcArg, pWordAlignment):
		'''
		Applies D-Rule used by Classic project and returns the position(s)
		of target argument if successful otherwise an empty list
		
		This is the quote from Classic delivery 2 report:
		
		For any pair of translated sentences E and F and a semantic 
		relationship R(xE, yE) in E, if there exists a word-alignment 
		between predicates xE and xF but not between roles yE and any yF,
		and if the POS of yE is 'IN' or 'TO', then we find the dependent
		y'E of yE and if there exists a word-alignment between y'E and 
		some y'F, we transfer the semantic relationship R(xE, yE) to 
		R(xF, y'F).
		'''
		
		if not pSrcArg.getPOSTag() in ["IN", "TO"]:
			return []
		
		vlTrgArgPos = []
		
		for vDepPosition in pSrcArg.getNode().getDepPositions():
			vlTrgArgPos = pWordAlignment.getTrgAlignedTo(vDepPosition)
			if len(vlTrgArgPos) > 0:
				return vlTrgArgPos
		
		return vlTrgArgPos