Inference Web: Portable and Sharable Explanations for ... - CiteSeerX

Inference Web:

Portable and Sharable

Explanationsfor Question Answering DeborahLMcGuinness . andPaulo Pinheiro dS a ilva Knowledge SystemsLaboratory StanfordUniversity Stanford,CA 94305 {dlm pp}@ksl.stanford.edu |

Abstract TheWorldWideWeblackssupportforexplaining informationprovenance.Whenwebapplicationsretur n results,manyusersdonotknowwhatinformationso urces wereused,whentheywereupdated,howreliablethe source was,whatinformationwaslookedupversusderived, andif somethingwasderived,howitwasderived. Inthis paper weintroducetheInferenceWeb(IW)thataddresses the problemsoof paquequeryanswersbyproviding sharable, combinable,anddistributed explanations. Theexplanations includeinformationconcerningwhereanswerscamef rom andhowtheywerededuced(orretrieved).TheIWso lution includes:anextensibleregistrycontaining detailson information sources and reasoners, a portableproof specification,andanexplanationbrowser.

Motivation InferenceWeb(IW)aimstoenableapplicationsthat can generatesharableanddistributedexplanationsfor anyof theirresults. Therearemanyreasonsthatusersa ndagents needtounderstand theprovenanceoinformation f thatthey getbackfromapplications. Themainmotivatingfa ctors forusareinteroperability,reuse,andtrust. Int eroperability isessentialifagentsaretocollaborate. Trusta ndreuseof retrievalanddeductionprocessesisfacilitatedwh en explanationsareavailable.Ultimately,ifusersan d/or agentsareexpectedtotrustinformationandaction sof applicationsandiftheyareexpectedtouseandre use applicationresultspotentiallyincombinationwith other informationorotherapplicationresults,theymay needto haveaccesstomanykindsoifnformationsuchasso urce, recency,authoritativeness,methodofreasoning,te rm meaning andinterrelationships,etc. Inourwork,webuildonpastexperiencedesigning explanation components for reasoning systems [McGuinness,1996;McGuinnessandBorgida,1995; Borgida,et.al,1999]andexperiencedesigningque ry componentsforframe-likesystems[McGuinness,1996 ; BorgidaandMcGuinness,1996]togeneraterequireme nts. WealsoobtainedinputfromcontractorsinthreeDA RPAsponsored programs concerning knowledge-based

applications(theHighPerformanceKnowledgeBase 2 program1Rapid , KnowledgeFormationProgram and , the 3 DARPAAgentMarkupLanguageProgram ).Wealso obtainedrequirementsfromliteratureonexplanatio nfor expertsystems[Swartout,et.al.,1991]andusabil ityof knowledge representation systems [McGuinness-PatelSchneider,1998and2003]. Inthispaper,weinclude a listofrequirementsgathered frompastworkandfromsurveyingusers.Wepresent the IWarchitectureandprovideadescriptionofthema jor componentsincludingtheportableproofspecificati on,the registry(containinginformationaboutinferenceen gines, proofmethods,andontologies),andthejustificati on browser. Wealsoprovidesomesimpleusageexample s. Weconcludewithourcontributionsintheareasof application interoperability,reuse,andtrust.

Requirements Ifhumansandagentsneedtomakeinformeddecision aboutwhenandhowtouseanswersfromapplications therearemanythingstoconsider. Decisionswill onthequalityothe f sourceinformation,thesuita qualityofthereasoningengine,andthecontextof situation.Firstwewillconsiderissuesconcernin sourceinformation. Evenwhenapplicationssuchasearchenginesor systemsjustlookupassertedor“told”information (andagents)mayneedtounderstandwherethesourc informationcamefromatvaryingdegreesofdetail. Sometimesthisinformationiscalledprovenanceand beviewedasmetainformationabouttoldinformatio Provenanceinformation may include: • Sourcename(e.g.,CIA WorldFactBook) • Dateand author(s)of lastupdate • Author(s)of originalinformation 1 http://reliant.teknowledge.com/HPKB/ 2 http://reliant.teknowledge.com/RKF/ 3 http://www.daml.org

s , bebased bilityand the gthe database users , e may n.

• Authoritativenessotfhesource(isthisknowledge store considered or certifiedareliable s bthird ay party?) • Degreeobelief f • Degreeocompleteness f (Within particular a scope, is the sourceconsideredcomplete. Forexample,doesthis sourcehavealloftheemployeesofaparticular organizationupuntilaparticular date?Ifso,not finding particular a employeewouldmean thatthey are notemployed,countingemployeeswouldbean accurateresponsetonumber of employees,etc.) Theinformationabovecouldbehandledwithmeta informationaboutcontentsourcesandaboutindivid assertions.Additionaltypesofinformationmaybe requiredifusersneedtounderstandthemeaningof orimplicationsofqueryanswers. Ifapplications deductionsootherwise r manipulateinformation,use needtounderstandhowdeductionsweremadeandwha manipulationsweredone.Informationconcerningte meaning,derivedomanipulated r information may inc • Termorphrasemeaning(innaturallanguageofaro language) • Terminter-relationships(ontologicalrelationsinc subclass,superclass,part-of,etc.) • Thesourceofderivedinformation(reasonerused, reasoner method,reasoner inferencerule,etc.) • Reasonerdescription(isthereasonerusedknownto soundandcomplete?) • Term uniqueness (is D.L. McGuinness the same individualas Deborah McGuinness?) • Term coherence(is paarticular definition incohere • Sourceconsistency(istheresupportinsystem a fo A and~A) • Wereassumptionsusedinaderivation? Ifso,have assumptions changed?

ual terms make rsmay t rm lude: rmal luding

Inference Web be

nt?) both r the

Use Cases Everycombinationofaquerylanguagewithaqueryansweringenvironmentisapotentialnewcontextfo trhe InferenceWeb.Weprovidetwoscenariosasmotivati ng usecases.Considerthesituationwheresomeonehas analyzedasituationpreviouslyandwantstoretrie vethis analysis. Inordertopresentthefindings,thean alystmay needtodefendtheconclusionsbyexposingthereas oning pathusedalongwiththesourceoftheinformation. In orderfortheanalysttoreusethepreviouswork,s /hewill alsoneedtodecideifthesourceinformationused previouslyistillvalid(andpossiblyitfhereas oningpath is stillvalid). Anothersimplemotivatingexampleariseswhen use a asks r forinformationfromawebapplicationandthennee dsto decidewhethertoactontheinformation. Forexam ple,a usermightuseasearchengineinterfaceoramore 1 forretrieving sophisticatedquerylanguagesuchasDQL 1

http://www.daml.org/2002/08/dql/.

lookupinformationsuchas“redcashmeresweater”o r “findaredcashmeresweatercostinglessthan200 dollars thatisreadyforimmediateshipping”. Moreover,theuser mightaskforanexplanationalongwithanswerssin ces/he may wantinformationsuchatshedatacamefrom ra eliable merchantandthedatawas updateditnhelast24ho urs. Inorderforthisscenariotoboe perationalized,w ne eedto havethefollowing: • Away forapplications(reasoners,retrievalengine s,etc.) todumpjustificationsfortheiranswersiforma na that t otherscanunderstand.Tosolvethisproblemwe introduce portable a andsharableproof specificati on. • Aplaceforreceiving,storing,manipulating,annot ating, metainformationusedto comparing,andreturning enrichproofsandprooffragments. Inordertoadd ress thisrequirement,weintroducetheInferenceWeb Registryforstoringthemetainformationandthe InferenceWebRegistrarwebapplicationforhandlin g theRegistry. • Awaytopresentjustificationstotheuser.Ason e solution tothis problem,weintroduce proof a brow ser.

Webeginwithshort a descriptionodifferent f cate goriesof InferenceWebusers. Theseusersalongwiththeus age examplesabovemotivatethethreemaincomponentso f InferenceWeb:portableproofs,registry,andproo f browsers. Theprimeusers of inferencewebare: • Applicationdevelopers(authorsofreasoners,sear ch engines,database systems,etc.)whowouldliketo defendwhy their answers toqueries shouldbbeelie ved orwhowouldliketostateunderwhatconditionsth eir systems arebestused. • Authorsofhybridsolutionsprogramsinterestedin combining multiple answering systems and/or knowledgebases. Thesepeopleneedtounderstand howtermsrelatetoeachotherandhowanswerswere derivedandmightintegrate. Examplesosuch f peop le includeontologybuilderswhoaremergingontologie s orextendingontologies,crawlerorwrapperauthors , peoplecombiningdatabasesorknowledgebased systems,etc. • Humanoragents needingtodecideiftheycantrust eitherretrievedinformationorinferenceprocesses usedtoretrieveinformation.Thisusermayview partialor completejustifications for answers. PortableProofs: Systemsthatmaybeaskedtoreturna justificationforanansweralongwithananswerne exposeprovenanceinformationalongwiththeirdedu processincludingpossiblymetainformationaboutt

edto ctive he

systemitself. Weprovideaspecificationinwritt eninthe webmarkuplanguageDAML+OIL [Dean,et.al,2002]. Ourportableproofspecificationincludesthefour major componentsofIWprooftrees: inferencerules,inference steps,wellformedformulae(WFFs),andontologies. Inferencerules(suchasmodusponens)canbeused to deduceaconsequent(awellformedformula)froman y numberofantecedents(alsowellformedformulae). An inferencestep isasingleapplicationofaninferencerule. Theinferencestepwillbeassociatedwiththecons equent WFFanditwillcontainpointerstotheantecedent WFFs, theinferenceruleused,andanyvariablebindings usedin theinferenceruleapplication.TheantecedentWFFs may comefromotherinferencesteps,existingontologie s, extraction from documents,or they may baessumptio ns. Aproofcanthenbedefinedasatreeofinference steps explainingtheprocessodeducing f theconsequentW FF.In Inference Web,proofsare treesopf rooffragments rather thansinglemonolithicproofs.IWproofsareproof fragmentssincethelastinferencestepusedtoder ivea WFFcanbepresentedalonewithlinkstoitsantece dents andvariablebindings. Askinghoweachantecedent WFF was derived generates follow-up questions. These individualprooffragmentsmaybecomposedtogether to generatecomplete a proof,i.e.,saetofinference stepsthat havenoantecedentsthatarederivedratherthanas serted information. AWFFmaybteheconsequentofanynumberofinfere nce steps. Thiscanbeusedtosupportmultiplejustif ications foranyparticularWFF. WFFsmaynotbetheconseq uent ofanyinferencestepiftheyareassumptionsorme rely assertedinformationinanontologythattheuseri s referencing. ThespecificationoIW f conceptsis available athttp://www.ksl.stanford.edu/software/IW/spec. Registry: TheIWregistryiscurrentlyacentralized repositoryoifnformationusedtoenrichexplanatio nswith detailsabout authoritativesources , ontologies, inference engines,and inferencerules Our . registry includes template informationabouteachoftheclassesintheregist ry. For example,inferenceenginesmayhavethefollowing propertiesassociatedwiththem: name,URL,author (s), date,versionnumber,organization,etc.Thecurre nt demonstration registry is available at: http://belo.stanford.edu:8080/iwregistry/BrowseRegistry.js p. Informationintheregistrycontainstheinformatio nlinked tointheproofs.Everyinferencestepshouldhave lainkto atleastoneinferenceenginethatwasresponsible for instantiatingtheinferencestepitself,ascanbe observedin Figure1. Thedescriptionofinferencerulesisoneofthemo st importantfeaturesotfheRegistry.Registeredrule csanbe atomicocan r bde erivedfromotherregisteredrule s.Thus, reasoner-specificrulescanbeexplainedintheReg istry beforethereasoneriasctuallyusedtogenerateIW proofs. Inferencewebthusprovidesawaytouseonereason erto explainanotherreasoner’sinferencerules. Thism aybe

usefulforexplainingheavilyoptimizedinferencee Inferenceweb’sregistry,whenfully populated,wil inferencerulesetsformanycommonreasoningsyste Usersmayviewinferencerulesetstohelpthemdec whether touse particular a inferenceengine.

ngines. contain l ms. ide

(a WFF is stored as a predicate logic conjunctive normal-form sentence) type (…) (a WFF can be associated to a set of Inference steps) (…) (inference step antecedents are IW files with their own URIs) (…)

Figure 1AnInference

Webproof.

Browser: WeprovideanInferenceWebbrowserthat displaysprooffragmentstoendusersinanumbero f formats. Initially,weincludelimited a Englishf orm,KIF 1, andconjunctivenormalform. Wealsoexpectthats ome applicationsmayimplementtheirowndisplaysusing our API. Thebrowserimplementsalensmetaphorresponsible for renderingafixednumberoflevelsofinferenceste ps dependingonthelensmagnitude. Thedefaultlen s numberisone,thusthebrowserdisplaystheinfere ncestep usedtoderiveiincluding t its antecedents. 1

http://logic.stanford.edu/kif/kif.html.

Webelievethatoneofthekeystopresentationof justificationsisbreakingproofsintoseparablepi eces. Sincewepresentfragments,automaticfollow-upque stion supportisacriticalfunctionoftheIWbrowser. Every elementintheviewinglenscantriggerabrowsera ction. Theselectionoan fantecedentthatis derivedre-f ocuses the lensonanantecedent’sinferencestep.Forotherl ens elements, associated actions present Registry metainformation intheTrustDisclosurePanel.Theselectionof theconsequentpresentsdetailsabouttheinference engine usedtoderivetheactualtheorem.Theselectionof an inferencerulepresentsadescriptionoftherule. The selectionofanaxiompresentsdetails aboutontologies wheretheaxiom isdefined.Anexampleothis f proc esscan be seen from the Inference Web web pages at: http://www.ksl.stanford.edu/software/iw/Ex1/.

ContributionsandFuture W

ork

InferenceWebprovides thefollowing contributions: • Anarchitecturesupportinginteroperabilitybetween agentsusingportableproofs.Portableproofsare specifiedintheemergingwebstandardDAML+OIL soastoleverageXML-,RDF-,andDAML-based informationservices. Prooffragmentsaw s ellase ntire proofs may bienterchanged. • Lightweightproofbrowsingusingthelens-basedIW proofbrowsersupportingeitherprunedjustificatio ns or guidedviewing ocafompletereasoning path. • Supportforalternativejustificationsusingmultip le inferencesteps.Thiscanallowderivationstobe performedbyperformancereasonerswithexplanation s beinggeneratedbaylternativereasonersmoreaimed at human consumption. • Registry oinference f engines,ontologies,andsour ces. 1 WearecurrentlyextendingtheStanford’sJTP theorem provertoproduceportableproofsandsimultaneousl y populatingtheIWregistrywithJTPinformation. F uture workincludesexpandingtoincludeotherinference engines. Wealsointendto providespecializedsupportfor why-notquestionsexpandingupon[Chalupsky-Russ,20 02] and[McGuinness,1996]. Wearealsolookingaaddi t tional supportfor proof browsing andpruning.

Conclusion Inferencewebenablesapplicationsthatcangenerat e portableandsharableexplanationsoftheirconclus ions. WedescribedthethreemajorcomponentsofIW –the portableproofspecificationbasedontheemerging web language-DAML, the registry of inference engine, inferencerule,andontologyinformation,andtheI Wproof 1

http://www.ksl.stanford.edu/software/jtp/.

browser. WehaveimplementedtheIWapproachforo inferenceengine(JTP)andwencourageadditionalu

ne sers.

Acknowledgements: Wehavereceivedvaluableinputfrom manycolleaguesincludinginparticularRF . ikes,G F . rank,J. Jenkins,andpreviouslyY.LfiromStanford. Addit ionallywe obtainedvaluablecommentsonourspecificationfro mmany including HChalupsky, . P.Clark,K.Forbus,andK. Murray.

References Borgida,A.,EF. ranconi,IH . orrocks,DMcGuin . ness, andP.Patel-Schneider.``ExplainingALCsubsumptio n” ProceedingsotfheInternationalWorkshoponDescri ption Logics(DL-99),Linköping,Sweden,July 1999,pp3336. Chalupsky,Ha. ndT.Russ.WhyNot:DebuggingFail ed QueriesinLargeKnowledgeBases.In Proceedingsothe f Fourteenth Innovative Applications of Artificial IntelligenceConference(IAAI-02) pages , 870-877. Dean,M.,DC . onnolly,Fv. anHarmelen,JH . endle r,I. Horrocks,D.McGuinness,P.Patel-Schneider,andL. Andrea Stein. OWL Web Ontology Language 1.0 Reference.WorldWideWebConsortium(W3C)Working Draft29July2002.Latestversionisavailableat http://www.w3.org/TR/owl-ref/ . RichardFikes,PatHayes,IanHorrocks,editors. DAML Query Language (DQL) Abstract Specification. http://www.daml.org/2002/08/dql/. McGuinness, D. 1996. Explaining Reasoning in DescriptionLogics. Ph.D.Thesis,RutgersUniversity, TechnicalReportLSCR-TR-277. McGuinness,D.and Borgida,A. ExplainingSubsumption inDescriptionLogics .In Proceedingsofthe14th InternationalJointConferenceonArtificialIntell igence, Montreal,Canada. McGuinness, D. and P. Patel-Schneider. ``From DescriptionLogicProverstoKnowledgeRepresentati on Systems''.FranzBaader,DeborahMcGuinness,Daniel e Nardi,andPeterPatel-Schneider,editorsTheDescr iption Logic Handbook: Theory, Implementation, and Applications.CambridgeUniversity Press,2003. McGuinness,D.andPPatel-Schneider. . ``Usabilit Iyssues inKnowledgeRepresentationSystems''.In Proceedingsof the Fifteenth National Conference on Artificial Intelligence,Madison,Wisconsin,July,1998.Updated versiono``Usability f IssuesinDescriptionLogic Systems'' publishedin ProceedingsofInternationalWorkshopon DescriptionLogics ,GifsurYvette,(Paris),France, September,1997. Smith,M.,D.McGuinness,R.VolzandC.Welty.W eb OntologyLanguage(OWL)GuideVersion1.0.World WideWebConsortium(W3C)WorkingDraft.Latest http://www.w3.org/TR/owl-guide/ . version iavailable s at Swartout,W.,C. Paris,andJ.Moore. “ Explanationsin Knowledge Systems: Design for Explainable Expert Systems”. In IEEE Intelligent Systems,June1991. http://www.computer.org/intelligent/ex1991/x3058abs.htm.