ISOIECTR24028-2020信息技术人工智能人工智能的可信度概述.docx

TECHNICALREPORTISO/IECTR24028Firstedition2020-05InformationtechnologyArtificialintelligence-OverviewoftrustworthinessinartificialintelligenceTechnologiesdeVinformationIntelligenceartificielleExamend'ensembledelaJkIbiIiiCenmatieredintelligenceartificielle3 by *<Siow hNo reproduction Or Gtworkng PeEned without license from IHSNot for fwte.06103-202020 29:40 MDTReferencenumberISO/IECTR24028:2020(E)COPYRIGHTPROTECTEDDOCUMENTISO/EC2()20Allrightsreserved.Unlessotherwisespecified,orrequiredinthecontextofitsimplementation,nopartofthispublicationmaybereproducedorutilizedotherwiseinanyformorbyanymeans,electronicormechanical,includingphotocopying,orpostingontheinternetoranintranet,withoutpriorwrittenpermission.PermissioncanberequestedfromeitherISOaltheaddressbeloworISO'smemberbodyinthecountryoftherequesterISOcopyrightofficeCP401Ch.deBlandonnet8CH-1214Vemier1GenevaPhone:+41227490111Fax41227490947Email:copyrightiso.orgWebsite:www.iso.orgPublishedinSwitzerlandmationalOrganizatianforStarxSarcizationProvidedbyIHSMar¼tunderIioemewithIECNorproductaarnetwolhgPennaedwithotlicensefromHSPageviJ J J.7.77.881010UU12121313131414141515151516161616161717171718181919192121212122222323232424iiiContentsIntroduction1 Scope2 Normativereferences3 Termsanddefinitions4 Overview5 Existingframeworksapplicabletotrustworthiness5.1 Background5.2 Recognitionoflayersoftrust5.3 Applicationofsoftwareanddataqualitystandards5.4 Applicationofriskmanagement5.5 Hardware-assistedapproaches6 Stakeholders6.1 Generalconcepts6.2 Types6.3 AsSets6.4 Values7 Recognitionofhigh-levelconcerns7.1 Responsibility,accountabilityandgovernance7.2 Safety8 Vulnerabilities,threatsandchallenges8.1 General8.2 AIspecificsecuritythreats8.2.1 General8.2.2 Datapoisoning8.2.3 Adversarialattacks8.2.4 Modelstealing8.2.5 Hardware-focusedthreatstoconfidentialityandintegrity8.3 Alspecificprivacythreats8.3.1 General8.3.2 Dataacquisition8.3.3 Datapre-processingandmodelling8.3.4 Modelquery8.4 Bias8.5 Unpredictability8.6 Opaqueness8.7 ChallengesrelatedtothespecificationofAlsystems8.8 ChallengesrelatedtotheimplementationofAlsystems8.8.1 Dataacquisitionandpreparation8.8.2 Modelling8.8.3 Modelupdates8.8.4 Softwaredefects8.9 ChallengesrelatedtotheuseOfAlSyStems8.9.1 Human-computerinteraction(HCI)factors8.9.2 MisapplicationofAlsystemsthatdemonstraterealistichumanbehaviour.8.10 Systemhardwarefaults9 Mitigationmeasures9.1 General9.2 Transparency9.3 ExplainabilityCoprioht nenstlSO/ IEC 2O2> Allrights rescncrPRividoI by IHS Markil IlXJCT iccnse with IECNV *eMi*liAif *M (l*tvlit: (»3«ifNcd *ilHwl Iktfnc f<n IliS9.3.1 GeneralO14.56.7.89JJ99.999.9999.3.2 Aimsofexplanation9.3.3 Ex-antevsex-postexplanation9.3.4 Approachestoexplainability9.3.5 Modesofex-postexplanation9.3.6 1.evelsofexplainability9.3.7 EvaluationoftheexplanationsControllability9.4.1 General9.4.2 Human-in-the-loopcontrolpointsStrategiesforreducingbiaPrivacyReliability,resilienceandrobustnessMitigatingsystemhardwarefaultsFunctionalsafetyTestingandevaluation9.10.1 General9.10.2 Softwarevalidationandverificationmethods9.10.3Robustnessconsiderations9.10.4 Privacy-relatedconsiderations9.10.5 SystempredictabilityconsiderationsUseandapplicability9.11.1 Compliance9.11.2 Managingexpectations9.11.3 Productlabelling9.11.4 Cognitivescienceresearch.24242525,26272727282828282929303030323333M.34M.3434AnnexA(infbrmative)RelatedWOrkolISOCietalissues36Bibliography37CoPvVOhlIGrMrtiOnalOrganizationRxSumdiWEitionPiwidedbvIHSMarhkiiunderIiCabewithIECNoptsJulMi*etM>lMUeemilloJsiiaUKittlefruIHSISO/IEC2020-A11rightsreservedNoifo<Resale.O6IO32O2O2029:40MDTForewordISO(theInternationalOrganizationforStandardization)andIEC(theInternationalElectrotechnicalCommission)formthespecializedsystemforworldwidestandardization.NationalbodiesthataremembersofISOorIECparticipateinthedevelopmentofInternationalStandardsthroughtechnicalcommitteesestablishedbytherespectiveorganizationtodealwithparticularfieldsoftechnicalactivity.ISOandIECtechnicalcommitteescollaborateinfieldsofmutualinterest.Otherinternationalorganizations,governmentalandnon-governmental,inliaisonwithISOandIEC,alsotakepartinthework.TheproceduresusedtodevelopthisdocumentandthoseintendedforitsfurthermaintenancearedescribedintheISO/IECDirectives,Partl.Inparticular,thedifferentapprovalcriterianeededforthedifferenttypesofdocumentshouldbenoted.ThisdocumentwasdraftedinaccordancewiththeeditorialrulesoftheISO/IECDirectives,Part2(seewwwiso.org/directives).Attentionisdrawntothepossibilitythatsomeoftheelementsofthisdocumentmaybethesubjectofpatentrights.ISOandIECshallnotbeheldresponsibleforidentifyinganyorallsuchpatentrights.DetailsofanypatentrightsidentifiedduringthedevelopmentofthedocumentwillbeintheIntroductionand/orontheISOlistofpatentdeclarationsreceived(seewww.iso.org/patents)ortheIEClistofpatentdeclarationsreceived(seehttp:/patents.iec.ch).Anytradenameusedinthisdocumentisinformationgivenfortheconvenienceofusersanddoesnotconstituteanendorsement.Foranexplanationofthevolunt<rynatureofstandards,themeaningofISOspecifictermsandexpressionsrelatedtoconformityassessment,aswellasinformationaboutISO'sadherencetotheWorldTradeOrganization(WTO)principlesintheTechnicalBarrierstoTrade(TBT),seewww.iso.org/iso/fbreword.html.ThisdocumentwaspreparedbyJointTechnicalCommitteeISO/IECJTC!,InformationTechnology,SubcommitteeSC42,ArtificialIntelligence.Anyfeedbackorquestionsonthisdocumentshouldbedirectedtotheuser,snationalstandardsbody.Acompletelistingofthesebodiescanbefoundatwww.iso.org/members.html.Capriohtm>tatClSO/(EC2C0-AUrightsProvidedbyIHSMarkilundeclicensewithIECNorproduct)aarnetworkingPennaedwithou!IICenSefromHSIntroductionThegoalofthisdocumentistoanalysethefactorsthatcanimpactthetrustworthinessofsystemsprovidingorusingALcalledhereafterartificialintelligence(AI)systems.ThedocumentbrieflysurveystheexistingapproachesthatcansupportorimprovetrustworthinessintechnicalsystemsanddiscussestheirpotentialapplicationtoAlsystems.ThedocumentdiscussespossibleapproachestomitigatingAlsystemvulnerabilitiesthatrelatetotrustworthiness.ThedocumentalsodiscussesapproachestoimprovingthetrustworthinessofAlsystems.Oi>mam<iGrSUoUdWkMiThSUahiiUYiMIconM.hECMim<«IKCMalirvcrniudvituKctMfxIKSInformationtechnology-ArtificialintelligenceOverviewoftrustworthinessinartificialintelligence1 ScopeThisdocumentsurveystopicsrelatedtotrustworthinessinAlsystems,i11cludingthefollowing:一approachestoestablishtrustinAIsystemsthroughtransparency,explainability,controllability,etc.;一engineeringpitfallsandtypicalassociatedthreatsandriskstoAlsystems,alongwithpossiblemitigationtechniquesandmethods;andapproachestoassessandachieveavailability,resiliency,reliability,accuracy,safety,securityandprivacyofAlsystems.ThespecificationoflevelsoftrustworthinessforAlsystemsisoutofthescopeofthisdocument.2 NormativereferencesTherearenonormativereferencesinthisdocument.3 TermsanddefinitionsForthepurposesofthisdocument,thefollowingtermsanddefinitionsapply.ISOandIECmaintainterminologicaldatabasesforuseinstandardizationatthefollowingaddresses:ISOOnlinebrowsingplatform:availableathttps:WWW.iso.org/obD一IECElectropediaiavailableathttp:/www.electropedia.org/3.1accountabilitypropertythatensuresthattheactionsofanentity(3.16)maybetraceduniquelytothatentitySOURCE:ISO/IEC2382:2015,2126250,modifiedTheNotestoentryhavebeenremoved.3.2actorentity(3.16)thatcommunicatesandinteractsSOURCE:ISO/IECTR22417:2017,3.13.3algorithmsetofrulesfortransformingthelogicalrepresentationofdata(3.11)SOURCE:ISO/IEC11557:1992,4.33.4artificialintelligenceAIcapabilityofanengineeredsystem(3.38)toacquire,processandapplyknowledgeandskillsNote1toentry!Knowledgearefacts,information(3.20)andskillsacquiredthroughexperienceoreducation.CapyightntcmatonslOnGlSQGIEG2O2O.-AIIrights1servedC8%Ml3AaM%¾蝴炉盛?%1晟fceremHSNororReale.06l03(,202020.2940MDT3.5assetanythingthathasvalue(3.46)toastakeholder(3.3Z)Note1toentry:Therearemanytypesofassets,including:a) ufnation(3.20);b) software,suchasacomputerprogram;c) physical,suchascomputer;d) services;e) peopleandtheirqualifications,skillsandexperience;andf) intangibles,suchasreputationandimage.SOURCE:ISO/IEC21827:2008,3-4,modifiedInthedefinition,ttheorganization,hasbeenchangedto“aStakeholder”.Note1toentryhasbeenremoved.3.6attributepropertyorcharacteristicofanobjectthatcanbedistinguishedquantitativelyorqualitativelybyhumanorautomatedmeansSOURCE:ISO/IEC/IEEE15939:2017,3.23.7autonomyautonomouscharacteristicofasystem(3.38)govemedbyitsownrulesastheresultofself-learningNote1toentry:Suchsystemsarenotsubjecttoexternalcontrol(3.10)oroversight.3.8 biasfavouritismtowardssomethings,peopleorgroupsoverothers3.9consistencydegreeofuniformity,standardizationandfreedomfromcontradictionamongthedocumentsorpartsofasystem(3.38)orcomponentSOURCE:ISO/IEC21827:2008,3.143.10 controlpurposefulactiononorinaprocess(3.29)tomeetspecifiedobjectivesSOURCE:TEC61800-7-1:2015,3.2,63.11datare-interpretablerepresentationofinfbrmation(3.20)inaformalizedmannersuitableforCommunicationjnterpretationorprocessingNote1toentry:Data(3.11)canbeprocessedbyhumanorautomaticmeans.SOURCE:ISO/IEC2382:2015,2121272,modified-Notes2and3toentryhavebeenremoved.L IOJTkG BbyIHS Marht uOor fcerGC*ih IEC tNo repcductcn or IKmPrking PCEittCd withot liccmr from IIISISO/IEC2020-Allrightsreserved3.12datasubjectindividualaboutwhompersonaldata(3.27)arerecordedSOURCE:ISO5127:2017,3.13.4.01,modified-Note1toentryhasbeenremoved.3.13decisiontreesupen,ised-leamingmodelforwhichinferencecanberepresentedbytraversingoneormoretree-likestructures3.14effectivenessextenttowhichplannedactivitiesarerealizedandplannedresultsachievedSOURCE:ISO9000:2015,3-7.11,modifiedNoteItoentryhasbeenremoved.3.15efficiencyrelationshipbetweentheresultsachievedandtheresourcesusedSOURCE:TSO9000:2015,3.7,103.16entityanyconcreteorabstractthingofinterestSOURCE:ISO/IEC10746-2:2009,6.13.17harminjutyordamagetothehealthofpeopleordamagetopropertyortheenvironmentSOURCEJSO/IECGuide51:2014,3.13.18hazardpotentialsourceofharm(3.1Z)SOURCE:ISO/IECGuide51:2014,3.23.19humanfactorsenvironmental,organizationalandjobfactors,inconjunctionwithcognitivehumancharacteristics,whichinfluencethebehaviourofpersonsororganizations3.20informationmeaningfuldata(3.11)SOURCE:ISO9000:2015,3.8.23.21integritypropertyofprotectingtheaccuracyandcompletenessofassets(3.5)SOURCE:ISO/IEC27000:2018,336,modified-Inthedefinition,"protectingthe"hasbeenaddedbefore"accuracy"and"ofassets"hasbeenaddedafter"completeness”.jCopriohtnenstISO/IEC2020-Al!rightsrescrverM>*d,H5Maitr*½*nMRKChyr*<artM<l*A*Umm*At3.22intendeduseuseinaccordancewithinformation(3.20)providedwithaproductorsystem(3.38)or,intheabsenceofsuchinformation,bygenerallyunderstoodpatterns(3.26)ofusage.SOURCE:ISO/IECGuide51:2014,3.63.23machinelearningMLprocess(3.29)bywhichafunctionalunitimprovesitsperformancebyacquiringnewknowledgeorskillsorbyreorganizingexistingknowledgeorskillsSOURCE:ISO/IEC2382:2015,21237893.24machinelearningmodelmathematicalconstructthatgeneratesaninferenceorprediction,basedoninputdata(3.11)3.25neuralnetworkcomputationalmodelutilizingdistributed,parallellocalprocessingandconsistingofanetworkofsimpleprocessingelementscalledartificialneurons,whichcanexhibitcomplexglobalbehaviourSOURCE:ISO18115-1:2013,8.13.26patternsetoffeaturesandtheirrelationshipsusedtorecognizeanentity(3.16)withinagivencontextSOURCE:ISO/IEC2382:2015,21237983.27personaldatadata(3.11)relatingtoanidentifiedoridentifiableindividualSOURCE:ISo5127:2017,3.1.10.14,modifiedTheadmittedtermsandNotes1and2toentryhavebeenremoved.3.28privacyfreedomfromintrusionintotheprivatelifeoraffairsofanindividualwhenthatintrusionresultsfromundueorillegalgatheringanduseofdata(3.1l)aboutthatindividualSOURCE:ISO/1EC2382:2015,2126263,modified-Notes1and2toentryhavebeenremoved.3.29processsetOfinterrelatedorinteractingactivitiesthatuseinputstodeliveranintendedresultSOURCE:ISO9000:2015,3.4.1,mcxlifiedThenotestoentryhavebeenomitted.3.30reliabilitypropertyofconsistentintendedbehaviourandresultsSOURCE:ISO/IEC270(X):2018,3.553.31 riskeffectofuncertaintyonobjectivesNote1toentry:Aneffectisadeviationfromtheexpected.ltcanbepositive,negativeorbothandcanaddress,createorresultinopportunitiesandthreats(3.39).Note2toentry!Objectivescanhavedifferentaspectsandcategoriesandcanbeappliedatdifferentlevels.Note3toentry:Riskisusuallyexpressedintermsofrisksources,potentialevents,theirconsequencesandtheirlikelihood.SOURCE:ISO31000:2018,3.13.32robotprogrammedactuatedmechanismwithadegreeofautonomy(3.7),movingwithinitsenvironment,toperformintendedtasksNote1toentry:Arobotincludesthecontrol(3.10)systemandinterfaceofthecontrolsystem(3.38).Note2toentry:Theclassificationofrobotintoindustrialrobotorservicerobotisdoneaccordingtoitsintendedapplication.SOURCE:ISO18646-2:2019,3.13.33roboticsscienceandpracticeofdesigning,manufacturingandapplyingrobots(3.32)SOURCE:TSO8373:2012,2.163.34 safetyfreedomfromrisk(3.31)whichisnottolerableSOURCE:ISO/IECGuide51:2014,3.143.35securitydegreetowhichaproductorsystem(3.38)protectsinformation(3.20)anddata(3.1l)sothatpersonsorotherproductsorsystemshavethedegreeofdataaccessappropriatetotheirtypesandlevelsofauthorizationSOURCE:ISO/IEC25010:2011,4.2.63.36 sensitivedatadata(3.1l)withpotentiallyharmfuleffectsintheeventofdisclosureormisuseSOURCErISO5127:2017,3.1.10.163.37stakeholderanyindividual,groupororganizationthatcanaffect,beaffectedbyorperceiveitselftobeaffectedbyadecisionoractivitySOURCE:ISO/IEC38500:2015,2.24CoprichtnentISO/IBC2020-llritsreserveProMgjtyIHSMamtUrKSerSgvthIECNoreproductionornetxngPefmitedthoutlicenseIromIHS3.38systemcombinationofinteractingelementsorganizedtoachieveoneormorestatedpurposesNote1toentry:Asystemissometimesconsideredasaproductorastheservicesitprovides.SOURCE:ISO/IEC/IEEE15288:2015,3.383.39threatpotentialcauseofanunwantedincident,whichmayresultinharm(3.lZ)tosystems(3.38),organizationsorindividuals3.40trainingprocess(3.29)toestablishortoimprovetheparametersofamachinelearningmodel(3.24)basedonamachinelearningalgorithm(3.3)byusingtrainingdata(3.11)3.41trustdegreetowhichauser(3.43)orotherstakeholder(3.3Z)hasconfidencethataproductorsystem(3.38)willbehaveasintendedSOURCE:TSO/IEC25010:2011,4.1.3.2