Subjectivity and complexity of facial attractiveness - Nature

According to this hypothesis, a face is judged on average as attractive according to a set of innate rules typical of the human species, which ... Skiptomaincontent Thankyouforvisitingnature.com.YouareusingabrowserversionwithlimitedsupportforCSS.Toobtain thebestexperience,werecommendyouuseamoreuptodatebrowser(orturnoffcompatibilitymodein InternetExplorer).Inthemeantime,toensurecontinuedsupport,wearedisplayingthesitewithoutstyles andJavaScript. Advertisement nature scientificreports articles article Subjectivityandcomplexityoffacialattractiveness DownloadPDF Subjects AppliedmathematicsHumanbehaviour AbstractTheoriginandmeaningoffacialbeautyrepresentalongstandingpuzzle.Despitetheprofuseliteraturedevotedtofacialattractiveness,itsverynature,itsdeterminantsandthenatureofinter-persondifferencesremaincontroversialissues.Herewetacklesuchquestionsproposinganovelexperimentalapproachinwhichhumansubjects,insteadofratingnaturalfaces,areallowedtoefficientlyexploretheface-spaceand“sculpt”theirfavoritevariationofareferencefacialimage.Theresultsrevealthatdifferentsubjectspreferdistinguishableregionsoftheface-space,highlightingtheessentialsubjectivityofthephenomenon.Thedifferentsculptedfacialvectorsexhibitstrongcorrelationsamongpairsoffacialdistances,characterisingtheunderlyinguniversalityandcomplexityofthecognitiveprocesses,andtherelativerelevanceandrobustnessofthedifferentfacialdistances. DownloadPDF IntroductionThenotionsofbodybeautyandharmonyofproportionshavefascinatedscholarsforcenturies.FromtheancientGreekcanons,acountlessnumberofstudieshavefocusedonunfoldingwhatisbehindthebeautyofthefaceandthebody.Nowadaysthenotionoffacialbeautyisafastexpandingfieldinmanydifferentdisciplinesincludingdevelopmentalpsychology,evolutionarybiology,sociology,cognitivescienceandneuroscience1,2,3,4,5.Still,despiteaprofuseandmulti-disciplinaryliterature,questionsliketheverynatureoffacialattractiveness,itsdeterminants,andtheoriginofinter-subjectvariabilityofaestheticcriteria,eludeasatisfactoryunderstanding.Here,werevisitthequestiondrawingconclusionsbasedonanempiricalapproachthroughwhichweallowhumansubjectsto“sculpt”theirfavoritefacialvariationsbynavigatingthesocalledface-spaceandconvergingonspecificattractors,orpreferredregionsintheface-space.Thefaceisthepartofthehumanbodyfromwhichweinferthemostinformationaboutothers,suchas:gender,identity,intentions,emotions,attractiveness,age,orethnicity6,7,8.Inparticular,lookingataface,weareabletoimmediatelyacquireaconsistentimpressionofitsattractiveness.Still,wecouldhaveahardtimeexplainingwhatmakesafaceattractivetous.Asamatteroffact,whichvariablesdetermineattractivenessandtheirinteractionsarestillpoorlyunderstoodissues3.Manyworkshavebeendevotedtoassessingthevalidityofthenaturalselectionhypothesis,orbeautyasa“certificate”ofgoodphenotypiccondition7.Accordingtothishypothesis,afaceisjudgedonaverageasattractiveaccordingtoasetofinnaterulestypicalofthehumanspecies,whichstandoutwithrespecttoothersocialorindividualfactors.Somedegreeofconsensushas,indeed,beenreported9,10,11,12,13.Mostoftheseexperimentsarebasedonthemeasurementofcorrelationsamongnumericalratingsassignedtoasetofnatural(orsynthetic14,15)facialimagesbyratersbelongingtodifferentculturalgroups.Muchworkinthisfieldhasalsobeendevotedtoassessingthecovariationoftheperceivedbeautyofafacewithfacialtraitsthatarebelievedtosignalgoodphenotypiccondition,mainly:facialsymmetry,averagenessandsecondarysexualtraits.Afterdecadesofintenseresearch,theroleplayedbythesetraitsisknowntobelimited:facialbeautyseemstobemorecomplexthansymmetry5,averageness14,16andsecondarysexualtraits7,17.Indeed,ithasbeendocumentedthatcultural,between-personandintra-persondifferencesinfluenceattractivenessperceptioninvariousways4.Asarepresentativeexample,thelinkbetweenmasculinityandattractivenessinmalefacesissubjecttosignificantinter-andintra-subjectdifferences4,5,7,18.Anevolutionaryexplanationisthatexaggeratedmasculinitycouldbeperceivedasdenotingalackofsomepersonalityfacetssuchashonestyorexpressiveness15.Inthiscontext,thesocalledmultiplefitnessormultiplemotivemodel4,11,19proposesthatattractivenessvariesaccordingtoavarietyofmotives,eachoneevokingadifferentabstractattributeofthepersonwhosefaceisevaluated.Ontheotherhand,animpressiveamountofworkiscommittedtotheautomaticfacialbeautyrating.Thisistackledasasupervisedinferenceproblemwhosetrainingdatabaseiscomposedofnaturalfacialimagescodifiedbyvectorsoffacialcoordinatesinface-space3,20,21,alongwith(inter-subjectaveraged)numericalratingsassignedtothembyhumansubjects,tobeinferred.Worksdiffermainlyonthecodificationoffacesintheface-space:fromageometricfacedescription(2Dor3Dspatialcoordinatesofthefaciallandmarks),toadetaileddescriptionofthetextureorluminositydegreesoffreedomthatprovideacuetothefacialshapeindepth(therealsoexistholisticrepresentations,extractinglower-dimensional,non-localinformationfromthefacialimageaccordingtosomecriterion(PrincipalComponenteigenfacesorGaborfilters);orusingrichertechniqueswhichintegrategeometricfromskintexturalandreflectivitycharacteristics).Withtheadventofdeephierarchicalneuralnetworks,therawfacialimageisgivenasaninputtothealgorithm,whichautomaticallyextractstheputativerelevantfeaturesintheinferenceprocess,althoughinahardlyaccessibleway(theblackboxproblem).Thesupervisedinferenceofratingsmayhelptoaddress,albeitindirectly,theimpactofvariousfacialfeaturesonattractiveness.Althoughtherelativerelevanceofdifferentfeatureshasbeendiscussedinvariousarticles,robustconclusionsarelacking3,22,23,24,25,26,27,28.Theresultsabouttherelativerelevanceofthekind(geometric,texturalandholistic)offacialattributestoattractivenessarecontroversialaswell3,29,30,31,32,33.Inanycase,theintegrationofdifferentkindsofvariablesseemstoimprovetheinferenceresults29,34,suggestingthatthesearecomplementarilytakenintoaccountinthecognitiveprocessofattractivenessassessment.Facialbeautyis,hence,probablynotauniversalfunctionofasetoffewfacialproperties,asimplicitlyassumedinmanyreferences,buttheresultofacomplexprocessinwhichmultiplesemanticconcepts,providingcuestopersonalityfacets,areinferred.Theliteratureconcerninginferenceofpersonalitytraitsindicatesthatsuchsemanticconceptsmaybeencodedinglobalcombinationsoffacialfeatures,inacomplexway35.Thismotivatesastudyoffacialbeautybeyondthesubject-averagedrating,focusingontheinter-subjectheterogeneityandontheglobalcombinationsofvariousfacialfeaturesgeneratingsuchadiversity.Insummary,thecomplexityoffacialattractivenessperceptionsofarpreventedasatisfactoryunderstandingofhowattractivenessrelatestovariousfacialelements3,andofthenatureofinter-personaldifferences.Inordertomakeprogress,fromamethodologicalpointofviewitisimportanttohighlightthreekeyfactors.(A)Thepossiblemutualinfluenceamonggeometric,textureanddetailedfeatures36.Evenconsideringtheproblemintermsofgeometricvariablesonly,thepossibleexistenceofinteractionsormutualdependenciesbetweendifferentfacialcomponentsmayinduceavarietyofpossiblepleasantfaces,evenforthesinglesubject.(B)Theundersamplingoftherelevantface-space,duetothemanydifferentprototypesoffacialbeauty14,29.(C)Thesubjectivityofthephenomenon,probablyhinderedbytheuseoftheaveragenumericalbeautyratings.Thecomplexityandrichnessoftheperceptualprocess,suggestedbythemultiple-motivehypothesisandbypreviousworkaboutperceptionofpersonalitydimensions6,37,38,39,eludesadescriptionintermsofaverageratings,aquantitythathasalreadybeenobservedtobeinadequate3.Inlightoftheseconsiderations,wehereaddressthephenomenonoffacialpreferencethroughanempiricalapproachthataimsatremovingthebiasesofratings,focusinginsteadonthepossibilitygiventohumansubjectstofreelyexploreasuitablydefinedface-space.Bymeansofadedicatedsoftware,basedonimagedeformationandgeneticalgorithms,wefocusoninter-subjectdifferencesinaestheticcriterionandletseveralsubjectssculpttheirfavoritevariationofareferenceportrait,parametrizedbyavectorofgeometricfacialcoordinates.Weobservehowdifferentsubjectstendtosystematicallysculptfacialvectorsindifferentregionsoftheface-space,whichwecallattractors,pointingtowardsastrongsubjectivityintheperceptionoffacialbeauty.Inaddition,thefacialvectorssculptedbydifferentsubjectsexhibitstrongcorrelationsforpairsoffacialdistances,whichisamanifestationoftheunderlyinguniversalityandcomplexityofthecognitiveprocessoffacialimagediscrimination.Thecorrelationscontaininformationregardingthedifferentsourcesofvariabilityinthedatasetofselectedvectors.Forinstance,thoughadifferencebetweenmale-femalesubjectsisclearlyobserved,thelargestdifferencesamongfacialvariations,elicitedbyaprincipalcomponentanalysis,resultfromcriteriathataretransversalwithrespecttothegenderonly.Athirdimportantresultconcernstheassessmentoftherobustnessoftheresultswithrespecttothedegreesoffreedomnotdescribedintheface-space.Crucially,inourapproach,theluminance,textureanddetaileddegreesoffreedomaredecoupledfromthegeometricfeaturesdefiningtheface-space,anddeliberatelykeptfixed,andcommonforallthesubjects.Finally,weobservethattheoverallexperimentalresultsare,interestingly,partiallyrobustandindependentofthedetaileddegreesoffreedom(thereferenceportrait).Thecurrentexperimentalschemebypassesthethreeconfoundingfactors(A–C)mentionedintheprecedentparagraph.(A)Uncontrolledsourcesofbiasesareabsentinourstudy,sinceallpossiblefacialvariations(giventhereferenceportrait)aredescribedbypointsintheface-space.(B)Inourface-spaceofreduceddimensionalityandunchangedtexturedegreesoffreedomtheundersamplingismitigated,makingpossibleanefficientexplorationofthefacespaceandallowingforanaccuratecharacterisationofthesingle-subjectattractor.(C)Thisallowustofullyaccountforsubjectivity:weareabletoanalysethedifferencesamongdifferentsubject’spreferredfacialmodifications.ResultsPreferredfacialimagesasextremainface-spaceWeconsideraface-spacedefinedbyasetofgeometriccoordinatesillustratedinFig. 1A.Afaceisparametrizedintermsofasetof10non-redundantCartesiancoordinatesof7singlelandmarks\({\overrightarrow{\ell}}_{\alpha}=\)(xα,yα)or,alternatively,intermsofavectorofD = 11inter-landmarkdistances\({\bf{d}}={({d}_{i})}_{i=1}^{D}\).Theface-spacevectorcomponentsfiare,inthisway,eitherlandmarkCartesiancoordinatesorinter-landmarkdistances.Fromavectoroffacialcoordinatesfandareferencefacialportraitcorrespondingtoarealperson,wethenconstructafacialimagebyacontinuousdeformationofthereferenceportraitsuchthatitslandmarkgeometriccoordinatesacquirethedesiredvalue,f(Fig. 1B,C).Withinasingleexperiment,thereferenceportrait(theimagetexture)isunchangedandonlythegeometricpositionofthelandmarkscanchange(foranin-depthexplanationseeSec.MethodsandtheSupplementaryInformation).Figure1(A)Theparametersdefiningthefacespace.Theredpointsindicatethelandmarks,α = 1,…,18,whose2DvaryingCartesiancoordinatesgeneratethecontinuumoffacespace.ThefacespacepointsareparametrisedintermsofvectorsfwhosecomponentsaretheCartesiancoordinatesofasetofnon-redundantlandmarks\({\overrightarrow{\ell}}_{\alpha}\)(signaledwithanemptycircle),orintermsof(verticalorhorizontal)distancesdi(i = 0,…,10)amongsomepairsoflandmarks\({d}_{i}=|{x}_{\alpha(i)}-{x}_{\beta(i)}|\)or\({d}_{i}=|{y}_{\alpha(i)}-{y}_{\beta(i)}|\)(arrows).(B)ReferenceportraitRP1usedinexperimentE1alongwithitscorrespondinglandmarks(inblue).(C)ImagedeformationofRP1accordingtoagivenvectorofinter-landmarkdistancesd:thebluereferenceportraitlandmarksareshifted(leadingtotheredpoints)sothattheirinter-landmarkdistancesared,andthereferenceimage(B)isconsequentlydeformed.(D)ImagedeformationofthereferenceportraitRP2accordingtothesamevectorofdistancesdasin(C).FullsizeimageTheaimoftheexperimentalmethodistoprovideapopulationofNfacialvectors,{f (s,n)}n,withn = 1,…,Nand\({{\bf{f}}}^{(s,n)}\in{{\mathbb{R}}}^{D}\),foreachexperimentalsubject,s.Suchapopulationisconsideredasanempiricalsampleofthesubject’sattractor,ortheface-spaceregionofhis/herpreferredmodificationsofthereferenceportrait.Thismeansthatthesubjectwouldprobabilisticallypreferfacialimagesassociatedwithvectorsthatareclosetotheattractor,ratherthanlocalfluctuationsawayfromit(foraprecisedefinitionseetheSupplementarySection S2).Inourexperimentalscheme,thesubjectdoesnotsculptthepopulationbysuccessivediscriminationamongfacesdifferingbyasinglecoordinate,whichturnsouttobeaninefficientstrategyofface-spaceexploration,butratherthroughtheinteractionwithageneticalgorithm(seesectionsMethods,SupplementarySection S3).Inafirstexperiment(E1),wehaveletS1 = 95subjectssculpttheirfacialvariationsofreferenceportraitRP1(see1-A).Thisresultsinafinalpopulation,\({{\mathscr{S}}}_{1}={\{{{\bf{f}}}^{(s,n)}\}}_{s=1,n=1}^{{S}_{1},N}\)ofN = 28facialvectorsforeachsubject.StartingfromNinitialrandomfacialvectors,theFACEXPLOREsoftwaregeneratespairsoffacialimagesthatarepresentedtothesubject,whoselectstheonethathe/sheprefers.BasedonNleft/rightchoices,ageneticalgorithmproducesasuccessivegenerationofNvectors,inaconstantfeedbackloopofoffspringgenerationandselectionoperatedbythesubject.TheiterationofthisprocessleadstoasequenceofTgenerationsoffacialvectors,eachonemoreadaptedthanthelasttothesubject’sselectioncriteria,eventuallyconvergingtoapseudo-stationaryregimeinwhichthepopulationsaresimilartothemselvesandamongconsecutivegenerations.Figure 2reportstheevolution(versusthegenerationindex,t = 1,…,T = 10)oftheintra-populationdistance,thedistanceamongfaceswithinthesinglepopulationssculptedby10different,randomlychosen,subjectsinE1(seeSupplementarySection S4fordetails).Inthenextsubsection,wediscussthedegreeofreproducibilityofourresultsasafunctionofN,TandS1.Figure2Intra-populationdistanceofthepopulationssculptedbydifferentsubjects(s)asafunctionofthegeneration(t).TheEuclideanmetricsinfacespacehasbeenused(seeSupplementarySec. S4),althoughtheresultsarequalitativelyequalforotherrelevantmetrics.Eachcurvecorrespondstoadifferentsubject(for10randomlychosensubjects).Theuppercurveofjoinedcirclescorrespondstothenullmodelgeneticexperiment,inwhichtheleft/rightchoicesarerandom.FullsizeimageTheintra-populationdistancedecreaseswiththegenerationindex,indicatingthatthepopulationssculptedbysinglesubjectstendtoclusterizeinaregionoftheface-space.Thisclusteringisnotobservedinanullexperimentinwhichtheleft-rightdecisionsaretakenrandomly.Remarkably,adiversityofbehaviorstowardsthepseudo-stationaryregimeisobserved,alreadysignalingdifferencesinthewaytheface-spaceisexplored.Fromnowon,wewillconsiderthefinalpopulationsculptedbythes–thsubject,\({\{{{\bf{f}}}^{(s,n)}\}}_{n=1}^{N}\),asthefinal,T = 10-thgenerationofthesequenceofpopulationssculptedbythissubjectinE1.Inthenextsubsectionweshowthattheface-spaceattractorsofdifferentsubjectsareactuallysignificantlyandconsistentlydifferent.Thisexperimentalschemeis,therefore,abletoresolvethesubjectivecharacterofattractiveness,asthesinglesubjecttendstosculptpopulationsofvectorsclusteredinanarrowregionintheface-spaceinsuccessiverealisationsoftheexperiment.Allthesefactsimplythatthesinglesubjectattractorcanbeoperationallycharacterisedasanextremumofasubject-dependent,probabilisticfunctioninface-space,whichmaybeinferredfromthepopulationssculptedbythesubjectinseveralinstancesoftheexperiment(seeSupplementarySection S2foracompletedefinition).Theattractorsareextremaofsuchafunctioninthesensethatasignificantfluctuationofavectorcoordinateawayfromitsvalueintheattractorwilltendtoloweritsprobabilityofbeingselectedbythesubject,giventhereferenceportrait.Assessmentofsubjectivity:distinguishableaestheticidealsInordertoassessthesubjectivityofthesculptingprocess,weneedtomeasuretowhatextentthesamesubject,byrepeatingthesameexperiment,wouldsculptpopulationsoffacialvectorsclosertoeachotherthantopopulationssculptedbydistinctsubjects.Tothisendweperformedasecondexperiment(E2),inwhichasubsetofSsc = 6subjectswereaskedtoperformm = 6instancesofanexperimentE1,withthecommonreferenceportraitRP1,different(random)initialconditionsandsequenceofrandomnumbersinthegeneticalgorithm.Thesubjectivityisassessedthroughthecomparisonoftwosetsofdistances:(i)the(Sscm(m − 1)/2)self-consistencydistancesamongfacialpopulationssculptedbythesamesubjectindifferentinstancesoftheexperimentE2;(ii)the(S1(S1 − 1)/2)inter-subjectdistancesbetweencouplesofpopulationssculptedbydifferentsubjectsinexperimentE1(seeSupplementarySection S4fordetails).Ifsubjectivitywasatplayinthesculptingprocess,andnothinderedbythestochasticityofthealgorithm,theself-consistencydistanceswouldbelowerthaninter-subjectdistances.Thisisclearlythecase,seeFig. 3:self-consistencydistancesarelowerthaninter-subjectdistances(Student’sp  |σ|/D,allthesubjectsappeartoagreeintheirchoices.Underthisperspective,thereporteduniversalityofbeautycouldbetheside-effectofanexperimentalprocedurewheresubjectsexpresstheirpreferencesamongalimitedsetofpredefinedoptions,therealfacialimages,inahigh-dimensionalface-space(indeed,theeffectivenumberofrelevantfacialdimensionsmaybeoftheorderofhundreds42).Insuchanundersamplingsituation,differentnaturalfacesexhibitverydifferentnumberoffacialcoordinatesgi(or,moreprecisely,ofPC’s,seebefore),closetothemostprobablevalue〈gi〉,withrespecttotheirstandarddeviation(say,σ(gi)).Thefacesexhibitingmanycoordinatesinthecommonlypreferredregionareconsensuallypreferred,andmosthighlyrated20.Bylettingthesubjectssculptinsteadtheirpreferredmodificationinalower-dimensionalfacespace,asinexperimentsE1-2,thesubjectsexcludeextremevaluesofthecoordinates,andmanagetofine-tunethemaccordingtotheirpersonalcriterion.Inthiscircumstance,itispossibletoresolvethesubjects’preferenceswithhigheraccuracy,μsc  0intermsofinter-landmarkdistances)tendtoconsequentlypresentpositiveandnegativefluctuationsofotherfacialcoordinatesyj≠i(e.g.,ahighermouth,y4 > 0).Thesignandmagnitudeofsuchcovariationsisgivenbythecorrelationmatrixamongfluctuationsoffacialcoordinates.Thisisthepositivedefinite,symmetricmatrix\({C}_{ij}=\langle{y}_{i}{y}_{j}\rangle\),averagedoversubjects\(\langle\cdot\rangle={\sum}_{s=1}^{S}\cdot\,/S\).Inordertosubtracttheinfluenceofcorrelationswithinthesingle-subjectattractor,onlyonepopulationvector,ofindexnb(s),uncorrelatedandrandomlydistributed,isconsideredforeachsubjects;theaverageandstandarddeviationofthematrixelementsCijhavebeenobtainedfrommanybootstrappingrealisations,labelledbyb,oftheindicesnb(s),seeSupplementarySection S4.TheexperimentalmatrixCexhibitsaproliferationofnon-zeroelements(32%ofthematrixelementspresentingap-value  〈y9〉),the9–12anglecanberestoredonlybyincreasingthex12-coordinate,x12 > 〈x12〉.FullsizeimageThesefindingsindicatethat,forameaningfulinferenceoftheperceivedattractivenessinface-space,oneshouldconsidertheimpactofatleastlinearcombinationsoffacialcoordinates,ratherthantheimpactofsinglefacialcoordinates.Theintrinsiccomplexityofattractivenessperceptioncannotbesatisfactorilyinferredthroughasimpleregressionoffacialdatasetsusingasumoffunctionsofsinglefacialcoordinates(seealsoSupplementarySection S14and43).Relevanceoffacialfeatures:thevariablehierarchyInthissectionwediscusstherobustnessoftheresultspresentedabove.Oneofthecrucialquestionsinfacialattractivenessiswhatistherelevantsetofvariableswhichmainlydeterminetheperceivedattractivenessofaface3,36.Aformulationoftheproblemintheoretical-informationtermsisthatoffindingahierarchyofrelevantfacialfeatures.Itissuchthat,whenenrichingthedescriptionwithmorevariablesinhighlevelsofthehierarchy,theresultingvariablesinlowerlevelsresultunchanged.Inthepresentstudy,thegeometricquantitiescanbeconsideredaslow-levelvariablesintheextenttowhichtheyarenotinfluencedbythereferenceportrait,orbytheluminanceandtexturefacialfeaturesthathavebeendisregardedandkeptunchangedintheface-spacedescription.Tosettlethisquestion,weperformedathirdexperiment,dubbedE3,inwhichweaskedtheS1participantsinE1torepeattheexperimentusingadifferentreferenceportrait(RP2,seeFig. 1D).Afterwards,wehavecomparedtheresultingsetofsculptedfacialvectors,\({{\mathscr{S}}}_{3}\),withtheoutcomeofexperimentE1,\({{\mathscr{S}}}_{1}\).Interestingly,astatisticalt-testshowsthat,whilesomefacialcoordinatesresultclearlydistinguishable,othersresultstatisticallyindistinguishable,signifyingtheirrobustnesswithrespecttothetexturefacialfeaturesdeterminedbythereferenceportrait.Theseare,intermsofinter-landmarkdistances,di,thecoordinatesd2,6,7,10,indistinguishablewithp > 0.1(seeSupplementaryFig. S6).If,insteadoffocusingonthedistributionofsinglequantitiesyi,oneconsidersinsteadthecorrelations,yiyj,theresults(seeSupplementaryTable S4)turnouttoberobustwithintheirstatisticalerrors,sinceonly2%ofthematrixelementsCijresultsignificantlydistinguishable(p