From-ShaderX-2-–-Shader-Programming-Tips-and-Trick.docx

AdvancedImageProcessingwithDirectX9PixelShadersJasonL.Mitchell,MarwanY.AnsariandEvanHart3DApplicationResearchGroupATIResearchIntroductionWiththeintroductionoftheps_2_0pixelshadermodelinDirectX9.0,weareabletosignificantlyexpandourabilitytouseconsumergraphicshardwaretoperformimageprocessingoperations.Thisisduetothelongerprogramlength,theabilitytosamplemoretimesfromtheinputimage(s)andtheadditionoffloatingpointinternaldatarepresentation.InthefirstShaderXbook,weusedtheps_l_4pixelshadermodelinDirectX8.1toperformbasicimageprocessingtechniquessuchassimpleblurs,edgedetection,transferfunctionsandmorphologicaloperatorsMitchell02.Inthischapter,wewillextendourimageprocessingtoolboxtoincludecolorspaceconversion,abetteredgedetectionfiltercalledtheCannyfilter,separableGaussianandmedianfilters,andareal-timeimplementationoftheFastFourierTransform.ReviewAsshowninouroriginalimageprocessingchapterinthefirstShaderXbook,post-processingof3Dframesisfundamentaltoproducingavarietyofinterestingeffectsingamescenes.ImageprocessingisperformedonaGPUbyusingthesourceimageasatextureanddrawingascreen-alignedquadrilateralintothebackbufferoranothertexture.Apixelshaderisusedtoprocesstheinputimagetoproducethedesiredresultintherendertarget.Figure 1 - Using a pixel shader for image processing by rendering from one image to anotherOutputImageImageprocessingisespeciallypowerfulwhenthecolorofthedestinationpixelistheresultofcomputationsdoneonmultiplepixelsfromthesourceimage.Inthiscase,wesamplethesourceimagemultipletimesandusethepixelshadertocombinethedatafromthemultiplesamples(ortaps)toproduceasingleoutput.ColorSpaceConversionBeforewegetintointerestingmulti-tapfilters,we,llpresentapairofshaderswhichcanbeusedtoconvertbetweenHSVandRGBcolorspaces.Theseshadersperformsomerelativelycomplexoperationstoconvertbetweencolorspaceseventhoughtheyareonlysingle-tapfilters.ForthosewhomaynotbefamiliarwithHSVspace,itisacolorspacewhichisdesignedtobeintuitivetoartistswhothinkofacolor'stint,shadeandtoneSmith78.InterpolationinthiscolorspacecanbemoreaestheticallypleasingthaninterpolationinRGBspace.Additionally,whencomparingcolors,itmaybedesirabletodosoinHSVspace.Forexample,inRGBspace,thecolor100,0,0)isverydifferentfromthecolor0,0,100.However,theirVcomponentsinHSVspaceareequal.Colors,representedbyhue,saturation,valuetriplesaredefinedtoliewithinahexagonalpyramidasshowninFigure2below.Thehueofacolorisrepresentedbyananglebetween0°and360oaroundthecentralaxisofthehexagonalcone.Acolor,ssaturationisthedistancefromthecentral(achromatic)axisanditsvalueisthedistancealongtheaxis.Bothsaturationandvaluearedefinedtobebetween0and1.WehavetranslatedthepseudocodeRGB-to-HSVtransformationfromFoley90totheDirectX9HighLevelShadingLanguage(HLSL)andcompileditfortheps_2_0target.IfyouareunfamiliarwithHLSL,youcanreferbacktotheintroductorychapter"'IntroductiontotheDirectX®9HighLevelShadingLanguage."AsdescribedinSmith79,youcanseethattheRGB_to_HSV()functioninthisshaderfirstdeterminestheminimumandmaximumchannelsoftheinputRGBcolor.ThemaxchanneldeterminesthevalueoftheHSVcolor,orhowfaralongtheachromaticcentralaxisofthehexagonalconetheHSVcolorwillbe.ThesaturationisthencomputedasthedifferencebetweenthemaxandminRGBchannelsdividedbythemax.Hue(theanglearoundthecentralachromaticaxis)isthenafunctionofwhichchannelhadthemaxmagnitudeandthusdeterminedthevalue.float4RGB_to_HSV(float4color)(floatrzg,b,delta;floatColorMaxzcolorMin;floath=0,s=0,v=0;float4hsv=0;r=color0;g=color1;b=color2;ColorMax=max(rzg);ColorMax=max(colorMax,b);colorMin=min(rzg);colorMin=min(colorMin,b);v=colorMax;/thisisvalueif(colorMax!=O)(s=(colorMax-colorMin)/colorMax;if(s!=O)/ifnotachromatic(delta=colorMax-colorMin;if(r=colorMax)h=(g-b)/delta;elseif(g=colorMax)Ih=2.0+(b-r)/delta;else/bismaxIh=4.0+(r-g)/delta;h*=60;if(h<0)(h+=360;hsv0=h/360.0;/movinghtobebetween0and1.hsv1=s;hsv2=v;returnhsv;TheHSV-to-RGBtransformation,alsotranslatedfromFoley901,isshownbelowinHLSL.float4HSV_to_RGB(float4hsv)(float4color=0;floatfzpzq,t;floath,s,v;floatr=0,g=0,b=0;floati;if(hsvl=0)(if(hsv2!=0)(color=hsv2;else(h=hsv.x*360.0;s=hsv.y;v=hsv.z;if(h=360.0)(h=0;)h/=60;i=floor(h);f=h-i;p=v*(1.0-s);q=V*t=V*(1.0-(s(1.0-(s*f)；*(1.0-f);if(i=0)(r=v;g=t;b=)P；elseif(i=1)r=q；g=v;b=)P；elseif(i=2)r=P；g=v;b=)t;elseif(i=3)r=P；g=q；b=V；elseifI(i=4)r=t;g=P；b=V；elseif(i=5)r=v;g=P；b=)q；color.r=r;color.g=g；color.b)=b;returncoloOtherColorSpacesItisworthnotingthatRGBandHSVarenottheonlycolorspacesofinterestincomputergraphics.Forexample,theoriginalpaperSmith78whichintroducedHSValsointroducedacolorspacecalledHSL(forhue,saturationandlightness)whereLisoftenthesameastheLuminance(Y)channelusedintheYIQcolorspace.Ifyouareinterestedinlearningmoreaboutcolorspaces,Smith78andFoley90bothprovideexcellentdiscussions.NowthatWehaveintroducedsomereasonablyadvancedsingle-tapimageoperationsforconvertingbetweencolorspaces,Wewilldiscussafewmulti-tapfilterswhichperformsomesophisticatedimageprocessingoperations.AdvancedEdgeDetectionInthefirstShaderXbook,wediscussedtheRobertsandSobeledgedetectionfiltersMitchell02.Here,wewillexpanduponthosefiltersandintroduceanimplementationoftheCannyedgedetectionfilter.Step-by-StepApproachAsoutlinedinJain95,theCannyedgedetectionfiltercanbeimplementedbyperformingthefollowingoperations:1) ApplyaGaussianblur2) Computethepartialderivativesateachtexel3) ComputetheMagnitudeanddirectionoftheline(tan,)ateachpoint4) Sampletheneighborsinthedirectionofthelineandperformnonmaxima-suppression.Naturally,wewillimplementthisinaseriesofsteps,eachusingadifferentshadertooperateontheoutputfromtheprecedingstep.AGaussianbluristhefirstshaderrunovertheinputimage.Thisisdonetoeliminateanyhighfrequencynoiseintheinputimage.Variousfilterkernelsizescanbeusedforthisstep.Thenextstepintheprocessiscomputationofthepartialderivatives(PandQ)intheuandvdirectionsrespectively:Partial u KernelPartial v KernelThenthemagnitudeofthederivativeiscomputedusingthestandardformula:Magnitude=7P2+Q2Finally,thePandQvaluesareusedtodeterminethedirectionoftheedgeatthattexelusingthestandardequation:6>=atan2(,P)MagnitudeandarewrittenouttoanimagesothatthenextshadercanusethemtocompletetheCannyfilteroperation.Theedgedirection,yisasignedquantityintherangeof-toandmustbepackedintothe0to1rangeinordertopreventlossofdatabetweenrenderingpasses.Inordertodothis,wewillpackitbycomputing:A=abs(8)/You'veprobablynoticedthat,duetotheabsolutevalue,thisfunctionisnotinvertible,hencedataiseffectivelylost.Thisdoesnotpresentaproblemforthisparticularapplicationduetosymmetriesinthefollowingstep.ThefinalpassinvolvessamplingtheimagetogettheMagnitudeandtheedgedirection,8,atthecurrentlocation.Theedgedirection,mustnowbeunpackedintoitsproperrange.Figure3belowshowsapartitioningofallvaluesof(indegrees)intofoursectors.Figure3-The360degreesofananglepartitionedintofoursectorsThesectorsaresymmetricandmaptothepossiblewaysalinecanpassthrougha3×3setofpixels.Inthepreviousstep,wetooktheabsolutevalueofanddivideditbytoputitinthe0to1range.Sinceweknowthatisalreadybetween0and1fromthepreviousstep,wearealmostdone.Sincethepartitioningissymmetric,itwasanexcellentwaytoreducethenumberofcomparisonsneededtofindthecorrectneighborstosample.Normally,tocompletethemappingwewouldmultiplyAby4andwewouldbedone.However,ifyoulookcloselyatFigure3youwillthatthesectorsarecenteredaround0and18().Inordertocompensateforthis,theproperequationis:Sector=floor(A-16)*4)Next,wecomputetheneighboringtexelcoordinatesbycheckingwhichsectorthisedgegoesthrough.Nowthattheneighborshavebeensampled,wecomparethecurrenttexesmagnitudetothemagnitudesofitsneighbors.Ifitsmagnitudeisgreaterthanbothofitsneighbors,thenitisthelocalmaximumandthevalueiskept.Ifitsmagnitudeislessthaneitherofitsneighbors,thenthistexesvalueissettozero.Thisprocessisknownasnontnaximasuppression,anditsgoalistothintheareasofchangesothatonlythegreatestlocalchangesareretained.Asafinalstep,wecanthresholdtheimageinordertoreducethenumberfalseedgesthatmightbepickedupbythisprocess.Thethresholdisoftensetbytheuserwhenheorshefindstherightbalancebetweentrueandfalseedges.Figure4-One-Pixel-WideEdgesfromCannyFilterFigure5-GradientMagnitudesfromSobelFilter(seeMitchell02)AsyoucanseeinFigure4,theCannyfilterproducesonepixelwideedgesunlikemorebasicfilterssuchasaSobeledgefilter.ImplementationDetailsThisshaderisimplementedintheVideoShaderapplicationontheCDusingHLSLandcanbecompiledfortheps_2_0targetorhigher.Inthisimplementation,thesamplesaretakenfromtheeightneighborsadjacenttothecenterofthefilter.LookingattheHLSLcode,you,llseeanarrayoffloattwo-tuplescalledsampleoffsets.Thisarraydefinesasetof2Doffsetsfromthecentertapwhichareusedtodeterminethelocationsfromwhichtosampletheinputimage.ThelocationsofthesesamplesrelativetothecentertapareshowninFigure6.Figure6-LocationsoftapsasdefinedinsampleoffsetsThefourstepsoftheCannyedgedetectionfilterdescribedabovehavebeencollapsedintotworenderingpasses,requiringthetwoshadersshownbelow.ThefirstshadercomputesthegradientsPandQfollowedbytheMagnitudeanddirection().Afterpacking0intothe0to1range,Magnitudeand0arewrittenouttoatemporarysurface.samplerInputimage;float2sampleoffsets8:register(cl);structPS_INPUT(float2texCoord:TEXCOORDO;float4main(PS_INPUTIn):COLOR(inti=0;float4result;floatMagnitude,Theta;floatp=0zq=0;floatpKernel4=-l,1,-1,1);floatqKernel4=-1,-1,1,1);float2texCoords4;float3texSamples4;floatPI=3.1415926535897932384626433832795;texCoords(0=In.texCoord+samleffsets1;texCoords1=In.texCoord+SampleOffsets2;texCoords2=In.texCoord;texCoords3=In.texCoord+SampleOffsets4;for(i=0;i<4;i+)(texSamplesi.xyz=tex2D(Inputimage,texCoordsi);texSamplesi=dot(texSamplesi,0.33333333f);p+=texSamplesi*pKerneli;q+=texSamplesi*qKernel(i;)=2.0;q/=2.0;sqrt(p*p)+(q*q)；result=Magnitude;directionofthe/linetoprepforNonmaximasupression./,ttheMax,/makeit0(hence,supressit)Theta=atan2(qzp);/resultisis0to1/Justsoitcanbewrittenout.returnresult;InthesecondpassoftheCannyedgedetector,Magnitudeandarereadbackfromthetemporarysurface.Theedgedirection,仇isclassifiedintooneoffoursectorsandtheneighborsalongtheproperdirectionaresampledusingdependentreads.TheMagnitudesoftheseneighborsamplesalongwithauser-definedthresholdarethenusedtodeterminewhetherthispixelisalocalmaximumornot,resultingineither0or1beingoutputasthefinalresult.samplerInputimage;fl8:register(cl);float4Userinput:register(c24);structPS_INPUT(rd：Texcoordo；;flt4main(PS_INPUTIn):COLOR(float4result;floatMagnitude,Theta;float2LexCoords(4;float4texSamples3;floatPI=3.1415926535897932384626433832795;TapthecurrenttexelandfigureoutlinedirectiontexSamples0=tex2D(Inputimage,In.texCoord);Magnitude=texSamples0.r;/Sampletwoneighborsthatlieinthedirectionoftheline/Thenfindoutif_thistexelhasagreaterMagnitude.Theta=texSamples0.a;/Mustunpacktheta.PriorpassmadeThetarangebetween0and1/Butwereallywantittobeeither0,1,2,or4.SeeJain95/formoredetails.Theta=(Theta-PI/1Theta=floor(Theta);/NowthetaisanINT.texCoords2In.texCoord In.texCoordsamplef fsets (4 ;sampleoffsets3; eltexCoords2.texCoord In.texCoordsampleffsets2;sampleffsets5;) el.LexCoordtexCoords2In.texCoordSampleOffsets1;sampleoffsets6;) elexCoordtexCoords2In.texCoordsampleffsets0;sampleoffsets7;Takeothertwosamples/Remembertheyareinthedirectionoftheedgefor(i=l;i<3;i+)(=tex2D(Inputimage,texCoordsi);)Nowit,stimeforNonmaximasupression./Nonmaximasupression-IfthistexelisnttheMax,/makeit0(hence,supressit)/Thiskeepstheedgesniceandresult=Magnitude;if ( Magnitude < texStude < texSamples2.x )Thresholdtheresult.if(result.x<Userinput.z)elseresult=1;YoucanseeinFigure4thatthisproducesone-pixel-wideedges,whichmaybemoredesirableforsomeapplications.Youmayseesomegapsinthedetectededgesand,insomecases,itmaybeusefultoapplyadilationoperationtofillinthesegapsMitchell02.SeparableTechniquesCertainfilteringoperationshaveinherentsymmetrywhichallowsustoimplementthemmoreefficientlyinaseparablemanner.Thatis,wecanperformthese2DimageprocessingoperationswithasequenceofIDoperationsandobtainequivalentresultswithlesscomputation.Conversely,wecanimplementalargeseparablefilterkernelwiththesameamountofcomputationasasmallnon-separablefilter.Thisisparticularlyimportantwhenattemptingtoapply“blooms“tofinalframesinhighdynamicrangespacetosimulatelightscattering.Inthisfinalsectionofthechapter,wewilldiscussthreeseparablefilteringoperations:theGaussianblur,amedianfilterapproximationandtheFastFourierTransform.SeparableGaussianAverycommonly-usedseparablefilteristheGaussianfilter,whichcanbeusedtoperformblurringof2Dimages.The2Disotropic(i.e.circularlysymmetric)Gaussianfilter,g2D(x,y),samplesacircularneighborhoodofpixelsfromtheinputimageandcomputestheirweightedaverage,accordingtothefollowingequation:13gflD(X、y)=2c202whereisthestandarddeviationoftheGaussianandxandyarethecoordinatesofimagesamplesrelativetothecenterofthefilter.Thestandarddeviation,determinesthesizeofthefilter.Whatthismeansisthatwewillsamplealocalareaoftexelsfromtheinputimageandweightthemaccordingtotheaboveequation.Forexample,foraGaussianwith=1,Wecomputethefollowingfilterkernel(afternormalization).0.00370.01460.02560.01460.00370.01460.05860.09520.05860.01460.02560.09520.15020.09520.02560.01460.05860.09520.05860.01460.00370.01460.02560.01460.0037Intheory,theGaussianhasinfiniteextent,butthecontributiontothefinalresultisinsignificantforinputtexelsoutsideofthis5×5region.AnextremelyimportantpropertyoftheGaussianisthatitisseparable.Thatis,itcanberearrangedinthefollowingmanner:giD(%)gs(y)ThismeansthatwecanimplementagivenGaussianwithaseriesof1Dfilteringoperations:onehorizontal(gm(x)andoneverticald()0)ThisallowsustoimplementGaussianswithmuchlargerkernels(larger)whileperformingthesameamountofcalculationsthatwouldberequiredtoimplementasmallernon-separablefilterkernel.Thistechniquewasusedinourreal-timeimplementationofPaulDcbevec,sRenderingwithNaturalLightanimationasseeninFigure7.Figure7-FramefromReal-TimeRenderingwithNaturalLightAfterrenderingthesceneinhighdynamicrangespace,DebevecperformedanumberoflargeGaussianblursonhis2Drenderedscenetoobtainbloomsonbrightareasofthescene.Inordertodothisinreal-time,WeexploitedtheGaussian,sseparabilitytoperformtheoperationefficiently.Inourcase,weused=7,whichresultedina25×25Gaussian