IBM/MAX-Image-Resolution-Enhancer - GitHub

This repository contains code to instantiate and deploy an image resolution enhancer. This model is able to upscale a pixelated image by a factor of 4, ... Skiptocontent {{message}} IBM / MAX-Image-Resolution-Enhancer Public Notifications Star 551 Fork 76 Upscaleanimagebyafactorof4,whilegeneratingphoto-realisticdetails. developer.ibm.com/exchanges/models/all/max-image-resolution-enhancer/ Apache-2.0License 551 stars 76 forks Star Notifications Code Issues 5 Pullrequests 4 Actions Projects 0 Wiki Security Insights More Code Issues Pullrequests Actions Projects Wiki Security Insights master Branches Tags Couldnotloadbranches Nothingtoshow Loading {{refName}} default Couldnotloadtags Nothingtoshow {{refName}} default Loading 4 branches 3 tags Code Loading Latestcommit xuhdev Mergepullrequest#48fromIBM/tensorflow-2 … b1f179c Aug20,2021 Mergepullrequest#48fromIBM/tensorflow-2 Upgradetensorflowtov2andMAX-Basetov1.5.1 b1f179c Gitstats 70 commits Files Permalink Failedtoloadlatestcommitinformation. Type Name Latestcommitmessage Committime api Upgradetensorflowtov2andMAX-Basetov1.5.1 Aug20,2021 core Upgradetensorflowtov2andMAX-Basetov1.5.1 Aug20,2021 docs Replacesampleimage(#15) Aug26,2019 samples SeparatethesamplesandassetsandupdatetheREADME(#10) Jun17,2019 tests SeparatethesamplesandassetsandupdatetheREADME(#10) Jun17,2019 .dockerignore Variousimprovements(#5) Apr24,2019 .gitignore add.idea/togitignore Feb19,2020 .travis.yml Update.travis.yml Dec7,2020 Dockerfile Upgradetensorflowtov2andMAX-Basetov1.5.1 Aug20,2021 LICENSE Variousimprovements(#5) Apr24,2019 README.md AddAVXprereq Oct20,2020 app.py Upgradetensorflowtov2andMAX-Basetov1.5.1 Aug20,2021 config.py Upgradetensorflowtov2andMAX-Basetov1.5.1 Aug20,2021 max-image-resolution-enhancer.yaml switchtoquay.io Sep15,2020 requirements-test.txt Locktestrequirementversions Dec13,2020 requirements.txt Upgradetensorflowtov2andMAX-Basetov1.5.1 Aug20,2021 sha512sums.txt addbandit May26,2020 Viewcode IBMDeveloperModelAssetExchange:ImageResolutionEnhancer ModelMetadata Benchmark References Licenses Pre-requisites: Deploymentoptions DeployfromQuay DeployonRedHatOpenShift DeployonKubernetes RunLocally 1.BuildtheModel 2.DeploytheModel 3.UsetheModel 4.Development 5.Cleanup Troubleshooting ResourcesandContributions README.md IBMDeveloperModelAssetExchange:ImageResolutionEnhancer Thisrepositorycontainscodetoinstantiateanddeployanimageresolutionenhancer. Thismodelisabletoupscaleapixelatedimagebyafactorof4,whilegeneratingphoto-realisticdetails. TheGANisbasedonthisGitHubrepositoryandonthisresearcharticle. Themodelwastrainedon600,000imagesoftheOpenImagesV4dataset,andthemodelfilesarehostedon IBMCloudObjectStorage. ThecodeinthisrepositorydeploysthemodelasawebserviceinaDockercontainer.Thisrepositorywasdeveloped aspartoftheIBMDeveloperModelAssetExchangeandthepublicAPIispoweredbyIBMCloud. ModelMetadata Domain Application Industry Framework TrainingData InputDataFormat Vision Super-Resolution General TensorFlow OpenImagesV4 Image(RGB/HWC) Benchmark Set5 Author'sSRGAN ThisSRGAN PSNR 29.40 29.56 SSIM 0.85 0.85 Set14 Author'sSRGAN ThisSRGAN PSNR 26.02 26.25 SSIM 0.74 0.72 BSD100 Author'sSRGAN ThisSRGAN PSNR 25.16 24.4 SSIM 0.67 0.67 Theperformanceofthisimplementationwasevaluatedonthreedatasets:Set5,Set14,andBSD100. ThePSNR(peaksignaltonoiseratio)andSSIM(structuralsimilarityindex)metricswereevaluated,althoughthepaperdiscusses theMOS(meanopinionscore)asthemostfavorablemetric.Inessence,theSRGANimplementationtradesabetterPSNRorSSIMscoreforaresultmoreappealingtothehumaneye.Thisleadstoacollectionofoutputimageswithmorecrispandrealisticdetails. NOTE:TheSRGANinthepaperwastrainedon350kImageNetsamples,whereasthisSRGANwastrainedon600kOpenImagesV4pictures. References C.Ledig,L.Theis,F.Huszar,J.Caballero,A.Cunningham,A.Acosta,A.Aitken,A.Tejani,J.Totz,Z.Wang,W.Shi,Photo-RealisticSingleImageSuper-ResolutionUsingaGenerativeAdversarialNetwork,ArXiv,2017. SRGAN-tensorflow(modelcodesource) tensorflow-SRGAN DeconvolutionandCheckerboardArtefacts Licenses Component License Link Thisrepository Apache2.0 LICENSE ModelWeights Apache2.0 LICENSE ModelCode(3rdparty) MIT LICENSE Testsamples CCBY2.0 AssetREADME CC0 AssetREADME Pre-requisites: docker:TheDockercommand-lineinterface.Followtheinstallationinstructionsforyoursystem. Theminimumrecommendedresourcesforthismodelis8GBMemory(seeTroubleshooting)and4CPUs. Ifyouareonx86-64/AMD64,yourCPUmustsupportAVXattheminimum. Deploymentoptions DeployfromQuay DeployonRedHatOpenShift DeployonKubernetes RunLocally DeployfromQuay Torunthedockerimage,whichautomaticallystartsthemodelservingAPI,run: $dockerrun-it-p5000:5000quay.io/codait/max-image-resolution-enhancer Thiswillpullapre-builtimagefromtheQuay.iocontainerregistry(oruseanexistingimageifalreadycachedlocally)andrunit. Ifyou'drathercheckoutandbuildthemodellocallyyoucanfollowtherunlocallystepsbelow. DeployonRedHatOpenShift Youcandeploythemodel-servingmicroserviceonRedHatOpenShiftbyfollowingtheinstructionsfortheOpenShiftwebconsoleortheOpenShiftContainerPlatformCLIinthistutorial,specifyingquay.io/codait/max-image-resolution-enhancerastheimagename. DeployonKubernetes YoucanalsodeploythemodelonKubernetesusingthelatestdockerimageonQuay. OnyourKubernetescluster,runthefollowingcommands: $kubectlapply-fhttps://github.com/IBM/max-image-resolution-enhancer/raw/master/max-image-resolution-enhancer.yaml Themodelwillbeavailableinternallyatport5000,butcanalsobeaccessedexternallythroughtheNodePort. AmoreelaboratetutorialonhowtodeploythisMAXmodeltoproductiononIBMCloudcanbefoundhere. RunLocally BuildtheModel DeploytheModel UsetheModel Development Cleanup 1.BuildtheModel Clonethisrepositorylocally.Inaterminal,runthefollowingcommand: $gitclonehttps://github.com/IBM/max-image-resolution-enhancer.git Changedirectoryintotherepositorybasefolder: $cdmax-image-resolution-enhancer Tobuildthedockerimagelocally,run: $dockerbuild-tmax-image-resolution-enhancer. Allrequiredmodelassetswillbedownloadedduringthebuildprocess.NotethatcurrentlythisdockerimageisCPUonly(wewilladdsupportforGPUimageslater). 2.DeploytheModel Torunthedockerimage,whichautomaticallystartsthemodelservingAPI,run: $dockerrun-it-p5000:5000max-image-resolution-enhancer 3.UsetheModel TheAPIserverautomaticallygeneratesaninteractiveSwaggerdocumentationpage.Gotohttp://localhost:5000toloadit.FromthereyoucanexploretheAPIandalsocreatetestrequests. Usethemodel/predictendpointtoloadatestimage(youcanuseoneofthetestimagesfromthesamples/test_examples/low_resolutionfolder)inordertogetahighresolutionoutputimagereturned. TheidealinputimageisaPNGfilewitharesolutionbetween100x100and500x500,preferablywithoutanypost-captureprocessingandflashycolors.Themodelisabletogeneratedetailsfromapixelatedimage(lowDPI),butisnotabletocorrecta'blurred'image. Left:inputimage(128×80).Right:outputimage(512×320) Youcanalsotestitonthecommandline,forexample: $curl-F"image=@samples/test_examples/low_resolution/food.png"-XPOSThttp://localhost:5000/model/predict>food_high_res.png Theabovecommandwillsendthelowresolutionfood.pngfiletothemodel,andsavethehighresolutionoutputimagetothefood_high_res.pngfileintherootdirectory. 4.Development ToruntheFlaskAPIappindebugmode,editconfig.pytosetDEBUG=Trueundertheapplicationsettings.Youwillthenneedtorebuildthedockerimage(seestep1). PleaseremembertosetDEBUG=Falsewhenrunningthemodelinproduction. 5.Cleanup TostoptheDockercontainer,typeCTRL+Cinyourterminal. Troubleshooting Callingthemodel/predictendpointkillstheDockercontainerwiththemessageKilled ThisislikelycausedduetothedefaultlimitationofDocker'smemoryallocationto2GB.NavigatetothePreferencesmenuundertheDockerDesktopapplication.Usetheslidertoincreasetheavailablememoryto8GBandrestartDockerDesktop. Thedetailsintheoutputimagearedifferentthanwhatmaybeexpectedandaresometimesnotphysicallypossible Thismodelgeneratesdetailsbasically'outofthinair'.Creatingsomethingoutofnothingisnotpossiblewithoutmakingassumptions. Thenetworkattemptstorecognizeelementsinthelow-resolutionimagefromwhichitcaninferwhatthereality(humaneye|super-resolution)couldhavelookedlike.Ifagroupofpixelsstronglyresemblesanobservationthatisnotrelatedtothecontentoftheimage,itmightleadtoobservingresultsthatarenot'physicallypossible'. Forexample:awhitepixelinalow-resolutionimagemighthavebeenconvertedtoasnowflake,althoughtheoriginalpicturemighthavebeentakeninthedesert.Thisexampleisimaginaryandhasnotactuallybeenobserved. Artefactsintheoutputimage Observingartefactsinsomeimagesisunfortunatelyinevitableasanyneuralnetworkissubjecttotechnicallimitationsandcharacteristicsofthetrainingdata. Keepinmindthatthebestresultsareachievedwiththefollowing: APNGimage Animagethatissufficientlyzoomedin.Duringtheprocess,thenetworkgroupsablockofpixelstogether.Iftheblockcontainsmoredetailsthanthenetworkproduces,theresultwillbespurious. Animagetakenundernaturallight,withoutfilters,andwithfewbrightorflashycolors.Theneuralnetworkwasnottrainedonheavilyeditedimages. Animagethathassufficientlyhighresolutiontonotconfusethenetworkwithmultiplepossibilities(e.g.asolepixelinaverylow-resolutionimagecouldrepresentanentirecar,person,sandwich,..) Themodelisabletogeneratedetailsfromapixelatedimage(lowDPI),butisnotabletocorrecta'blurred'image. ResourcesandContributions IfyouareinterestedincontributingtotheModelAssetExchangeprojectorhaveanyqueries,pleasefollowtheinstructionshere. About Upscaleanimagebyafactorof4,whilegeneratingphoto-realisticdetails. developer.ibm.com/exchanges/models/all/max-image-resolution-enhancer/ Topics machine-learning ai computer-vision neural-network tensorflow docker-image ibm machine-learning-models codait Resources Readme License Apache-2.0License Releases 3 v1.1.0 Latest Jun18,2019 +2releases Packages0 Nopackagespublished Contributors9 Languages Python 98.0% Dockerfile 2.0% Youcan’tperformthatactionatthistime. Yousignedinwithanothertaborwindow.Reloadtorefreshyoursession. Yousignedoutinanothertaborwindow.Reloadtorefreshyoursession.