{"id":56,"date":"2023-05-01T13:45:46","date_gmt":"2023-05-01T06:45:46","guid":{"rendered":"https:\/\/viet.im\/blog\/?p=56"},"modified":"2023-05-11T14:32:14","modified_gmt":"2023-05-11T07:32:14","slug":"testing-stable-diffusion-with-ncnn-framework-in-termux-on-android","status":"publish","type":"post","link":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/","title":{"rendered":"Testing Stable Diffusion with ncnn framework in Termux on Android"},"content":{"rendered":"

From the Github repository Tencent\/ncnn<\/a>:<\/p>\n

ncnn<\/em> is a high-performance neural network inference computing framework optimized for mobile platforms. ncnn<\/em> is deeply considerate about deployment and uses on mobile phones from the beginning of design. ncnn<\/em> does not have third party dependencies.<\/p><\/blockquote>\n

Another very good thing is that the ncnn<\/em> framework supports GPU acceleration via Vulkan API. ncnn<\/em> was added to the official repository of Termux (PR #15976<\/a>) and can be easily installed like other packages.<\/p>\n

There is an implementation of Stable Diffusion with ncnn<\/em> framework by EdVince\/Stable-Diffusion-NCNN<\/a>. I’m going to use the fork of that repository by fengwang\/Stable-Diffusion-NCNN<\/a> to test the image generation capability.<\/p>\n

The device (rate my setup) to be used is my ZTE nubia Red Magic 6R (Snapdragon 888 chipset with 8GB RAM). Below is the workflow:<\/p>\n

1. Install necessary packages<\/strong><\/p>\n

apt install clang curl git glslang libncnn pkg-config -y<\/pre>\n
2. Clone the repo<\/strong><\/p>\n
git clone https:\/\/github.com\/fengwang\/Stable-Diffusion-NCNN\r\ncd Stable-Diffusion-NCNN<\/pre>\n
3. Download and extract the model<\/strong><\/p>\n
curl https:\/\/github.com\/fengwang\/Stable-Diffusion-NCNN\/releases\/download\/release\/assets.20221204.tar.zst --output assets.20221204.tar.zst\r\ntar --use-compress-program=unzstd -xvf assets.20221204.tar.zst<\/pre>\n
At this point you should have a sub-folder named assets<\/em> with these files:<\/p>\n
assets\r\n\u251c\u2500\u2500 AutoencoderKL-fp16.bin\r\n\u251c\u2500\u2500 AutoencoderKL-fp16.param\r\n\u251c\u2500\u2500 FrozenCLIPEmbedder-fp16.bin\r\n\u251c\u2500\u2500 FrozenCLIPEmbedder-fp16.param\r\n\u251c\u2500\u2500 log_sigmas.bin\r\n\u251c\u2500\u2500 RealESRGAN_x4plus_anime_6B.fp32-sim-sim-opt.bin\r\n\u251c\u2500\u2500 RealESRGAN_x4plus_anime_6B.fp32-sim-sim-opt.param\r\n\u251c\u2500\u2500 UNetModel-fp16.bin\r\n\u251c\u2500\u2500 UNetModel-fp16.param\r\n\u2514\u2500\u2500 vocab.txt<\/pre>\n
4. Compile the code<\/strong><\/p>\n
g++ -o test test.cpp -funsafe-math-optimizations -Ofast -flto=auto -pipe -march=native -std=c++20 -Wall -Wextra `pkg-config --cflags --libs ncnn` -lstdc++ -pthread -Wl,--gc-sections -flto -lvulkan -lglslang -lSPIRV -fopenmp<\/pre>\n
By default the original resolution of the generated image will be 512×512 then it will be upscaled 4x to 2048×2048 using xinntao\/Real-ESRGAN<\/a>.<\/p>\n
For my phone with limited RAM amount, I have to modify the file stable_diffusion.hpp<\/code> to disable upscaling (replacing from line #825 to #833 with below code):<\/p>\n
\/\/x_samples_ddim = esr4x( x_samples_ddim );\r\nstd::cout << \"----------------[save]--------------------\" << std::endl;\r\n{\r\n    std::vector<std::uint8_t> buffer;\r\n    buffer.resize( 512 * 512 * 3 );\r\n    x_samples_ddim.to_pixels( buffer.data(), ncnn::Mat::PIXEL_RGB );\r\n    save_png( buffer.data(), 512, 512, 0, output_png_path.c_str() );\r\n}<\/pre>\n
5. Run the code<\/strong><\/p>\n
.\/text<\/pre>\n
Here is one sample result:
\n\"Stable<\/a><\/p>\n
My phone took around 13 minutes to generate a image (and of course it was screaming a lot with the heat also). I consider this as very good, as the phone was able to load the model and generate a 512×512 image, while still spared some little RAM for other apps to not get killed.<\/p>\n
\"\" <\/a>\"\"<\/a><\/p>\n
Unfortunately at the moment there is no easy way to use different models with ncnn<\/em> according to this <\/a> (That’s why I put “Testing” in the title of the post). Hope that in the future there will be someone who can figure this out.<\/p>\n","protected":false},"excerpt":{"rendered":"
From the Github repository Tencent\/ncnn: ncnn is a high-performance neural network inference computing framework optimized for mobile platforms. ncnn is deeply considerate about deployment and uses on mobile phones from the beginning of design. ncnn does not have third party dependencies. Another very good thing is that the ncnn framework supports GPU acceleration via Vulkan API. ncnn was added to...<\/p>\n","protected":false},"author":1,"featured_media":70,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"cybocfi_hide_featured_image":"yes","_jetpack_memberships_contains_paid_content":false,"footnotes":"","_wp_rev_ctl_limit":""},"categories":[1],"tags":[],"class_list":["post-56","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"yoast_head":"\nTesting Stable Diffusion with ncnn framework in Termux on Android - Viet's Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Testing Stable Diffusion with ncnn framework in Termux on Android - Viet's Blog\" \/>\n<meta property=\"og:description\" content=\"From the Github repository Tencent\/ncnn: ncnn is a high-performance neural network inference computing framework optimized for mobile platforms. ncnn is deeply considerate about deployment and uses on mobile phones from the beginning of design. ncnn does not have third party dependencies. Another very good thing is that the ncnn framework supports GPU acceleration via Vulkan API. ncnn was added to...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/\" \/>\n<meta property=\"og:site_name\" content=\"Viet's Blog\" \/>\n<meta property=\"article:published_time\" content=\"2023-05-01T06:45:46+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-05-11T07:32:14+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/viet.im\/blog\/wp-content\/uploads\/2023\/05\/ncnn_2.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1080\" \/>\n\t<meta property=\"og:image:height\" content=\"2400\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Viet\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Viet\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\\\/\"},\"author\":{\"name\":\"Viet\",\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/#\\\/schema\\\/person\\\/0f7d8f880ca4892bf08c50b02c35809b\"},\"headline\":\"Testing Stable Diffusion with ncnn framework in Termux on Android\",\"datePublished\":\"2023-05-01T06:45:46+00:00\",\"dateModified\":\"2023-05-11T07:32:14+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\\\/\"},\"wordCount\":318,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/viet.im\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/05\\\/ncnn_2.png\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/viet.im\\\/blog\\\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\\\/\",\"url\":\"https:\\\/\\\/viet.im\\\/blog\\\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\\\/\",\"name\":\"Testing Stable Diffusion with ncnn framework in Termux on Android - Viet's Blog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/viet.im\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/05\\\/ncnn_2.png\",\"datePublished\":\"2023-05-01T06:45:46+00:00\",\"dateModified\":\"2023-05-11T07:32:14+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/#\\\/schema\\\/person\\\/0f7d8f880ca4892bf08c50b02c35809b\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/viet.im\\\/blog\\\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/viet.im\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Testing Stable Diffusion with ncnn framework in Termux on Android\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/viet.im\\\/blog\\\/\",\"name\":\"Viet's Blog\",\"description\":\"-\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/viet.im\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/viet.im\\\/blog\\\/#\\\/schema\\\/person\\\/0f7d8f880ca4892bf08c50b02c35809b\",\"name\":\"Viet\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/36c308454c1e52331326d520483707bdd317dca24a5b16686d04023eaeb1240d?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/36c308454c1e52331326d520483707bdd317dca24a5b16686d04023eaeb1240d?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/36c308454c1e52331326d520483707bdd317dca24a5b16686d04023eaeb1240d?s=96&d=mm&r=g\",\"caption\":\"Viet\"},\"sameAs\":[\"https:\\\/\\\/viet.im\\\/blog\"],\"url\":\"https:\\\/\\\/viet.im\\\/blog\\\/author\\\/viet\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Testing Stable Diffusion with ncnn framework in Termux on Android - Viet's Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/","og_locale":"en_US","og_type":"article","og_title":"Testing Stable Diffusion with ncnn framework in Termux on Android - Viet's Blog","og_description":"From the Github repository Tencent\/ncnn: ncnn is a high-performance neural network inference computing framework optimized for mobile platforms. ncnn is deeply considerate about deployment and uses on mobile phones from the beginning of design. ncnn does not have third party dependencies. Another very good thing is that the ncnn framework supports GPU acceleration via Vulkan API. ncnn was added to...","og_url":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/","og_site_name":"Viet's Blog","article_published_time":"2023-05-01T06:45:46+00:00","article_modified_time":"2023-05-11T07:32:14+00:00","og_image":[{"width":1080,"height":2400,"url":"https:\/\/viet.im\/blog\/wp-content\/uploads\/2023\/05\/ncnn_2.png","type":"image\/png"}],"author":"Viet","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Viet","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/#article","isPartOf":{"@id":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/"},"author":{"name":"Viet","@id":"https:\/\/viet.im\/blog\/#\/schema\/person\/0f7d8f880ca4892bf08c50b02c35809b"},"headline":"Testing Stable Diffusion with ncnn framework in Termux on Android","datePublished":"2023-05-01T06:45:46+00:00","dateModified":"2023-05-11T07:32:14+00:00","mainEntityOfPage":{"@id":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/"},"wordCount":318,"commentCount":0,"image":{"@id":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/#primaryimage"},"thumbnailUrl":"https:\/\/viet.im\/blog\/wp-content\/uploads\/2023\/05\/ncnn_2.png","inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/","url":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/","name":"Testing Stable Diffusion with ncnn framework in Termux on Android - Viet's Blog","isPartOf":{"@id":"https:\/\/viet.im\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/#primaryimage"},"image":{"@id":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/#primaryimage"},"thumbnailUrl":"https:\/\/viet.im\/blog\/wp-content\/uploads\/2023\/05\/ncnn_2.png","datePublished":"2023-05-01T06:45:46+00:00","dateModified":"2023-05-11T07:32:14+00:00","author":{"@id":"https:\/\/viet.im\/blog\/#\/schema\/person\/0f7d8f880ca4892bf08c50b02c35809b"},"breadcrumb":{"@id":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/viet.im\/blog\/testing-stable-diffusion-with-ncnn-framework-in-termux-on-android\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/viet.im\/blog\/"},{"@type":"ListItem","position":2,"name":"Testing Stable Diffusion with ncnn framework in Termux on Android"}]},{"@type":"WebSite","@id":"https:\/\/viet.im\/blog\/#website","url":"https:\/\/viet.im\/blog\/","name":"Viet's Blog","description":"-","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/viet.im\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/viet.im\/blog\/#\/schema\/person\/0f7d8f880ca4892bf08c50b02c35809b","name":"Viet","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/36c308454c1e52331326d520483707bdd317dca24a5b16686d04023eaeb1240d?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/36c308454c1e52331326d520483707bdd317dca24a5b16686d04023eaeb1240d?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/36c308454c1e52331326d520483707bdd317dca24a5b16686d04023eaeb1240d?s=96&d=mm&r=g","caption":"Viet"},"sameAs":["https:\/\/viet.im\/blog"],"url":"https:\/\/viet.im\/blog\/author\/viet\/"}]}},"jetpack_featured_media_url":"https:\/\/viet.im\/blog\/wp-content\/uploads\/2023\/05\/ncnn_2.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/viet.im\/blog\/wp-json\/wp\/v2\/posts\/56","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/viet.im\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/viet.im\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/viet.im\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/viet.im\/blog\/wp-json\/wp\/v2\/comments?post=56"}],"version-history":[{"count":5,"href":"https:\/\/viet.im\/blog\/wp-json\/wp\/v2\/posts\/56\/revisions"}],"predecessor-version":[{"id":76,"href":"https:\/\/viet.im\/blog\/wp-json\/wp\/v2\/posts\/56\/revisions\/76"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/viet.im\/blog\/wp-json\/wp\/v2\/media\/70"}],"wp:attachment":[{"href":"https:\/\/viet.im\/blog\/wp-json\/wp\/v2\/media?parent=56"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/viet.im\/blog\/wp-json\/wp\/v2\/categories?post=56"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/viet.im\/blog\/wp-json\/wp\/v2\/tags?post=56"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}