NEWBlE

LiteRT provides a unified interface to use Neural Processing Units (NPUs) without requesting you to navigate vendor-specific compilers, runtimes, or library dependencies. Using LiteRT for NPU acceleration boosts performance for real-time and large-model inference and minimizes memory copies through zero-copy hardware buffer usage. ## Get Started ### Classical ML models For classical ML models, see the following demo applications. - [Image segmentation Kotlin App](https://github.com/google-ai-edge/litert-samples/tree/main/compiled_model_api/image_segmentation/kotlin_npu): [AOT](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/image_segmentation/kotlin_npu/android) and [on-device(JIT)](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/image_segmentation/kotlin_npu/android_jit) compilation. - [Image segmentation C++ App](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/image_segmentation/c%2B%2B_segmentation): AOT and on-device (JIT) compilation in the same app. ### GenAI models For GenAI models, see the following demos and guide: - [EmbeddingGemma semantic similarity C++ App](https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/semantic_similarity/build_from_source): CPU/GPU/NPU inference. - Guide on [running LLMs using LiteRT-LM](https://developers.google.com/edge/litert/next/litert_lm_npu). ## NPU Vendors LiteRT supports NPU acceleration with the following vendors: ### Google Tensor - Support AOT execution through the `CompiledModel` API. - See [Google Tensor](https://developers.google.com/edge/litert/next/tensor-sdk) for setup details. > [!WARNING] > **Beta:** Google Tensor SDK doesn't yet support on-device (JIT) compilation. ### Qualcomm AI Engine Direct - Support AOT and on-device compilation execution through the `CompiledModel` API. - See [Qualcomm AI Engine Direct](https://developers.google.com/edge/litert/next/qualcomm) for setup details. - See [Unlo